Commit Graph

562 Commits

Author SHA1 Message Date
KN4CK3R 2bb32006fd
Test if LFS object is accessible (#16865)
* Test if object is accessible.

* Added more logging.
2021-08-31 14:35:08 +01:00
zeripath 06b9d553bc
Timeout on flush in testing (#16864)
* Timeout on flush in testing

At the end of each test the queues are flushed. At present there is no limit on the
length of time a flush can take which can lead to long flushes.

However, if the CI task is cancelled we lose the log information as to where the long
flush was taking place.

This PR simply adds a default time limit of 2 minutes - at which point an error will
be produced. This should allow us to more easily find the culprit.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* return better error

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-08-30 00:27:51 -04:00
Kyle Evans cad70599a6
Refactor the fork service slightly to take ForkRepoOptions (#16744)
* Refactor the fork service slightly to take ForkRepoOptions

This reduces the number of places we need to change if we want to add other
options during fork time.

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Fix integrations and tests after ForkRepository refactor

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* Update OldRepo -> BaseRepo

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>

* gofmt pass

Signed-off-by: Kyle Evans <kevans@FreeBSD.org>
2021-08-28 16:37:14 +08:00
zeripath cd8db3a83d
Prevent "Race" detected in TestAdmin*User (#16830)
These tests are missing the defer prefix.

Related #1441

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-08-27 15:00:37 +02:00
Jimmy Praet a4962a9440
Add filter by owner and team to issue/pulls search endpoint (#16662)
* Filter by owner and team in API issue/pulls search

* Add integration test
2021-08-13 22:47:25 +02:00
6543 2289580bb7
[API] generalize list header (#16551)
* Add info about list endpoints to CONTRIBUTING.md

* Let all list endpoints return X-Total-Count header 

* Add TODOs for GetCombinedCommitStatusByRef

* Fix models/issue_stopwatch.go

* Rrefactor models.ListDeployKeys

* Introduce helper func and use them for SetLinkHeader related func
2021-08-12 14:43:08 +02:00
nitul1991 2d25b7d44b
Add an api endpoint to fetch git notes (#15373) (#16649)
close #15373
2021-08-11 03:01:40 +02:00
a1012112796 3705168837
Add agit flow support in gitea (#14295)
* feature: add agit flow support

ref: https://git-repo.info/en/2020/03/agit-flow-and-git-repo/

example:

```Bash
git checkout -b test
echo "test" >> README.md
git commit -m "test"
git push origin HEAD:refs/for/master -o topic=test
```

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix lint

* simplify code add fix some nits

* update merge help message

* Apply suggestions from code review. Thanks @jiangxin

* add forced-update message

* fix lint

* splite writePktLine

* add refs/for/<target-branch>/<topic-branch> support also

* Add test code add fix api

* fix lint

* fix test

* skip test if git version < 2.29

* try test with git 2.30.1

* fix permission check bug

* fix some nit

* logic implify and test code update

* fix bug

* apply suggestions from code review

* prepare for merge

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fix permission check bug

- test code update
- apply suggestions from code review @zeripath

Signed-off-by: a1012112796 <1012112796@qq.com>

* fix bug when target branch isn't exist

* prevent some special push and fix some nits

* fix lint

* try splite

* Apply suggestions from code review

- fix permission check
- handle user rename

* fix version negotiation

* remane

* fix template

* handle empty repo

* ui: fix  branch link under the title

* fix nits

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-07-28 17:42:56 +08:00
Lunny Xiao 9f31f3aa8a
Add an abstract json layout to make it's easier to change json library (#16528)
* Add an abstract json layout to make it's easier to change json library

* Fix import

* Fix import sequence

* Fix blank lines

* Fix blank lines
2021-07-24 18:03:58 +02:00
zeripath 5d2e11eedb
Refactor: Move login out of models (#16199)
`models` does far too much. In particular it handles all `UserSignin`.

It shouldn't be responsible for calling LDAP, SMTP or PAM for signing in.

Therefore we should move this code out of `models`.

This code has to depend on `models` - therefore it belongs in `services`.

There is a package in `services` called `auth` and clearly this functionality belongs in there.

Plan:

- [x] Change `auth.Auth` to `auth.Method` - as they represent methods of authentication.
- [x] Move `models.UserSignIn` into `auth`
- [x] Move `models.ExternalUserLogin`
- [x] Move most of the `LoginVia*` methods to `auth` or subpackages
- [x] Move Resynchronize functionality to `auth`
  - Involved some restructuring of `models/ssh_key.go` to reduce the size of this massive file and simplify its files.
- [x] Move the rest of the LDAP functionality in to the ldap subpackage
- [x] Re-factor the login sources to express an interfaces `auth.Source`?
  - I've done this through some smaller interfaces Authenticator and Synchronizable - which would allow us to extend things in future
- [x] Now LDAP is out of models - need to think about modules/auth/ldap and I think all of that functionality might just be moveable
- [x] Similarly a lot Oauth2 functionality need not be in models too and should be moved to services/auth/source/oauth2
  - [x] modules/auth/oauth2/oauth2.go uses xorm... This is naughty - probably need to move this into models.
  - [x] models/oauth2.go - mostly should be in modules/auth/oauth2 or services/auth/source/oauth2 
- [x] More simplifications of login_source.go may need to be done
- Allow wiring in of notify registration -  *this can now easily be done - but I think we should do it in another PR*  - see #16178 
- More refactors...?
  - OpenID should probably become an auth Method but I think that can be left for another PR
  - Methods should also probably be cleaned up  - again another PR I think.
  - SSPI still needs more refactors.* Rename auth.Auth auth.Method
* Restructure ssh_key.go

- move functions from models/user.go that relate to ssh_key to ssh_key
- split ssh_key.go to try create clearer function domains for allow for
future refactors here.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-24 11:16:34 +01:00
6543 b26c3b482f
Add TestPrepareWikiFileName (#16487)
* Add TestPrepareWikiFileName

* use LsTree as LsFiles is index only

* ajust other tests

Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-07-20 15:16:20 +02:00
zeripath 2635778425
Add basic edit ldap auth test & actually fix #16252 (#16465)
One of the reasons why #16447 was needed and why #16268 was needed in
the first place was because it appears that editing ldap configuration
doesn't get tested.

This PR therefore adds a basic test that will run the edit pipeline.

In doing so it's now clear that #16447 and #16268 aren't actually
solving #16252. It turns out that what actually happens is that is that
the bytes are actually double encoded.

This PR now changes the json unmarshal wrapper to handle this double
encode.

Fix #16252

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: 6543 <6543@obermui.de>
2021-07-20 14:30:22 +03:00
Richard Nienaber 908136c557
add configuration option to restrict users by default (#16256)
* add configuration option to restrict users by default

* default IsRestricted permission only set on sign up

setting this in the model messes with other workflows (e.g. syncing LDAP users) where the IsRestricted permission needs to be explicitly set and not overridden by a config value

* fix formatting

* Apply suggestions from code review

* ensure newly created user is set to restricted

* ensure imports are in the correct order

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-07-15 15:19:48 -04:00
zeripath 3dcb3e9073
Second attempt at preventing zombies (#16326)
* Second attempt at preventing zombies

* Ensure that the pipes are closed in ssh.go
* Ensure that a cancellable context is passed up in cmd/* http requests
* Make cmd.fail return properly so defers are obeyed
* Ensure that something is sent to stdout in case of blocks here

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint 2

Signed-off-by: Andrew Thornton <art27@cantab.net>

* placate lint 3

Signed-off-by: Andrew Thornton <art27@cantab.net>

* fixup

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Apply suggestions from code review

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-07-14 10:43:13 -04:00
zeripath b82293270c
Add option to provide signature for a token to verify key ownership (#14054)
* Add option to provide signed token to verify key ownership

Currently we will only allow a key to be matched to a user if it matches
an activated email address. This PR provides a different mechanism - if
the user provides a signature for automatically generated token (based
on the timestamp, user creation time, user ID, username and primary
email.

* Ensure verified keys can act for all active emails for the user

* Add code to mark keys as verified

* Slight UI adjustments

* Slight UI adjustments 2

* Simplify signature verification slightly

* fix postgres test

* add api routes

* handle swapped primary-keys

* Verify the no-reply address for verified keys

* Only add email addresses that are activated to keys

* Fix committer shortcut properly

* Restructure gpg_keys.go

* Use common Verification Token code

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-13 15:28:07 +02:00
6543 b81106be3f
Let branch/tag name be a valid ref to get CI status (#16400)
* fix #16384#

* refactor: move shared helper func to utils package

* extend Tests

* use ctx.Repo.GitRepo if not nil
2021-07-13 08:14:14 +01:00
zeripath d06f9ce274
Redirect on bad CSRF instead of presenting bad page (#14937)
The current CSRF handler is a bit harsh with bad CSRF tokens on webpages
I think we can be a little kinder and redirect to base page with a flash error

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-07-08 15:57:24 +02:00
luzpaz fc1607b368
Fix source typos (#16374)
* Fix source typos

Follow up to e0296b6a6  
Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`

* rm "ignore destory on `make misspell-check`"

Co-authored-by: 6543 <6543@obermui.de>
2021-07-08 14:55:09 +02:00
luzpaz e0296b6a6d
Fix various documentation, user-facing, and source comment typos (#16367)
* Fix various doc, user-facing, and source comment typos

Found via `codespell -q 3 -S ./options/locale,./vendor -L ba,pullrequest,pullrequests,readby`
2021-07-08 13:38:13 +02:00
a1012112796 5bb97a12d7
Creating a repo from a template repo via API (#15958)
* Creating a repo from a template repo via API

fix #15934
ref:
https://docs.github.com/en/rest/reference/repos#create-a-repository-using-a-template

Signed-off-by: a1012112796 <1012112796@qq.com>
2021-07-05 17:29:08 +02:00
sebastian-sauer 92328a3394
Add API to get commits of PR (#16300)
* Add API to get commits of PR

fixes #10918

Co-authored-by: Andrew Bezold <andrew.bezold@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-07-02 14:19:57 +02:00
Jimmy Praet ce286f9d9c
Support custom mime type mapping for text files (#16304)
* Support custom mime type mapping for text files

* Apply suggested change to routers/common/repo.go

Co-authored-by: KN4CK3R <admin@oldschoolhack.me>

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
2021-06-30 17:31:54 -04:00
Sergey Dryabzhinsky 22a0636544
Add Visible modes function from Organisation to Users too (#16069)
You can limit or hide organisations. This pull make it also posible for users

- new strings to translte
- add checkbox to user profile form
- add checkbox to admin user.edit form
- filter explore page user search
- filter api admin and public user searches
- allow admins view "hidden" users
- add app option DEFAULT_USER_VISIBILITY
- rewrite many files to use Visibility field
- check for teams intersection
- fix context output
- right fake 404 if not visible

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-06-26 20:53:14 +01:00
Lunny Xiao e3c626834b
Let package git depend on setting but not opposite (#15241)
* Let package git depend on setting but not opposite

* private some package variables
2021-06-26 13:28:55 +02:00
siddweiker f573e93ed4
Fix heatmap activity (#15252)
* Group heatmap actions by 15 minute intervals

Signed-off-by: Sidd Weiker <siddweiker@gmail.com>

* Add multi-contribution test for user heatmap

Signed-off-by: Sidd Weiker <siddweiker@gmail.com>

* Add timezone aware summation for activity heatmap

Signed-off-by: Sidd Weiker <siddweiker@gmail.com>

* Fix api user heatmap test

Signed-off-by: Sidd Weiker <siddweiker@gmail.com>

* Update variable declaration style

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-25 12:59:25 -04:00
KN4CK3R 44b8b07631
Add tag protection (#15629)
* Added tag protection in hook.

* Prevent UI tag creation if protected.

* Added settings page.

* Added tests.

* Added suggestions.

* Moved tests.

* Use individual errors.

* Removed unneeded methods.

* Switched delete selector.

* Changed method names.

* No reason to be unique.

* Allow editing of protected tags.

* Removed unique key from migration.

* Added docs page.

* Changed date.

* Respond with 404 to not found tags.

* Replaced glob with regex pattern.

* Added support for glob and regex pattern.

* Updated documentation.

* Changed white* to allow*.

* Fixed edit button link.

* Added cancel button.

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-06-25 16:28:55 +02:00
sebastian-sauer 6c3433151f
API: Allow COMMENT reviews to not specify a body (#16229)
* Allow COMMENT reviews to not specify a body

when using web ui there is no need to specify a body.
so we don't need to specify a body if adding a COMMENT-review
via our api.

* Ensure comments or Body is provided

and add some integration tests for reviewtype COMMENT.

Signed-off-by: Sebastian Sauer <sauer.sebastian@gmail.com>
2021-06-25 00:05:51 +02:00
Lunny Xiao b223d36195
Rework repository archive (#14723)
* Use storage to store archive files

* Fix backend lint

* Add archiver table on database

* Finish archive download

* Fix test

* Add database migrations

* Add status for archiver

* Fix lint

* Add queue

* Add doctor to check and delete old archives

* Improve archive queue

* Fix tests

* improve archive storage

* Delete repo archives

* Add missing fixture

* fix fixture

* Fix fixture

* Fix test

* Fix archiver cleaning

* Fix bug

* Add docs for repository archive storage

* remove repo-archive configuration

* Fix test

* Fix test

* Fix lint

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 17:12:38 -04:00
6543 eb324a9402
[API] Add repoGetTag (#16166)
* GetTag -> GetAnnotatedTag

* API: Add repoGetTag

* fix swagger docs

* support "/" as tag name char

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-23 17:08:47 -04:00
zeripath 681e81babd
reqOrgMembership calls need to be preceded by reqToken (#16198)
ReqOrgMembership calls need to be preceded by reqToken

Fix #16192

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: 6543 <6543@obermui.de>
2021-06-21 14:01:44 +01:00
6543 889dea8fc1
Remove User.GetOrganizations() (#14032)
as title
2021-06-18 19:00:53 +02:00
6543 d5be9ddc97
fix TestAPIRepoTransfer (#16189) 2021-06-18 16:46:22 +02:00
6543 75205b5669
Fix some API bugs (#16184)
* Repository object only count releases as releases (fix #16144)

* EditOrg respect RepoAdminChangeTeamAccess option (fix #16013)
2021-06-18 01:24:55 +02:00
6543 f7cd394680
[API] Add repoCreateTag (#16165)
* Add API CreateTag

* Add Test

* API: expose Tag Message
2021-06-17 18:04:10 +02:00
6543 b3fbd37e99
[API] expose repo.GetReviewers() & repo.GetAssignees() (#16168)
* API: expose repo.GetReviewers() & repo.GetAssignees()

* Add tests

* fix unrelated swagger query type
2021-06-17 16:02:34 +02:00
6543 6ad5d0a306
[API] ListReleases add filter for draft and pre-releases (#16175)
* invent ctx.QueryOptionalBool

* [API] ListReleases add draft and pre-release filter

* Add X-Total-Count header

* Add a release to fixtures

* Add TEST for API ListReleases
2021-06-17 10:58:10 +02:00
6543 9469e14dc6
[API] Issue Search Add filter for MilestoneNames (#16173) 2021-06-17 08:40:59 +02:00
6543 0e081ff0ce
[API] ListIssues add more filters (#16174)
* [API] ListIssues add more filters:
optional filter repo issues by:
 - since
 - before
 - created_by
 - assigned_by
 - mentioned_by

* Add Tests

* Update routers/api/v1/repo/issue.go

Co-authored-by: Lanre Adelowo <adelowomailbox@gmail.com>

* Apply suggestions from code review

Co-authored-by: Lanre Adelowo <adelowomailbox@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-06-16 18:33:37 -04:00
KN4CK3R 440039c0cc
Add push to remote mirror repository (#15157)
* Added push mirror model.

* Integrated push mirror into queue.

* Moved methods into own file.

* Added basic implementation.

* Mirror wiki too.

* Removed duplicated method.

* Get url for different remotes.

* Added migration.

* Unified remote url access.

* Add/Remove push mirror remotes.

* Prevent hangs with missing credentials.

* Moved code between files.

* Changed sanitizer interface.

* Added push mirror backend methods.

* Only update the mirror remote.

* Limit refs on push.

* Added UI part.

* Added missing table.

* Delete mirror if repository gets removed.

* Changed signature. Handle object errors.

* Added upload method.

* Added "upload" unit tests.

* Added transfer adapter unit tests.

* Send correct headers.

* Added pushing of LFS objects.

* Added more logging.

* Simpler body handling.

* Process files in batches to reduce HTTP calls.

* Added created timestamp.

* Fixed invalid column name.

* Changed name to prevent xorm auto setting.

* Remove table header im empty.

* Strip exit code from error message.

* Added docs page about mirroring.

* Fixed date.

* Fixed merge errors.

* Moved test to integrations.

* Added push mirror test.

* Added test.
2021-06-14 19:20:43 +02:00
Lunny Xiao da057996d5
Fix http path bug (#16117)
* Fix http path bug

* Add missed request

* add tests

Co-authored-by: 6543 <6543@obermui.de>
2021-06-09 14:53:12 +02:00
Lunny Xiao 1bfb0a24d8
Refactor routers directory (#15800)
* refactor routers directory

* move func used for web and api to common

* make corsHandler a function to prohibit side efects

* rm unused func

Co-authored-by: 6543 <6543@obermui.de>
2021-06-09 01:33:54 +02:00
Lunny Xiao b9d611e917
Always store primary email address into email_address table and also the state (#15956)
* Always store primary email address into email_address table and also the state

* Add lower_email to not convert email to lower as what's added

* Fix fixture

* Fix tests

* Use BeforeInsert to save lower email

* Fix v180 migration

* fix tests

* Fix test

* Remove wrong submited codes

* Fix test

* Fix test

* Fix test

* Add test for v181 migration

* remove change user's email to lower

* Revert change on user's email column

* Fix lower email

* Fix test

* Fix test
2021-06-08 11:52:51 +08:00
KN4CK3R 3607f79d78
Fixed assert statements. (#16089) 2021-06-07 07:27:09 +02:00
KN4CK3R ee5e1c4a88
Rewrite of the LFS server (#15523)
* Restructured code. Moved static checks out of loop.

* Restructured batch api. Add support for individual errors.

* Let router decide if LFS is enabled.

* Renamed methods.

* Return correct status from verify handler.

* Unified media type check in router.

* Changed error code according to spec.

* Moved checks into router.

* Removed invalid v1 api methods.

* Unified methods.

* Display better error messages.

* Added size parameter. Create meta object on upload.

* Use object error on invalid size.

* Skip upload if object exists.

* Moved methods.

* Suppress fields in response.

* Changed error on accept.

* Added tests.

* Use ErrorResponse object.

* Test against message property.

* Add support for the old invalid lfs client.

* Fixed the check because MinIO wraps the error.

* Use individual repositories.

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-06-06 02:59:27 +03:00
Lunny Xiao effad26c0e
Improve assets handler middleware (#15961)
* Use route to serve assets but not middleware

* Fix build error with bindata tag

* convert path to absolute

* fix build

* reduce function stack

* Add tests for assets

* Remove test for assets because they are not generated

* Use a http function to serve assets

* Still use middleware to serve assets then less middleware stack for assets

* Move serveContent to original position

* remove unnecessary blank line change

* Fix bug for /assets* requests

* clean code

Co-authored-by: zeripath <art27@cantab.net>
2021-05-30 18:25:11 +08:00
Jimmy Praet c9480c5f60
Add links to toggle WIP status (#14677)
* Add links to toggle PR WIP status

* Allow PR author to toggle WIP status

* refactors and restyling, remove links from translations

Co-authored-by: Norwin <noerw@users.noreply.github.com>
Co-authored-by: silverwind <me@silverwind.io>
2021-05-27 22:02:04 +02:00
Patrick Schratz d957a1569a
Bump `postgres` and `mysql` versions (#15710)
* bump postgres and mysql DB versions

* posgres test against v10

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2021-05-21 18:11:48 -04:00
zeripath 17c5c654a5
Prevent double-login for Git HTTP and LFS and simplify login (#15303)
* Prevent double-login for Git HTTP and LFS and simplify login

There are a number of inconsistencies with our current methods for
logging in for git and lfs. The first is that there is a double login
process. This is particularly evident in 1.13 where there are no less
than 4 hash checks for basic authentication due to the previous
IsPasswordSet behaviour.

This duplicated code had individual inconsistencies that were not
helpful and caused confusion.

This PR does the following:

* Remove the specific login code from the git and lfs handlers except
for the lfs special bearer token
* Simplify the meaning of DisableBasicAuthentication to allow Token and
Oauth2 sign-in.
* The removal of the specific code from git and lfs means that these
both now have the same login semantics and can - if not
DisableBasicAuthentication - login from external services. Further it
allows Oauth2 token authentication as per our standard mechanisms.
* The change in the recovery handler prevents the service from
re-attempting to login - primarily because this could easily cause a
further panic and it is wasteful.

* add test

Signed-off-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: Andrew Thornton <art27@cantab.net>
2021-05-15 17:32:09 +02:00
zeripath 038e1db4df
Return go-get info on subdirs (#15642)
This PR is an alternative to #15628 and makes the go get handler a
handler.

Fix #15625

Close #15628

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-05-09 22:50:06 +01:00
6543 e278614753
[API] make change repo settings work on empty repos (#15778)
* API: Fix #15602

* Add TEST
2021-05-08 14:11:36 +02:00