Commit Graph

3125 Commits

Author SHA1 Message Date
wxiaoguang 013fb73068
Use `hostmatcher` to replace `matchlist`, improve security (#17605)
Use hostmacher to replace matchlist.

And we introduce a better DialContext to do a full host/IP check, otherwise the attackers can still bypass the allow/block list by a 302 redirection.
2021-11-20 17:34:05 +08:00
Paweł Bogusławski d4e281bc02
Allow Token API calls be authorized using the reverse-proxy header (#15119)
* API calls authorized with HTTP header

This mod allows API calls to be authorized with HTTP header
when ENABLE_REVERSE_PROXY_AUTHENTICATION is enabled. Without
it user authenticated by reverse proxy is able to access
gitea UI but not API which is inconsistent.

Author-Change-Id: IB#1107572

* Fixed API calls authorized with HTTP header

Only reqBasicAuth is modified to allow reverse proxy
auth as alternative and reqToken is left untouched.

Fixes: dc952c0632
Author-Change-Id: IB#1107572

* Reverse proxy API auth separated in docs

Related: https://github.com/go-gitea/gitea/pull/15119#discussion_r621322127
Author-Change-Id: IB#1107572

* Reverse proxy API auth separated in docs

Related: https://github.com/go-gitea/gitea/pull/15119#discussion_r621322127
Author-Change-Id: IB#1107572

* Reverse proxy API auth separated

Related: https://github.com/go-gitea/gitea/pull/15119#discussion_r621322127
Author-Change-Id: IB#1107572

* ReverseProxyAuth removed from swagger

ReverseProxyAuth removed from swagger as in upstream's suggestion.

Related: https://github.com/go-gitea/gitea/pull/15119#pullrequestreview-692180940
Author-Change-Id: IB#1107572

* ReverseProxyAuth API authorization fixed

Related: https://github.com/go-gitea/gitea/pull/15119#issuecomment-868465099
Author-Change-Id: IB#1107572

* ReverseProxyAuth API authorization fixed

Related: https://github.com/go-gitea/gitea/pull/15119#issuecomment-868465099
Author-Change-Id: IB#1107572
2021-11-19 22:54:44 +08:00
Lunny Xiao fc3d082609
Move attachment into models/repo/ (#17650)
* Move attachment into models/repo/

* Fix test

* Fix bug
2021-11-19 21:39:57 +08:00
Lunny Xiao 7a03473159
Use a standalone struct name for Organization (#17632)
* Use a standalone struct name for Organization

* recover unnecessary change

* make the code readable

* Fix template failure

* Fix template failure

* Move HasMemberWithUserID to org

* Fix test

* Remove unnecessary user type check

* Fix test

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-19 19:41:40 +08:00
Gusted 00448ebe91
Remove unused `user` paramater (#17723)
As title
2021-11-19 08:37:13 +00:00
KN4CK3R f34151bdb2
Move user/org deletion to services (#17673) 2021-11-19 01:42:27 +08:00
Gusted c3e020ca34
Add pagination to fork list (#17639)
- Resolves #14574
- Adds the necessary code to have pagination working in the forks list of
a repo. The code is mostly in par with the stars/watcher implementation.
2021-11-18 22:45:56 +08:00
Lunny Xiao 1f1ae57139
Move task from modules to services (#17680) 2021-11-18 14:47:57 +08:00
Lunny Xiao 3c3855a05c
Move user functions into user.go (#17659)
* Move user functions into user.go

* Fix test
2021-11-18 13:58:42 +08:00
Gusted c98dd7a3e0
Remove unnecessary variable assignments (#17695)
* Remove unnecessary variable assignments

As title

* enable ineffassign

Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-18 09:33:06 +08:00
KN4CK3R ea42d3c04e
Fixed commit count. (#17698)
Added "Tag" label.
Unified branch, tag and commit name.
2021-11-17 23:50:17 +00:00
zeripath 3c4724d70e
Add .gitattribute assisted language detection to blame, diff and render (#17590)
Use check attribute code to check the assigned language of a file and send that in to
chroma as a hint for the language of the file.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-17 20:37:00 +00:00
Lunny Xiao 81a4fc7528
Return 400 but not 500 when request archive with wrong format (#17691) 2021-11-17 14:47:35 -05:00
Lunny Xiao 5233051e64
Move some functions into services/repository (#17677) 2021-11-17 23:17:31 +08:00
wxiaoguang 750a8465f5
A better go code formatter, and now `make fmt` can run in Windows (#17684)
* go build / format tools
* re-format imports
2021-11-17 20:34:35 +08:00
Lunny Xiao 95d3266bee
Move user follow and openid into models/user/ (#17613)
* Move UserRedirect into models/user/

* Fix lint & test

* Fix lint

* Fix lint

* remove nolint comment

* Fix lint

* Move user follow and openid into models/user

* Ignore the lint

* Ignore the lint

* Fix test

* ignore stutters lint on UserOpenID
2021-11-17 17:58:31 +08:00
qwerty287 3be156f66a
Allow admins to change user avatars (#17661)
Adds the avatar change panel to the edit user page (bottom) and allows admins to change it this way

Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: zeripath <art27@cantab.net>
2021-11-16 19:13:13 +00:00
zeripath bbffcc3aec
Multiple Escaping Improvements (#17551)
There are multiple places where Gitea does not properly escape URLs that it is building and there are multiple places where it builds urls when there is already a simpler function available to use this.
    
This is an extensive PR attempting to fix these issues.

1. The first commit in this PR looks through all href, src and links in the Gitea codebase and has attempted to catch all the places where there is potentially incomplete escaping.
2. Whilst doing this we will prefer to use functions that create URLs over recreating them by hand.
3. All uses of strings should be directly escaped - even if they are not currently expected to contain escaping characters. The main benefit to doing this will be that we can consider relaxing the constraints on user names and reponames in future. 
4. The next commit looks at escaping in the wiki and re-considers the urls that are used there. Using the improved escaping here wiki files containing '/'. (This implementation will currently still place all of the wiki files the root directory of the repo but this would not be difficult to change.)
5. The title generation in feeds is now properly escaped.
6. EscapePound is no longer needed - urls should be PathEscaped / QueryEscaped as necessary but then re-escaped with Escape when creating html with locales Signed-off-by: Andrew Thornton <art27@cantab.net>

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-16 18:18:25 +00:00
Lunny Xiao 7e1ae38097
Move migrations into services and base into modules/migration (#17663)
* Move migrtions into services and base into modules/migration

* Fix imports

* Fix lint
2021-11-16 23:25:33 +08:00
Lunny Xiao 48ccd325a1
Move some functions into services/repository (#17660) 2021-11-16 21:30:11 +08:00
wxiaoguang 81926d61db
Decouple unit test, remove intermediate `unittestbridge` package (#17662)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-16 16:53:21 +08:00
Gusted ab1379743e
Fix nil checking on typed interface (#17598)
* Fix nil checking on typed interface

- Partially resoles #17596
- Resolves SA4023 errors.
- Ensure correctly that typed interface are nil.

* Remove unnecessary code

`NewBleveIndexer` will never return nil, even on errors.

* Patch `NewBleveIndexer`

* Fix low-level functions

* Remove deadcode

* Fix GetSession

* Close Elastic search when err isn't nil

* Update elastic_search.go

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-15 21:16:11 +08:00
Gusted 253d9e4158
Remove unnecassary calls to `filepath.Join` (#17608)
- Partialy resolvess #17596
- Resolves `badCall` errors from go-critic `badCall: suspicious Join on
1 argument`
- When only 1 argument is passed into `filepath.Join`, it won't do
anything special other than `filepath.Clean(...)` will be applied over
it.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: 6543 <6543@obermui.de>
2021-11-15 14:02:53 +08:00
zeripath bab95c3a86
Correctly handle failed migrations (#17575)
* Correctly handle failed migrations

There is a bug in handling failed migrations whereby the migration task gets decoupled
from the migration repository. This leads to a failure of the task to get deleted with
the repository and also leads to the migration failed page resulting in a ISE.

This PR removes the zeroing out of the task id from the migration but also makes
the migration handler tolerate missing tasks much nicer.

Fix #17571

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-11-13 19:28:50 +08:00
wxiaoguang df64fa4865
Decouple unit test code from business code (#17623) 2021-11-12 22:36:47 +08:00
Lunny Xiao 90eb9fb889
Move EmailAddress & UserRedirect into models/user/ (#17607)
* Move EmailAddress into models/user/

* Fix test

* rename user_mail to user_email

* Fix test

* Move UserRedirect into models/user/

* Fix lint & test

* Fix lint

* Fix lint

* remove nolint comment

* Fix lint
2021-11-11 15:03:30 +08:00
Gusted 492e1c2fbd
Refactor commentTags functionality (#17558)
* feat: Allow multiple tags on comments

- Allow for multiples tags(Currently Poster + {Owner, Writer}).
- Utilize the Poster tag within the commentTag function and remove the
checking from templates.
- Use bitwise on CommentTags to enable specific tags.
- Don't show poster tag(view_content.tmpl) on the initial issue comment.

* Change parameters naming

* Change function name

* refactor variable wording

* Merge 'master' branch into 'tags-comments' branch

* Change naming

* `tag` -> `role`

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2021-11-11 14:29:30 +08:00
Lunny Xiao 0e189eecaa
Move oauth2 error to oauth2 service package (#17603) 2021-11-11 01:51:09 +08:00
Lunny Xiao 43bbc54783
Fix 500 when a comment was deleted which has a notification (#17550)
* Fix 500 when a comment was deleted which has a notification

* Tolerate missing Comment in other places too

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: Andrew Thornton <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-10 13:48:45 +08:00
Lunny Xiao 33fca2b537
Move webhook into models/webhook/ (#17579) 2021-11-10 13:13:16 +08:00
Lunny Xiao 99b2858e62
Move unit into models/unit/ (#17576)
* Move unit into models/unit/

* Rename unit.UnitType as unit.Type
2021-11-09 20:57:58 +01:00
wxiaoguang b6b1e71665
tune sqlite3 message (#17601) 2021-11-09 13:55:24 -05:00
Lunny Xiao d6d147c2c4
Fix 500 when review pull request with anonymous (#17594) 2021-11-09 13:14:12 +00:00
Lunny Xiao 03de593402
Fix bug on detect issue/comment writer (#17591) 2021-11-09 15:44:12 +08:00
Kamil Domański 021df29623
Allow U2F 2FA without TOTP (#11573)
This change enables the usage of U2F without being forced to enroll an TOTP authenticator.
The `/user/auth/u2f` has been changed to hide the "use TOTP instead" bar if TOTP is not enrolled.

Fixes #5410
Fixes #17495
2021-11-08 23:47:19 +01:00
Gusted ebaf4c48ea
Avoid double imports (#17569)
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-08 09:04:13 +02:00
Lunny Xiao 69b61d4373
Fix bug on admin subcommand (#17533)
* Fix bug on admin subcommand

* Add signals for all initDB

Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-07 11:11:27 +08:00
Wim bd613c704c
Fix ipv6 parsing for builtin ssh server (#17561) 2021-11-06 14:23:32 +08:00
qwerty287 e5b48dcac4
Hide label comments if labels were added and removed immediately (#17455)
* Hide label comments if labels were added and removed immediately

* Add comment and rename var

* Fix unit test

* Add test case

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-11-04 22:51:30 +08:00
zeripath 9d855bd6a1
Simplify Gothic to use our session store instead of creating a different store (#17507)
* Simplify Gothic to use our session store instead of creating a different store

We have been using xormstore to provide a separate session store for our OAuth2 logins
however, this relies on using gorilla context and some doubling of our session storing.
We can however, simplify and simply use our own chi-based session store. Thus removing
a cookie and some of the weirdness with missing contexts.

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* as per review

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Handle MaxTokenLength

Signed-off-by: Andrew Thornton <art27@cantab.net>

* oops

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: techknowlogick <techknowlogick@gitea.io>
Co-authored-by: Lauris BH <lauris@nix.lv>
2021-11-03 08:33:54 +08:00
zeripath 4e8a81780e
Stop double encoding blame commit messages (#17498)
The call to html.EscapeString in routers/web/repo/blame.go:renderBlame is extraneous
as the commit message is now rendered by the template. The template will correctly
escape strings - therefore we are currently double escaping.

This PR fixes this.

Fix #17492

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-31 10:25:24 +02:00
Richard Mahn 40c8451b7d
Properly determine CSV delimiter (#17459)
* Fixes #16558 CSV delimiter determiner

* Fixes #16558 - properly determine CSV delmiiter

* Moves quoteString to a new function

* Adds big test with lots of commas for tab delimited csv

* Adds comments

* Shortens the text of the test

* Removes single quotes from regexp as only double quotes need to be searched

* Fixes spelling

* Fixes check of length as it probalby will only be 1e4, not greater

* Makes sample size a const, properly removes truncated line

* Makes sample size a const, properly removes truncated line

* Fixes comment

* Fixes comment

* tests for FormatError() function

* Adds logic to find the limiter before or after a quoted value

* Simplifies regex

* Error tests

* Error tests

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

* Adds comments

* Update modules/csv/csv.go

Co-authored-by: delvh <dev.lh@web.de>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: delvh <dev.lh@web.de>
2021-10-30 23:50:40 +08:00
Lunny Xiao 63c0dc89ef
Rename db Engines related functions (#17481)
* Rename db Engines related functions

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-30 22:32:11 +08:00
zeripath e6e3b212b3
Run Migrate in Install rather than just SyncTables (#17475)
* Run Migrate in Install rather than just SyncTables

The underlying problem in #17328 appears to be that users are re-running the install
page during upgrades. The function that tests and creates the db did not intend for
this and thus instead the migration scripts being run - a simple sync tables occurs.

This then causes a weird partially migrated DB which causes, in this release cycle,
the duplicate column in task table error. It is likely the cause of some weird
partial migration errors in other cycles too.

This PR simply ensures that the migration scripts are also run at this point too.

Fix #17328

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-29 09:23:10 +01:00
zeripath 3fc465ba5e
Fix mispelling of starred as stared (#17465)
There was a recent spelling mistake added to the locale file where stared was used
instead of starred.

This PR changes this to starred.

Signed-off-by: Andrew Thornton <art27@cantab.net>

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-28 15:05:00 +08:00
zeripath 0b4a8be26b
Ensure that restricted users can access repos for which they are members (#17460)
There is a small bug in the way that repo access is checked in
repoAssignment: Accessibility is checked by checking if the user has a
marked access to the repository instead of checking if the user has any
team granted access.

This PR changes this permissions check to use HasAccess() which does the
correct test. There is also a fix in the release api ListReleases where
it should return draft releases if the user is a member of a team with
write access to the releases.

The PR also adds a testcase.

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-28 10:54:40 +08:00
qwerty287 01fc24c78c
Add appearance section in settings (#17433)
* Add appearance section in settings

* Fix lint

* Fix lint

* Apply suggestions from code review

Co-authored-by: Lauris BH <lauris@nix.lv>

Co-authored-by: Lauris BH <lauris@nix.lv>
2021-10-27 23:40:08 +08:00
zeripath 849356deaf
Prevent panic in serv.go with Deploy Keys (#17434)
Unfortunately there was a regression in #17373 which missed that the user is not
for deploy keys. This leads to a panic when pushing with deploy keys.

Fix #17412

Signed-off-by: Andrew Thornton <art27@cantab.net>
2021-10-26 01:24:19 +02:00
qwerty287 3676fafdac
Add API to get/edit wiki (#17278)
* Add API to get/edit wiki

* Add swagger docs, various improvements

* fmt

* Fix lint and rm comment

* Add page parameter

* Add pagination to pages

* Add tests

* fmt

* Update func names

* Update error handling

* Update type name

* Fix lint

* Don't delete Home

* Update func name

* Update routers/api/v1/repo/wiki.go

Co-authored-by: delvh <dev.lh@web.de>

* Remove unnecessary check

* Fix lint

* Use English strings

* Update integrations/api_wiki_test.go

Co-authored-by: delvh <dev.lh@web.de>

* Update func and test names

* Remove unsed check and avoid duplicated error reports

* Improve error handling

* Return after error

* Document 404 error

* Update swagger

* Fix lint

* Apply suggestions from code review

Co-authored-by: delvh <dev.lh@web.de>

* Document file encoding

* fmt

* Apply suggestions

* Use convert

* Fix integration test

* simplify permissions

* unify duplicate key Title/Name

* improve types & return UTC timestamps

* improve types pt.2

- add WikiPageMetaData.LastCommit
- add WikiPageMetaData.HTMLURL
- replace WikiPageMetaData.Updated with .LastCommit.Committer.Created

also delete convert.ToWikiPage(), as it received too many arguments and
only had one callsite anyway. sorry for bad advice earlier 🙃

* WikiPage.Content is base64 encoded

* simplify error handling in wikiContentsByName()

* update swagger

* fix & DRY findWikiRepoCommit() error handling

ListWikiPages() previously wrote error twice when repo wiki didn't exist

* rename Content -> ContentBase64

* Fix test

* Fix tests

* Update var name

* suburl -> sub_url

Co-authored-by: delvh <dev.lh@web.de>
Co-authored-by: Norwin <git@nroo.de>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2021-10-25 11:43:40 +08:00
KN4CK3R 44f2c27d28
Fix CSV render error (#17406)
closed #17378 

Both errors from #17378 were caused by  #15175.

Problem 1 (error with added file):
`ToUTF8WithFallbackReader` creates a `MultiReader` from a `byte[2048]` and the remaining reader. `CreateReaderAndGuessDelimiter` tries to read 10000 bytes from this reader but only gets 2048 because that's the first reader in the `MultiReader`. Then the `if size < 1e4` thinks the input is at EOF and just returns that.

Problem 2 (error with changed file):
The blob reader gets defer closed. That was fine because the old version reads the whole file into memory. Now with the streaming version the close needs to defer after the method.
2021-10-25 00:42:32 +02:00