Commit Graph

55 Commits

Author SHA1 Message Date
KN4CK3R
2173f14708
Add user webhooks (#21563)
Currently we can add webhooks for organizations but not for users. This
PR adds the latter. You can access it from the current users settings.


![grafik](https://user-images.githubusercontent.com/1666336/197391408-15dfdc23-b476-4d0c-82f7-9bc9b065988f.png)
2023-03-10 08:28:32 -06:00
Lunny Xiao
bd820aa9c5
Add context cache as a request level cache (#22294)
To avoid duplicated load of the same data in an HTTP request, we can set
a context cache to do that. i.e. Some pages may load a user from a
database with the same id in different areas on the same page. But the
code is hidden in two different deep logic. How should we share the
user? As a result of this PR, now if both entry functions accept
`context.Context` as the first parameter and we just need to refactor
`GetUserByID` to reuse the user from the context cache. Then it will not
be loaded twice on an HTTP request.

But of course, sometimes we would like to reload an object from the
database, that's why `RemoveContextData` is also exposed.

The core context cache is here. It defines a new context
```go
type cacheContext struct {
	ctx  context.Context
	data map[any]map[any]any
        lock sync.RWMutex
}

var cacheContextKey = struct{}{}

func WithCacheContext(ctx context.Context) context.Context {
	return context.WithValue(ctx, cacheContextKey, &cacheContext{
		ctx:  ctx,
		data: make(map[any]map[any]any),
	})
}
```

Then you can use the below 4 methods to read/write/del the data within
the same context.

```go
func GetContextData(ctx context.Context, tp, key any) any
func SetContextData(ctx context.Context, tp, key, value any)
func RemoveContextData(ctx context.Context, tp, key any)
func GetWithContextCache[T any](ctx context.Context, cacheGroupKey string, cacheTargetID any, f func() (T, error)) (T, error)
```

Then let's take a look at how `system.GetString` implement it.

```go
func GetSetting(ctx context.Context, key string) (string, error) {
	return cache.GetWithContextCache(ctx, contextCacheKey, key, func() (string, error) {
		return cache.GetString(genSettingCacheKey(key), func() (string, error) {
			res, err := GetSettingNoCache(ctx, key)
			if err != nil {
				return "", err
			}
			return res.SettingValue, nil
		})
	})
}
```

First, it will check if context data include the setting object with the
key. If not, it will query from the global cache which may be memory or
a Redis cache. If not, it will get the object from the database. In the
end, if the object gets from the global cache or database, it will be
set into the context cache.

An object stored in the context cache will only be destroyed after the
context disappeared.
2023-02-15 21:37:34 +08:00
delvh
0f4e1b9ac6
Restructure webhook module (#22256)
Previously, there was an `import services/webhooks` inside
`modules/notification/webhook`.
This import was removed (after fighting against many import cycles).
Additionally, `modules/notification/webhook` was moved to
`modules/webhook`,
and a few structs/constants were extracted from `models/webhooks` to
`modules/webhook`.

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2023-01-01 23:23:15 +08:00
KN4CK3R
a35749893b
Move convert package to services (#22264)
Addition to #22256

The `convert` package relies heavily on different models which is
[disallowed by our definition of
modules](https://github.com/go-gitea/gitea/blob/main/CONTRIBUTING.md#design-guideline).
This helps to prevent possible import cycles.

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
2022-12-29 10:57:15 +08:00
Lunny Xiao
0a7d3ff786
refactor some functions to support ctx as first parameter (#21878)
Co-authored-by: KN4CK3R <admin@oldschoolhack.me>
Co-authored-by: Lauris BH <lauris@nix.lv>
2022-12-03 10:48:26 +08:00
flynnnnnnnnnn
e81ccc406b
Implement FSFE REUSE for golang files (#21840)
Change all license headers to comply with REUSE specification.

Fix #16132

Co-authored-by: flynnnnnnnnnn <flynnnnnnnnnn@github>
Co-authored-by: John Olheiser <john.olheiser@gmail.com>
2022-11-27 18:20:29 +00:00
oliverpool
b6e81357bd
Add Webhook authorization header (#20926)
_This is a different approach to #20267, I took the liberty of adapting
some parts, see below_

## Context

In some cases, a weebhook endpoint requires some kind of authentication.
The usual way is by sending a static `Authorization` header, with a
given token. For instance:

- Matrix expects a `Bearer <token>` (already implemented, by storing the
header cleartext in the metadata - which is buggy on retry #19872)
- TeamCity #18667
- Gitea instances #20267
- SourceHut https://man.sr.ht/graphql.md#authentication-strategies (this
is my actual personal need :)

## Proposed solution

Add a dedicated encrypt column to the webhook table (instead of storing
it as meta as proposed in #20267), so that it gets available for all
present and future hook types (especially the custom ones #19307).

This would also solve the buggy matrix retry #19872.

As a first step, I would recommend focusing on the backend logic and
improve the frontend at a later stage. For now the UI is a simple
`Authorization` field (which could be later customized with `Bearer` and
`Basic` switches):


![2022-08-23-142911](https://user-images.githubusercontent.com/3864879/186162483-5b721504-eef5-4932-812e-eb96a68494cc.png)

The header name is hard-coded, since I couldn't fine any usecase
justifying otherwise.

## Questions

- What do you think of this approach? @justusbunsi @Gusted @silverwind 
- ~~How are the migrations generated? Do I have to manually create a new
file, or is there a command for that?~~
- ~~I started adding it to the API: should I complete it or should I
drop it? (I don't know how much the API is actually used)~~

## Done as well:

- add a migration for the existing matrix webhooks and remove the
`Authorization` logic there


_Closes #19872_

Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
Co-authored-by: Gusted <williamzijl7@hotmail.com>
Co-authored-by: delvh <dev.lh@web.de>
2022-11-03 20:23:20 +02:00
KN4CK3R
1887c95254
Decouple HookTask from Repository (#17940)
At the moment a repository reference is needed for webhooks. With the
upcoming package PR we need to send webhooks without a repository
reference. For example a package is uploaded to an organization. In
theory this enables the usage of webhooks for future user actions.

This PR removes the repository id from `HookTask` and changes how the
hooks are processed (see `services/webhook/deliver.go`). In a follow up
PR I want to remove the usage of the `UniqueQueue´ and replace it with a
normal queue because there is no reason to be unique.

Co-authored-by: 6543 <6543@obermui.de>
2022-10-21 18:21:56 +02:00
KN4CK3R
cad9adeff4
Display total commit count in hook message (#21400)
Fixes #21379

The commits are capped by `setting.UI.FeedMaxCommitNum` so
`len(commits)` is not the correct number. So this PR adds a new
`TotalCommits` field.

Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-10-17 00:22:34 +08:00
Jason Song
93a610a819
Fill the specified ref in webhook test payload (#20961)
The webhook payload should use the right ref when it‘s specified in the testing request.

The compare URL should not be empty, a URL like `compare/A...A` seems useless in most cases but is helpful when testing.
2022-09-04 12:18:07 +03:00
Lunny Xiao
fd7d83ace6
Move almost all functions' parameter db.Engine to context.Context (#19748)
* Move almost all functions' parameter db.Engine to context.Context
* remove some unnecessary wrap functions
2022-05-20 22:08:52 +08:00
6543
c764355676
RepoAssignment ensure to close before overwrite (#19449)
* check if GitRepo already open and close if

* only run RepoAssignment once

* refactor context helper for api to open GitRepo
2022-04-21 17:17:57 +02:00
KN4CK3R
80fd25524e
Renamed ctx.User to ctx.Doer. (#19161)
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: wxiaoguang <wxiaoguang@gmail.com>
2022-03-22 15:03:22 +08:00
Lunny Xiao
1fee11d69a
Move accessmode into models/perm (#17828) 2021-11-28 12:58:28 +01:00
Lunny Xiao
33fca2b537
Move webhook into models/webhook/ (#17579) 2021-11-10 13:13:16 +08:00
6543
2289580bb7
[API] generalize list header (#16551)
* Add info about list endpoints to CONTRIBUTING.md

* Let all list endpoints return X-Total-Count header 

* Add TODOs for GetCombinedCommitStatusByRef

* Fix models/issue_stopwatch.go

* Rrefactor models.ListDeployKeys

* Introduce helper func and use them for SetLinkHeader related func
2021-08-12 14:43:08 +02:00
KN4CK3R
aac663e0da
Implemented head_commit for webhooks (#16282)
* Removed Len field.

* Added head_commit webhook field.

* Added comment for returns.
2021-06-29 15:34:03 +02:00
6543
290cf75f93
[refactor] Unify the export of user data via API (#15144)
* [refactor] unify how user data is exported via API

* test time via unix timestamp
2021-03-27 17:45:26 +01:00
Lunny Xiao
6433ba0ec3
Move macaron to chi (#14293)
Use [chi](https://github.com/go-chi/chi) instead of the forked [macaron](https://gitea.com/macaron/macaron). Since macaron and chi have conflicts with session share, this big PR becomes a have-to thing. According my previous idea, we can replace macaron step by step but I'm wrong. :( Below is a list of big changes on this PR.

- [x] Define `context.ResponseWriter` interface with an implementation `context.Response`.
- [x] Use chi instead of macaron, and also a customize `Route` to wrap chi so that the router usage is similar as before.
- [x] Create different routers for `web`, `api`, `internal` and `install` so that the codes will be more clear and no magic .
- [x] Use https://github.com/unrolled/render instead of macaron's internal render
- [x] Use https://github.com/NYTimes/gziphandler instead of https://gitea.com/macaron/gzip
- [x] Use https://gitea.com/go-chi/session which is a modified version of https://gitea.com/macaron/session and removed `nodb` support since it will not be maintained. **BREAK**
- [x] Use https://gitea.com/go-chi/captcha which is a modified version of https://gitea.com/macaron/captcha
- [x] Use https://gitea.com/go-chi/cache which is a modified version of https://gitea.com/macaron/cache
- [x] Use https://gitea.com/go-chi/binding which is a modified version of https://gitea.com/macaron/binding
- [x] Use https://github.com/go-chi/cors instead of https://gitea.com/macaron/cors
- [x] Dropped https://gitea.com/macaron/i18n and make a new one in `code.gitea.io/gitea/modules/translation`
- [x] Move validation form structs from `code.gitea.io/gitea/modules/auth` to `code.gitea.io/gitea/modules/forms` to avoid dependency cycle.
- [x] Removed macaron log service because it's not need any more. **BREAK**
- [x] All form structs have to be get by `web.GetForm(ctx)` in the route function but not as a function parameter on routes definition.
- [x] Move Git HTTP protocol implementation to use routers directly.
- [x] Fix the problem that chi routes don't support trailing slash but macaron did.
- [x] `/api/v1/swagger` now will be redirect to `/api/swagger` but not render directly so that `APIContext` will not create a html render. 

Notices:
- Chi router don't support request with trailing slash
- Integration test `TestUserHeatmap` maybe mysql version related. It's failed on my macOS(mysql 5.7.29 installed via brew) but succeed on CI.

Co-authored-by: 6543 <6543@obermui.de>
2021-01-26 16:36:53 +01:00
Lunny Xiao
42354dfe45
Move webhook type from int to string (#13664)
* Move webhook type from int to string

* rename webhook_services

* finish refactor

* Fix merge

* Ignore unnecessary ci

Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2020-12-08 11:41:14 +01:00
6543
7ac8a770e1
Move Repo APIFormat to convert package (#13787)
* Move Repo APIFormat to convert package

* tweek
2020-12-02 16:38:30 -05:00
6543
1418288734
Refactor: move Commit To APIFormat Code & Lot of StopWatch related things (#12729)
* move GitCommit to APIFormat convertion into convert package

* rename Commit convert functions

* move stopwatch to api convertion into convert package & rm unused code & extend test

* fix compare time

* Gitea not Gogs ;)
2020-09-18 20:09:26 +08:00
Cirno the Strongest
27a6e03906
Remove page size limit comment from swagger (#11806)
Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-06-09 00:57:38 -04:00
SpaWn2KiLl
1f01f53c53 API add/generalize pagination (#9452)
* paginate results

* fixed deadlock

* prevented breaking change

* updated swagger

* go fmt

* fixed find topic

* go mod tidy

* go mod vendor with go1.13.5

* fixed repo find topics

* fixed unit test

* added Limit method to Engine struct; use engine variable when provided; fixed gitignore

* use ItemsPerPage for default pagesize; fix GetWatchers, getOrgUsersByOrgID and GetStargazers; fix GetAllCommits headers; reverted some changed behaviors

* set Page value on Home route

* improved memory allocations

* fixed response headers

* removed logfiles

* fixed import order

* import order

* improved swagger

* added function to get models.ListOptions from context

* removed pagesize diff on unit test

* fixed imports

* removed unnecessary struct field

* fixed go fmt

* scoped PR

* code improvements

* code improvements

* go mod tidy

* fixed import order

* fixed commit statuses session

* fixed files headers

* fixed headers; added pagination for notifications

* go mod tidy

* go fmt

* removed Private from user search options; added setting.UI.IssuePagingNum as default valeu on repo's issues list

* Apply suggestions from code review

Co-Authored-By: 6543 <6543@obermui.de>
Co-Authored-By: zeripath <art27@cantab.net>

* fixed build error

* CI.restart()

* fixed merge conflicts resolve

* fixed conflicts resolve

* improved FindTrackedTimesOptions.ToOptions() method

* added backwards compatibility on ListReleases request; fixed issue tracked time ToSession

* fixed build error; fixed swagger template

* fixed swagger template

* fixed ListReleases backwards compatibility

* added page to user search route

Co-authored-by: techknowlogick <matti@mdranta.net>
Co-authored-by: 6543 <6543@obermui.de>
Co-authored-by: zeripath <art27@cantab.net>
2020-01-24 14:00:29 -05:00
6543
2848c5eb8f Swagger info corrections (#9441)
* use numbers and not http.Status___ enum

* fix test

* add many missing swagger responses

* code format

* Deletion Sould return 204 ...

* error handling improvements

* if special error type ... then add it to swagger too

* one smal nit

* invalidTopicsError is []string

* valid swagger specification 2.0
 - if you add responses swagger can tell you if you do it right 👍

* use ctx.InternalServerError

* Revert "use numbers and not http.Status___ enum"

This reverts commit b1ff386e24.

* use http.Status* enum everywhere
2019-12-20 19:07:12 +02:00
Lunny Xiao
daab245167
Move code.gitea.io/gitea/routers/api/v1/convert to code.gitea.io/gitea/modules/convert (#8892)
* Move code.gitea.io/gitea/routers/api/v1/convert to code.gitea.io/gitea/modules/convert

* fix fmt
2019-11-10 12:41:51 +08:00
Lunny Xiao
f518fe6662
Rename HookQueue to hookQueue (#8778)
* Rename HookQueue to hookQueue

* fix lint
2019-11-02 10:35:12 +08:00
Lunny Xiao
0e7f7df3cf Move webhook to a standalone package under modules (#8747)
* Move webhook to a standalone package under modules

* fix test

* fix comments
2019-11-01 22:51:22 +00:00
renothing
cbf231a675 fix wrong email when use gitea as OAuth2 provider (#7640)
when you use gitea as OAuth2 provider, the /api/v1/user should return
user primary email as identifier, which is unique in OAuth2 clients.

this patch use convert.ToUser replace all u.APIFormat in api requests,
return primary email when caller is yourself or admin.
2019-07-27 16:15:30 +03:00
Lunny Xiao
34eee25bd4
Move sdk structs to modules/structs (#6905)
* move sdk structs to moduels/structs

* fix tests

* fix fmt

* fix swagger

* fix vendor
2019-05-11 18:21:34 +08:00
Lunny Xiao
d578b71d61
move code.gitea.io/git to code.gitea.io/gitea/modules/git (#6364)
* move code.gitea.io/git to code.gitea.io/gitea/modules/git

* fix imports

* fix fmt

* fix misspell

* remove wrong tests data

* fix unit tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* enable Debug to trace the failure tests

* fix tests

* fix tests

* fix tests

* fix tests

* fix tests

* comment commit count tests since git clone depth is 50

* fix tests

* update from code.gitea.io/git

* revert change to makefile
2019-03-27 17:33:00 +08:00
John Olheiser
cac9e6e760 Updates to API 404 responses (#6077) 2019-03-18 22:29:43 -04:00
zeripath
43f9233914 Fix Swagger JSON autogeneration issues. (#4845)
* Fix Swagger JSON.

Remove unnecessary schema references for the forbidden and empty responses

Signed-off-by: Andrew Thornton <art27@cantab.net>

* Fix swagger API for CreateAccessToken

* Fix admin create org swagger

* Fix swagger for adminCreateRepo

* More swagger fixes

Set int64 format for those which are int64
Some more form fixes

* Fix swagger description of GET /repos/{owner}/{repo}/pulls
2018-10-20 23:40:42 -04:00
Antoine GIRARD
908e8942cc Fix swagger errors (#4220)
Fix all the resting errors to have a valid swagger file.

They are still some warnings but nothing blocking.

Doing so I found that some request still misses son parameters for some POST/PUT/PATCH request. This means the a client generated from the swagger file will not work completely. 

Fix #4088 by activating validation in drone
Should fix #4010.
2018-06-12 22:59:22 +08:00
Ethan Koenig
7ea4bfc561 API endpoint for testing webhook (#3550)
* API endpoint for testing webhook

* Empty commit to rerun CI
2018-04-29 14:21:33 +08:00
Bo-Yi Wu
1c5cbc390b refactor: import order. (#3736) 2018-03-29 21:32:40 +08:00
Ethan Koenig
876bafb7ef Fix API status code for hook creation (#2814)
* Fix API status code for hook creation

* Named constants for response statuses
2017-11-20 15:00:53 +08:00
Ethan Koenig
f26f4a7e01 Update swagger documentation (#2899)
* Update swagger documentation

Add docs for missing endpoints
Add documentation for request parameters
Make parameter naming consistent
Fix response documentation

* Restore delete comments
2017-11-13 09:02:25 +02:00
Antoine GIRARD
fd8e8a421a Improve swagger doc (#2274)
* Add swagger comment for adminCreateOrg

* Add swagger comment for admin route

* add hook swagger doc

* Add tags

* Add auth

* Fix name of responses

* Edit name method

* Update vendor

* make generate-swagger
2017-08-21 14:13:47 +03:00
Antoine GIRARD
3edb0c5894 Generate swagger json (#1402)
- Generate swagger.json into public/
- Add swagger-ui auto-installation
- Add footer link to local swagger-ui
- Add /swagger url for using app url.
- Fix Swagger-UI version via git tag
2017-05-02 15:35:59 +02:00
Ethan Koenig
27fcf8d30a Bug fixes for webhook API (#650) 2017-01-14 10:14:48 +08:00
Ethan Koenig
9847b38518 Organization webhook API endpoints 2016-12-25 14:51:57 +01:00
Ethan Koenig
7b5b5178e1 Bug fix for edit-hook API endpoint 2016-12-09 20:11:56 +01:00
Lunny Xiao
3917ed45de golint fixed for routers (#208) 2016-11-24 15:04:31 +08:00
Sandro Santilli
a1c5f02444 Fix import path of go-sdk (#141)
From code.gitea.io/go-sdk/gitea
  To code.gitea.io/sdk/gitea
2016-11-11 17:39:44 +08:00
Sandro Santilli
4247304f5a Update import paths from github.com/go-gitea to code.gitea.io (#135)
- Update import paths from github.com/go-gitea to code.gitea.io
- Fix import path for travis

See https://docs.travis-ci.com/user/languages/go#Go-Import-Path
2016-11-10 17:24:48 +01:00
Sandro Santilli
80eea77953 Use MixedCase constant names
See https://github.com/golang/go/wiki/CodeReviewComments#mixed-caps
2016-11-07 17:05:08 +01:00
Thomas Boerger
4c6c16f358
Replaced go-gogs-client with go-sdk imports 2016-11-07 16:10:32 +01:00
Rémy Boulanouar
2d68bd1ef9 Change import reference to match gitea instead of gogs (#37) 2016-11-03 10:29:56 -02:00
Kurt Madel
67fb0fe6a5 added support to set pull_request event from api (#3513) 2016-08-24 20:44:58 -07:00