gitea/modules
Jason Song 1e4be0945b
Introduce globallock as distributed locks (#31908)
To help #31813, but do not replace it, since this PR just introduces the
new module but misses some work:

- New option in settings. `#31813` has done it.
- Use the locks in business logic. `#31813` has done it.

So I think the most efficient way is to merge this PR first (if it's
acceptable) and then finish #31813.

## Design principles

### Use spinlock even in memory implementation

In actual use cases, users may cancel requests. `sync.Mutex` will block
the goroutine until the lock is acquired even if the request is
canceled. And the spinlock is more suitable for this scenario since it's
possible to give up the lock acquisition.

Although the spinlock consumes more CPU resources, I think it's
acceptable in most cases.

### Do not expose the mutex to callers

If we expose the mutex to callers, it's possible for callers to reuse
the mutex, which causes more complexity.

For example:
```go
lock := GetLocker(key)
lock.Lock()
// ...
// even if the lock is unlocked, we cannot GC the lock,
// since the caller may still use it again.
lock.Unlock()
lock.Lock()
// ...
lock.Unlock()

// callers have to GC the lock manually.
RemoveLocker(key)
```

That's why
https://github.com/go-gitea/gitea/pull/31813#discussion_r1721200549

In this PR, we only expose `ReleaseFunc` to callers. So callers just
need to call `ReleaseFunc` to release the lock, and do not need to care
about the lock's lifecycle.
```go
_, release, err := locker.Lock(ctx, key)
if err != nil {
    return err
}
// ...
release()

// if callers want to lock again, they have to re-acquire the lock.
_, release, err := locker.Lock(ctx, key)
// ...
```

In this way, it's also much easier for redis implementation to extend
the mutex automatically, so that callers do not need to care about the
lock's lifecycle. See also
https://github.com/go-gitea/gitea/pull/31813#discussion_r1722659743

### Use "release" instead of "unlock"

For "unlock", it has the meaning of "unlock an acquired lock". So it's
not acceptable to call "unlock" when failed to acquire the lock, or call
"unlock" multiple times. It causes more complexity for callers to decide
whether to call "unlock" or not.

So we use "release" instead of "unlock" to make it clear. Whether the
lock is acquired or not, callers can always call "release", and it's
also safe to call "release" multiple times.

But the code DO NOT expect callers to not call "release" after acquiring
the lock. If callers forget to call "release", it will cause resource
leak. That's why it's always safe to call "release" without extra
checks: to avoid callers to forget to call it.

### Acquired locks could be lost

Unlike `sync.Mutex` which will be locked forever once acquired until
calling `Unlock`, in the new module, the acquired lock could be lost.

For example, the caller has acquired the lock, and it holds the lock for
a long time since auto-extending is working for redis. However, it lost
the connection to the redis server, and it's impossible to extend the
lock anymore.

If the caller don't stop what it's doing, another instance which can
connect to the redis server could acquire the lock, and do the same
thing, which could cause data inconsistency.

So the caller should know what happened, the solution is to return a new
context which will be canceled if the lock is lost or released:

```go
ctx, release, err := locker.Lock(ctx, key)
if err != nil {
    return err
}
defer release()
// ...
DoSomething(ctx)

// the lock is lost now, then ctx has been canceled.

// Failed, since ctx has been canceled.
DoSomethingElse(ctx)
```

### Multiple ways to use the lock

1. Regular way

```go
ctx, release, err := Lock(ctx, key)
if err != nil {
    return err
}
defer release()
// ...
```

2. Early release

```go
ctx, release, err := Lock(ctx, key)
if err != nil {
    return err
}
defer release()
// ...
// release the lock earlier and reset the context back
ctx = release()
// continue to do something else
// ...
```

3. Functional way

```go
if err := LockAndDo(ctx, key, func(ctx context.Context) error {
    // ...
    return nil
}); err != nil {
    return err
}
```
2024-08-26 22:27:57 +08:00
..
actions Support compression for Actions logs (#31761) 2024-08-09 10:10:30 +08:00
activitypub Upgrade to golangci-lint@v1.55.0 (#27756) 2023-10-24 02:54:59 +00:00
analyze Rename code_langauge.go to code_language.go (#26377) 2023-08-07 15:00:53 -04:00
assetfs Use Set[Type] instead of map[Type]bool/struct{}. (#26804) 2023-08-30 06:55:25 +00:00
auth Add Passkey login support (#31504) 2024-06-29 22:50:03 +00:00
avatar Use crypto/sha256 (#29386) 2024-02-25 13:32:13 +00:00
badge Implement actions badge svgs (#28102) 2024-02-27 18:56:18 +01:00
base fix OIDC introspection authentication (#31632) 2024-07-23 12:43:03 +00:00
cache Add cache test for admins (#31265) 2024-06-17 21:22:39 +02:00
charset Render embedded code preview by permlink in markdown (#30234) 2024-04-02 17:48:27 +00:00
container Allow disabling authentication related user features (#31535) 2024-07-09 17:36:31 +00:00
csv Render embedded code preview by permlink in markdown (#30234) 2024-04-02 17:48:27 +00:00
dump Refactor "dump" sub-command (#30240) 2024-04-03 02:16:46 +00:00
emoji Update emoji set to Unicode 15 (#25595) 2023-06-29 16:29:48 +00:00
eventsource Final round of db.DefaultContext refactor (#27587) 2023-10-14 08:37:24 +00:00
generate Refactor JWT secret generating & decoding code (#29172) 2024-02-16 15:18:30 +00:00
git Refactor the usage of batch catfile (#31754) 2024-08-20 17:04:57 +00:00
gitgraph More db.DefaultContext refactor (#27265) 2023-09-29 12:12:54 +00:00
gitrepo Use repo as of renderctx's member rather than a repoPath on metas (#29222) 2024-05-30 07:04:01 +00:00
globallock Introduce globallock as distributed locks (#31908) 2024-08-26 22:27:57 +08:00
graceful Remove unused error in graceful manager (#29871) 2024-03-18 21:14:51 +00:00
hcaptcha Consume hcaptcha and pwn deps (#22610) 2023-01-29 09:49:51 -06:00
highlight Add option to disable ambiguous unicode characters detection (#28454) 2023-12-17 14:38:54 +00:00
hostmatcher Support allowed hosts for webhook to work with proxy (#27655) 2023-10-18 09:44:36 +00:00
html Refactor backend SVG package and add tests (#26335) 2023-08-05 04:34:59 +00:00
httpcache Also match weakly validated ETags (#28957) 2024-01-29 16:18:40 +00:00
httplib Fix duplicate sub-path for avatars (#31365) 2024-06-15 11:43:57 +08:00
indexer Refactor the usage of batch catfile (#31754) 2024-08-20 17:04:57 +00:00
issue/template Support issue template assignees (#31083) 2024-08-12 16:00:40 +08:00
json Replace interface{} with any (#25686) 2023-07-04 18:36:08 +00:00
label Make label templates have consistent behavior and priority (#23749) 2023-04-10 16:44:02 +08:00
lfs Distinguish LFS object errors to ignore missing objects during migration (#31702) 2024-07-31 10:29:48 +00:00
log Add some tests to clarify the "must-change-password" behavior (#30693) 2024-04-27 12:23:37 +00:00
markup Fix raw wiki links (#31825) 2024-08-16 20:40:51 +08:00
mcaptcha Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
metrics Rename project board -> column to make the UI less confusing (#30170) 2024-05-27 08:59:54 +00:00
migration Use correct function name (#31887) 2024-08-21 21:05:48 +00:00
nosql Update tool dependencies, lock govulncheck and actionlint (#25655) 2023-07-09 11:58:06 +00:00
optional Resolve lint for unused parameter and unnecessary type arguments (#30750) 2024-04-29 08:47:56 +00:00
options Use a general approach to access custom/static/builtin assets (#24022) 2023-04-12 18:16:45 +08:00
packages Support compression for Actions logs (#31761) 2024-08-09 10:10:30 +08:00
paginator Use more specific test methods (#24265) 2023-04-22 17:56:27 -04:00
pprof Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
private Move database operations of merging a pull request to post receive hook and add a transaction (#30805) 2024-05-07 07:36:48 +00:00
process Update misspell to 0.5.1 and add misspellings.csv (#30573) 2024-04-27 08:03:49 +00:00
proxy Use proxy for pull mirror (#22771) 2023-02-11 08:39:50 +08:00
proxyprotocol Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
public Refactor CORS handler (#28587) 2023-12-25 20:13:18 +08:00
queue Fix queue test (#30646) 2024-04-22 23:55:43 +00:00
recaptcha Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
references Refactor to use UnsafeStringToBytes (#31358) 2024-06-14 01:26:33 +00:00
regexplru Upgrade go dependencies (#25819) 2023-07-14 11:00:31 +08:00
repository Distinguish LFS object errors to ignore missing objects during migration (#31702) 2024-07-31 10:29:48 +00:00
secret Use crypto/sha256 (#29386) 2024-02-25 13:32:13 +00:00
session Improve oauth2 client "preferred username field" logic and the error handling (#30622) 2024-04-25 11:22:32 +00:00
setting Support compression for Actions logs (#31761) 2024-08-09 10:10:30 +08:00
sitemap Fix sitemap (#22272) 2022-12-30 23:31:00 +08:00
ssh Remove SSH workaround (#27893) 2023-11-03 15:21:05 +00:00
storage Enable unparam linter (#31277) 2024-06-11 18:47:45 +00:00
structs Actions support workflow dispatch event (#28163) 2024-08-18 22:38:40 -04:00
svg Refactor markdown attention render (#29984) 2024-03-22 12:16:23 +00:00
sync Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
system Refactor to use UnsafeStringToBytes (#31358) 2024-06-14 01:26:33 +00:00
templates Refactor names (#31405) 2024-06-19 06:32:45 +08:00
test Remove sub-path from container registry realm (#31293) 2024-06-09 16:29:29 +08:00
testlogger Replace interface{} with any (#25686) 2023-07-04 18:36:08 +00:00
timeutil Refactor "dump" sub-command (#30240) 2024-04-03 02:16:46 +00:00
translation Render embedded code preview by permlink in markdown (#30234) 2024-04-02 17:48:27 +00:00
turnstile Add new captcha: cloudflare turnstile (#22369) 2023-02-05 15:29:03 +08:00
typesniffer Detect ogg mime-type as audio or video (#26494) 2023-08-15 10:31:25 +08:00
updatechecker Replace more db.DefaultContext (#27628) 2023-10-15 17:46:06 +02:00
uri Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
user Implement FSFE REUSE for golang files (#21840) 2022-11-27 18:20:29 +00:00
util Refactor to use UnsafeStringToBytes (#31358) 2024-06-14 01:26:33 +00:00
validation Check blocklist for emails when adding them to account (#26812) 2023-08-30 10:46:49 -05:00
web Refactor names (#31405) 2024-06-19 06:32:45 +08:00
webhook Fix schedule tasks bugs (#28691) 2024-01-12 21:50:38 +00:00
zstd Support compression for Actions logs (#31761) 2024-08-09 10:10:30 +08:00