rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/include/storage/s_lock.h

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

868 lines
25 KiB
C
Raw Normal View History

Inline frequently called functions.
1997-09-18 16:21:02 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* s_lock.h
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
* Hardware-dependent implementation of spinlocks.
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
*
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
* NOTE: none of the macros in this file are intended to be called directly.
* Call them through the hardware-independent macros in spin.h.
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
*
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
* The following hardware-dependent macros must be provided for each
* supported platform:
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
*
* void S_INIT_LOCK(slock_t *lock)
* Initialize a spinlock (to the unlocked state).
*
When LWLOCK_STATS is defined, count spindelays. When LWLOCK_STATS is *not* defined, the only change is that SpinLockAcquire now returns the number of delays. Patch by me, review by Jeff Janes.
2012-06-26 22:02:55 +02:00
* int S_LOCK(slock_t *lock)
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* Acquire a spinlock, waiting if necessary.
* Time out and abort() if unable to acquire the lock in a
* "reasonable" amount of time --- typically ~ 1 minute.
When LWLOCK_STATS is defined, count spindelays. When LWLOCK_STATS is *not* defined, the only change is that SpinLockAcquire now returns the number of delays. Patch by me, review by Jeff Janes.
2012-06-26 22:02:55 +02:00
* Should return number of "delays"; see s_lock.c
From: Jeroen van Vianen <jeroenv@design.nl> Attached patch will add a version() function to Postges, e.g. template1=> select version(); version ------------------------------------------------------------ PostgreSQL 6.3.2 on i586-pc-linux-gnu, compiled by gcc 2.8.1 (1 row)
1998-04-29 14:41:29 +02:00
*
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* void S_UNLOCK(slock_t *lock)
* Unlock a previously acquired lock.
From: Jeroen van Vianen <jeroenv@design.nl> Attached patch will add a version() function to Postges, e.g. template1=> select version(); version ------------------------------------------------------------ PostgreSQL 6.3.2 on i586-pc-linux-gnu, compiled by gcc 2.8.1 (1 row)
1998-04-29 14:41:29 +02:00
*
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* bool S_LOCK_FREE(slock_t *lock)
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 06:22:32 +02:00
* Tests if the lock is free. Returns true if free, false if locked.
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* This does *not* change the state of the lock.
From: Jeroen van Vianen <jeroenv@design.nl> Attached patch will add a version() function to Postges, e.g. template1=> select version(); version ------------------------------------------------------------ PostgreSQL 6.3.2 on i586-pc-linux-gnu, compiled by gcc 2.8.1 (1 row)
1998-04-29 14:41:29 +02:00
*
Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom.
2003-12-27 21:58:58 +01:00
* void SPIN_DELAY(void)
* Delay operation to occur inside spinlock wait loop.
*
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
* Note to implementors: there are default implementations for all these
* macros at the bottom of the file. Check if your platform can use
* these or needs to override them.
*
Improve spinlock performance for HP-UX, ia64, non-gcc. At least on this architecture, it's very important to spin on a non-atomic instruction and only retry the atomic once it appears that it will succeed. To fix this, split TAS() into two macros: TAS(), for trying to grab the lock the first time, and TAS_SPIN(), for spinning until we get it. TAS_SPIN() defaults to same as TAS(), but we can override it when we know there's a better way. It's likely that some of the other cases in s_lock.h require similar treatment, but this is the only one we've got conclusive evidence for at present.
2011-08-29 16:05:48 +02:00
* Usually, S_LOCK() is implemented in terms of even lower-level macros
* TAS() and TAS_SPIN():
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
*
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* int TAS(slock_t *lock)
* Atomic test-and-set instruction. Attempt to acquire the lock,
New ASM format: /* * Standard __asm__ format: * * __asm__( * "command;" * "command;" * "command;" * : "=r"(_res) return value, in register * : "r"(lock) argument, 'lock pointer', in register * : "r0"); inline code uses this register */
2001-01-19 03:58:59 +01:00
* but do *not* wait. Returns 0 if successful, nonzero if unable
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* to acquire the lock.
From: Jeroen van Vianen <jeroenv@design.nl> Attached patch will add a version() function to Postges, e.g. template1=> select version(); version ------------------------------------------------------------ PostgreSQL 6.3.2 on i586-pc-linux-gnu, compiled by gcc 2.8.1 (1 row)
1998-04-29 14:41:29 +02:00
*
Improve spinlock performance for HP-UX, ia64, non-gcc. At least on this architecture, it's very important to spin on a non-atomic instruction and only retry the atomic once it appears that it will succeed. To fix this, split TAS() into two macros: TAS(), for trying to grab the lock the first time, and TAS_SPIN(), for spinning until we get it. TAS_SPIN() defaults to same as TAS(), but we can override it when we know there's a better way. It's likely that some of the other cases in s_lock.h require similar treatment, but this is the only one we've got conclusive evidence for at present.
2011-08-29 16:05:48 +02:00
* int TAS_SPIN(slock_t *lock)
* Like TAS(), but this version is used when waiting for a lock
Use a non-locking test in TAS_SPIN() on all IA64 platforms. Per my testing, this works just as well with gcc as it does with HP's compiler; and there is no reason to think that the effect doesn't occur with icc, either. Also, rewrite the header comment about enforcing sequencing around spinlock operations, per Robert's gripe that it was misleading.
2011-08-29 19:18:44 +02:00
* previously found to be contended. By default, this is the
Improve spinlock performance for HP-UX, ia64, non-gcc. At least on this architecture, it's very important to spin on a non-atomic instruction and only retry the atomic once it appears that it will succeed. To fix this, split TAS() into two macros: TAS(), for trying to grab the lock the first time, and TAS_SPIN(), for spinning until we get it. TAS_SPIN() defaults to same as TAS(), but we can override it when we know there's a better way. It's likely that some of the other cases in s_lock.h require similar treatment, but this is the only one we've got conclusive evidence for at present.
2011-08-29 16:05:48 +02:00
* same as TAS(), but on some architectures it's better to poll a
* contended lock using an unlocked instruction and retry the
* atomic test-and-set only when it appears free.
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
*
Improve spinlock performance for HP-UX, ia64, non-gcc. At least on this architecture, it's very important to spin on a non-atomic instruction and only retry the atomic once it appears that it will succeed. To fix this, split TAS() into two macros: TAS(), for trying to grab the lock the first time, and TAS_SPIN(), for spinning until we get it. TAS_SPIN() defaults to same as TAS(), but we can override it when we know there's a better way. It's likely that some of the other cases in s_lock.h require similar treatment, but this is the only one we've got conclusive evidence for at present.
2011-08-29 16:05:48 +02:00
* TAS() and TAS_SPIN() are NOT part of the API, and should never be called
* directly.
From: Jeroen van Vianen <jeroenv@design.nl> Attached patch will add a version() function to Postges, e.g. template1=> select version(); version ------------------------------------------------------------ PostgreSQL 6.3.2 on i586-pc-linux-gnu, compiled by gcc 2.8.1 (1 row)
1998-04-29 14:41:29 +02:00
*
Improve spinlock performance for HP-UX, ia64, non-gcc. At least on this architecture, it's very important to spin on a non-atomic instruction and only retry the atomic once it appears that it will succeed. To fix this, split TAS() into two macros: TAS(), for trying to grab the lock the first time, and TAS_SPIN(), for spinning until we get it. TAS_SPIN() defaults to same as TAS(), but we can override it when we know there's a better way. It's likely that some of the other cases in s_lock.h require similar treatment, but this is the only one we've got conclusive evidence for at present.
2011-08-29 16:05:48 +02:00
* CAUTION: on some platforms TAS() and/or TAS_SPIN() may sometimes report
* failure to acquire a lock even when the lock is not locked. For example,
* on Alpha TAS() will "fail" if interrupted. Therefore a retry loop must
* always be used, even if you are certain the lock is free.
*
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
* It is the responsibility of these macros to make sure that the compiler
* does not re-order accesses to shared memory to precede the actual lock
* acquisition, or follow the lock release. Prior to PostgreSQL 9.5, this
* was the caller's responsibility, which meant that callers had to use
* volatile-qualified pointers to refer to both the spinlock itself and the
* shared data being accessed within the spinlocked critical section. This
* was notationally awkward, easy to forget (and thus error-prone), and
* prevented some useful compiler optimizations. For these reasons, we
* now require that the macros themselves prevent compiler re-ordering,
* so that the caller doesn't need to take special precautions.
Use a non-locking test in TAS_SPIN() on all IA64 platforms. Per my testing, this works just as well with gcc as it does with HP's compiler; and there is no reason to think that the effect doesn't occur with icc, either. Also, rewrite the header comment about enforcing sequencing around spinlock operations, per Robert's gripe that it was misleading.
2011-08-29 19:18:44 +02:00
*
* On platforms with weak memory ordering, the TAS(), TAS_SPIN(), and
* S_UNLOCK() macros must further include hardware-level memory fence
* instructions to prevent similar re-ordering at the hardware level.
* TAS() and TAS_SPIN() must guarantee that loads and stores issued after
* the macro are not executed until the lock has been obtained. Conversely,
* S_UNLOCK() must guarantee that loads and stores issued before the macro
* have been executed before the lock is released.
Make sure that inlined S_UNLOCK is marked as an update of a 'volatile' object. This should prevent the compiler from reordering loads and stores into or out of a critical section.
2001-12-11 03:58:49 +01:00
*
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* On most supported platforms, TAS() uses a tas() function written
* in assembly language to execute a hardware atomic-test-and-set
* instruction. Equivalent OS-supplied mutex routines could be used too.
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
*
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
* If no system-specific TAS() is available (ie, HAVE_SPINLOCKS is not
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
* defined), then we fall back on an emulation that uses SysV semaphores
* (see spin.c). This emulation will be MUCH MUCH slower than a proper TAS()
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* implementation, because of the cost of a kernel call per lock or unlock.
* An old report is that Postgres spends around 40% of its time in semop(2)
* when using the SysV semaphore code.
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
*
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
*
Update copyright for 2023 Backpatch-through: 11
2023-01-02 21:00:37 +01:00
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
* Portions Copyright (c) 1994, Regents of the University of California
*
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/include/storage/s_lock.h
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
*
*-------------------------------------------------------------------------
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
*/
Significant cleanups in SysV IPC handling (shared mem and semaphores). IPC key assignment will now work correctly even when multiple postmasters are using same logical port number (which is possible given -k switch). There is only one shared-mem segment per postmaster now, not 3. Rip out broken code for non-TAS case in bufmgr and xlog, substitute a complete S_LOCK emulation using semaphores in spin.c. TAS and non-TAS logic is now exactly the same. When deadlock is detected, "Deadlock detected" is now the elog(ERROR) message, rather than a NOTICE that comes out before an unhelpful ERROR.
2000-11-29 00:27:57 +01:00
#ifndef S_LOCK_H
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
#define S_LOCK_H
Don't include low level locking code from frontend code. Some frontend code like e.g. pg_xlogdump or pg_resetxlog, has to use backend headers. Unfortunately until now that code includes most of the locking code. It's generally not nice to expose such low level details, but de6fd1c898 made that a hard problem. We fall back to defining 'inline' away if the compiler doesn't support it - that can cause linker errors like on buildfarm animal pademelon if a inline function references backend only code. To fix that problem separate definitions from lock.h that are required from frontend code into lockdefs.h and use it in the relevant places. I've only removed the minimal amount of necessary definitions for now - it might turn out that we want more for other reasons. To avoid such details being exposed again put some checks against being included from frontend code into atomics.h, lock.h, lwlock.h and s_lock.h. It's otherwise fairly easy to indirectly include these headers. Discussion: 20150806070902.GE12214@awork2.anarazel.de
2015-08-07 15:10:56 +02:00
#ifdef FRONTEND
#error "s_lock.h may not be included from frontend code"
#endif
Have configure --without-spinlocks actually not use spinlock code, even if supported by the cpu.
2003-12-23 04:52:10 +01:00
#ifdef HAVE_SPINLOCKS /* skip spinlocks if requested */
Initial MVCC code. New code for locking buffer' context.
1998-12-15 13:47:01 +01:00
Apparently icc doesn't always define __ICC, and it's more correct to check for __INTEL_COMPILER. Per report from Dirk Tilger. Not back-patched since I don't fully trust it yet ...
2007-08-05 17:11:40 +02:00
#if defined(__GNUC__) || defined(__INTEL_COMPILER)
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
/*************************************************************************
* All the gcc inlines
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
* Gcc consistently defines the CPU as __cpu__.
* Other compilers use __cpu or __cpu__ so we test for both in those cases.
From: Jeroen van Vianen <jeroenv@design.nl> Attached patch will add a version() function to Postges, e.g. template1=> select version(); version ------------------------------------------------------------ PostgreSQL 6.3.2 on i586-pc-linux-gnu, compiled by gcc 2.8.1 (1 row)
1998-04-29 14:41:29 +02:00
*/
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
/*----------
* Standard gcc asm format (assuming "volatile slock_t *lock"):
Still further tweaking of s_lock assembler: do not assume that leading whitespace is unimportant in assembly code. Also, move VAX definition of typedef slock_t to port header files to be like all the other ports. Note that netbsd.h and openbsd.h are now identical, and I rather think that freebsd.h is broken in the places where it doesn't agree --- but I'll leave it to the freebsders to look at that.
2001-01-20 01:03:55 +01:00
__asm__ __volatile__(
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
" instruction \n"
" instruction \n"
" instruction \n"
: "=r"(_res), "+m"(*lock) // return register, in/out lock value
: "r"(lock) // lock pointer, in input register
: "memory", "cc"); // show clobbered registers here
* The output-operands list (after first colon) should always include
* "+m"(*lock), whether or not the asm code actually refers to this
* operand directly. This ensures that gcc believes the value in the
* lock variable is used and set by the asm code. Also, the clobbers
* list (after third colon) should always include "memory"; this prevents
* gcc from thinking it can cache the values of shared-memory fields
* across the asm code. Add "cc" if your asm code changes the condition
* code register, and also list any temp registers the code uses.
*----------
New ASM format: /* * Standard __asm__ format: * * __asm__( * "command;" * "command;" * "command;" * : "=r"(_res) return value, in register * : "r"(lock) argument, 'lock pointer', in register * : "r0"); inline code uses this register */
2001-01-19 03:58:59 +01:00
*/
From: Jeroen van Vianen <jeroenv@design.nl> Attached patch will add a version() function to Postges, e.g. template1=> select version(); version ------------------------------------------------------------ PostgreSQL 6.3.2 on i586-pc-linux-gnu, compiled by gcc 2.8.1 (1 row)
1998-04-29 14:41:29 +02:00
Update ASM comments.
2005-12-17 21:15:43 +01:00
#ifdef __i386__ /* 32-bit i386 */
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
#define HAS_TEST_AND_SET
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
typedef unsigned char slock_t;
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
#define TAS(lock) tas(lock)
static __inline__ int
tas(volatile slock_t *lock)
{
Remove uses of register due to incompatibility with C++17 and up The use in regexec.c could remain, since we only try to keep headers C++ clean. But there really doesn't seem to be a good reason to use register in that spot. Discussion: https://postgr.es/m/20220308185902.ibdqmasoaunzjrfc@alap3.anarazel.de
2022-09-24 21:01:06 +02:00
slock_t _res = 1;
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
Don't use a non-locked pre-test of the spinlock on x86_64 machines. The pre-test has been shown to be a big loss on Opterons and at best a wash on EM64T.
2005-10-11 22:01:30 +02:00
/*
* Use a non-locking test before asserting the bus lock. Note that the
* extra test appears to be a small loss on some x86 platforms and a small
* win on others; it's by no means clear that we should keep it.
Use a non-locking initial test in TAS_SPIN on x86_64. Testing done in 2011 by Tom Lane concluded that this is a win on Intel Xeons and AMD Opterons, but it was not changed back then, because of an old comment in tas() that suggested that it's a huge loss on older Opterons. However, didn't have separate TAS() and TAS_SPIN() macros back then, so the comment referred to doing a non-locked initial test even on the first access, in uncontended case. I don't have access to older Opterons, but I'm pretty sure that doing an initial unlocked test is unlikely to be a loss while spinning, even though it might be for the first access. We probably should do the same on 32-bit x86, but I'm afraid of changing it without any testing. Hence just add a note to the x86 implementation suggesting that we probably should do the same there.
2013-08-29 12:48:59 +02:00
*
* When this was last tested, we didn't have separate TAS() and TAS_SPIN()
* macros. Nowadays it probably would be better to do a non-locking test
* in TAS_SPIN() but not in TAS(), like on x86_64, but no-one's done the
* testing to verify that. Without some empirical evidence, better to
* leave it alone.
Don't use a non-locked pre-test of the spinlock on x86_64 machines. The pre-test has been shown to be a big loss on Opterons and at best a wash on EM64T.
2005-10-11 22:01:30 +02:00
*/
Add __volatile__ to all __asm__ and make consistent indenting
2001-01-19 04:58:35 +01:00
__asm__ __volatile__(
Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom.
2003-12-27 21:58:58 +01:00
" cmpb $0,%1 \n"
" jne 1f \n"
Still further tweaking of s_lock assembler: do not assume that leading whitespace is unimportant in assembly code. Also, move VAX definition of typedef slock_t to port header files to be like all the other ports. Note that netbsd.h and openbsd.h are now identical, and I rather think that freebsd.h is broken in the places where it doesn't agree --- but I'll leave it to the freebsders to look at that.
2001-01-20 01:03:55 +01:00
" lock \n"
" xchgb %0,%1 \n"
Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom.
2003-12-27 21:58:58 +01:00
"1: \n"
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
: "+q"(_res), "+m"(*lock)
Really fix compilation failure on MIPS. I missed an additional colon in previous patch. Oops. to make that mistake less likely in the future, add comments as placeholders for unused inputs and outputs in inline assembly.
2014-11-11 09:25:22 +01:00
: /* no inputs */
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
: "memory", "cc");
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
return (int) _res;
}
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom.
2003-12-27 21:58:58 +01:00
#define SPIN_DELAY() spin_delay()
static __inline__ void
spin_delay(void)
{
Document what the "rep; nop" x86 assembler sequence is actually equivalent to, and what it is intended to do.
2004-10-07 01:41:59 +02:00
/*
* This sequence is equivalent to the PAUSE instruction ("rep" is
* ignored by old IA32 processors if the following instruction is
* not a string operation); the IA-32 Architecture Software
* Developer's Manual, Vol. 3, Section 7.7.2 describes why using
* PAUSE in the inner loop of a spin lock is necessary for good
* performance:
*
* The PAUSE instruction improves the performance of IA-32
* processors supporting Hyper-Threading Technology when
* executing spin-wait loops and other routines where one
* thread is accessing a shared lock or semaphore in a tight
* polling loop. When executing a spin-wait loop, the
* processor can suffer a severe performance penalty when
* exiting the loop because it detects a possible memory order
* violation and flushes the core processor's pipeline. The
* PAUSE instruction provides a hint to the processor that the
* code sequence is a spin-wait loop. The processor uses this
* hint to avoid the memory order violation and prevent the
* pipeline flush. In addition, the PAUSE instruction
* de-pipelines the spin-wait loop to prevent it from
* consuming execution resources excessively.
*/
Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom.
2003-12-27 21:58:58 +01:00
__asm__ __volatile__(
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
" rep; nop \n");
Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom.
2003-12-27 21:58:58 +01:00
}
Don't use a non-locked pre-test of the spinlock on x86_64 machines. The pre-test has been shown to be a big loss on Opterons and at best a wash on EM64T.
2005-10-11 22:01:30 +02:00
#endif /* __i386__ */
#ifdef __x86_64__ /* AMD Opteron, Intel EM64T */
#define HAS_TEST_AND_SET
typedef unsigned char slock_t;
#define TAS(lock) tas(lock)
Use a non-locking initial test in TAS_SPIN on x86_64. Testing done in 2011 by Tom Lane concluded that this is a win on Intel Xeons and AMD Opterons, but it was not changed back then, because of an old comment in tas() that suggested that it's a huge loss on older Opterons. However, didn't have separate TAS() and TAS_SPIN() macros back then, so the comment referred to doing a non-locked initial test even on the first access, in uncontended case. I don't have access to older Opterons, but I'm pretty sure that doing an initial unlocked test is unlikely to be a loss while spinning, even though it might be for the first access. We probably should do the same on 32-bit x86, but I'm afraid of changing it without any testing. Hence just add a note to the x86 implementation suggesting that we probably should do the same there.
2013-08-29 12:48:59 +02:00
/*
* On Intel EM64T, it's a win to use a non-locking test before the xchg proper,
* but only when spinning.
*
* See also Implementing Scalable Atomic Locks for Multi-Core Intel(tm) EM64T
* and IA32, by Michael Chynoweth and Mary R. Lee. As of this writing, it is
* available at:
* http://software.intel.com/en-us/articles/implementing-scalable-atomic-locks-for-multi-core-intel-em64t-and-ia32-architectures
*/
#define TAS_SPIN(lock) (*(lock) ? 1 : TAS(lock))
Don't use a non-locked pre-test of the spinlock on x86_64 machines. The pre-test has been shown to be a big loss on Opterons and at best a wash on EM64T.
2005-10-11 22:01:30 +02:00
static __inline__ int
tas(volatile slock_t *lock)
{
Remove uses of register due to incompatibility with C++17 and up The use in regexec.c could remain, since we only try to keep headers C++ clean. But there really doesn't seem to be a good reason to use register in that spot. Discussion: https://postgr.es/m/20220308185902.ibdqmasoaunzjrfc@alap3.anarazel.de
2022-09-24 21:01:06 +02:00
slock_t _res = 1;
Don't use a non-locked pre-test of the spinlock on x86_64 machines. The pre-test has been shown to be a big loss on Opterons and at best a wash on EM64T.
2005-10-11 22:01:30 +02:00
__asm__ __volatile__(
" lock \n"
" xchgb %0,%1 \n"
: "+q"(_res), "+m"(*lock)
Really fix compilation failure on MIPS. I missed an additional colon in previous patch. Oops. to make that mistake less likely in the future, add comments as placeholders for unused inputs and outputs in inline assembly.
2014-11-11 09:25:22 +01:00
: /* no inputs */
Don't use a non-locked pre-test of the spinlock on x86_64 machines. The pre-test has been shown to be a big loss on Opterons and at best a wash on EM64T.
2005-10-11 22:01:30 +02:00
: "memory", "cc");
return (int) _res;
}
#define SPIN_DELAY() spin_delay()
static __inline__ void
spin_delay(void)
{
/*
* Adding a PAUSE in the spin delay loop is demonstrably a no-op on
* Opteron, but it may be of some use on EM64T, so we keep it.
*/
__asm__ __volatile__(
" rep; nop \n");
}
#endif /* __x86_64__ */
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
Add ARM64 (aarch64) support to s_lock.h. Use the same gcc atomic functions as we do on newer ARM chips. (Basically this is a copy and paste of the __arm__ code block, but omitting the SWPB option since that definitely won't work.) Back-patch to 9.2. The patch would work further back, but we'd also need to update config.guess/config.sub in older branches to make them build out-of-the-box, and there hasn't been demand for it. Mark Salter
2013-06-04 21:42:02 +02:00
/*
Remove swpb-based spinlock implementation for ARMv5 and earlier. Per recent analysis by Andres Freund, this implementation is in fact unsafe, because ARMv5 has weak memory ordering, which means tha the CPU could move loads or stores across the volatile store performed by the default S_UNLOCK. We could try to fix this, but have no ARMv5 hardware to test on, so removing support seems better. We can still support ARMv5 systems on GCC versions new enough to have built-in atomics support for this platform, and can also re-add support for the old way if someone has hardware that can be used to test a fix. However, since the requirement to use a relatively-new GCC hasn't been an issue for ARMv6 or ARMv7, which lack the swpb instruction altogether, perhaps it won't be an issue for ARMv5 either.
2014-07-06 20:52:25 +02:00
* On ARM and ARM64, we use __sync_lock_test_and_set(int *, int) if available.
*
* We use the int-width variant of the builtin because it works on more chips
* than other widths.
Add ARM64 (aarch64) support to s_lock.h. Use the same gcc atomic functions as we do on newer ARM chips. (Basically this is a copy and paste of the __arm__ code block, but omitting the SWPB option since that definitely won't work.) Back-patch to 9.2. The patch would work further back, but we'd also need to update config.guess/config.sub in older branches to make them build out-of-the-box, and there hasn't been demand for it. Mark Salter
2013-06-04 21:42:02 +02:00
*/
Remove unused symbol __aarch64 This was added as a possible variant of __aarch64__ back when 64-bit ARM was vaporware. It hasn't shown up in the wild since then, so remove. Nathan Bossart Discussion: https://www.postgresql.org/message-id/CAFBsxsEN5nW3uRh%3Djrs-QexDrC1btu0ZfriD3FFfb%3D3J6tAngg%40mail.gmail.com
2022-08-25 08:37:40 +02:00
#if defined(__arm__) || defined(__arm) || defined(__aarch64__)
Add a basic atomic ops API abstracting away platform/architecture details. Several upcoming performance/scalability improvements require atomic operations. This new API avoids the need to splatter compiler and architecture dependent code over all the locations employing atomic ops. For several of the potential usages it'd be problematic to maintain both, a atomics using implementation and one using spinlocks or similar. In all likelihood one of the implementations would not get tested regularly under concurrency. To avoid that scenario the new API provides a automatic fallback of atomic operations to spinlocks. All properties of atomic operations are maintained. This fallback - obviously - isn't as fast as just using atomic ops, but it's not bad either. For one of the future users the atomics ontop spinlocks implementation was actually slightly faster than the old purely spinlock using implementation. That's important because it reduces the fear of regressing older platforms when improving the scalability for new ones. The API, loosely modeled after the C11 atomics support, currently provides 'atomic flags' and 32 bit unsigned integers. If the platform efficiently supports atomic 64 bit unsigned integers those are also provided. To implement atomics support for a platform/architecture/compiler for a type of atomics 32bit compare and exchange needs to be implemented. If available and more efficient native support for flags, 32 bit atomic addition, and corresponding 64 bit operations may also be provided. Additional useful atomic operations are implemented generically ontop of these. The implementation for various versions of gcc, msvc and sun studio have been tested. Additional existing stub implementations for * Intel icc * HUPX acc * IBM xlc are included but have never been tested. These will likely require fixes based on buildfarm and user feedback. As atomic operations also require barriers for some operations the existing barrier support has been moved into the atomics code. Author: Andres Freund with contributions from Oskari Saarenmaa Reviewed-By: Amit Kapila, Robert Haas, Heikki Linnakangas and Álvaro Herrera Discussion: CA+TgmoYBW+ux5-8Ja=Mcyuy8=VXAnVRHp3Kess6Pn3DMXAPAEA@mail.gmail.com, 20131015123303.GH5300@awork2.anarazel.de, 20131028205522.GI20248@awork2.anarazel.de
2014-09-25 23:49:05 +02:00
#ifdef HAVE_GCC__SYNC_INT32_TAS
Add ARM64 (aarch64) support to s_lock.h. Use the same gcc atomic functions as we do on newer ARM chips. (Basically this is a copy and paste of the __arm__ code block, but omitting the SWPB option since that definitely won't work.) Back-patch to 9.2. The patch would work further back, but we'd also need to update config.guess/config.sub in older branches to make them build out-of-the-box, and there hasn't been demand for it. Mark Salter
2013-06-04 21:42:02 +02:00
#define HAS_TEST_AND_SET
#define TAS(lock) tas(lock)
typedef int slock_t;
static __inline__ int
tas(volatile slock_t *lock)
{
return __sync_lock_test_and_set(lock, 1);
}
#define S_UNLOCK(lock) __sync_lock_release(lock)
Use ISB as a spin-delay instruction on ARM64. This seems beneficial on high-core-count machines, and not harmful on lesser hardware. However, older ARM32 gear doesn't have this instruction, so restrict the patch to ARM64. Geoffrey Blake Discussion: https://postgr.es/m/78338F29-9D7F-4DC8-BD71-E9674CE71425@amazon.com
2022-04-07 00:57:57 +02:00
/*
* Using an ISB instruction to delay in spinlock loops appears beneficial on
* high-core-count ARM64 processors. It seems mostly a wash for smaller gear,
* and ISB doesn't exist at all on pre-v7 ARM chips.
*/
Remove unused symbol __aarch64 This was added as a possible variant of __aarch64__ back when 64-bit ARM was vaporware. It hasn't shown up in the wild since then, so remove. Nathan Bossart Discussion: https://www.postgresql.org/message-id/CAFBsxsEN5nW3uRh%3Djrs-QexDrC1btu0ZfriD3FFfb%3D3J6tAngg%40mail.gmail.com
2022-08-25 08:37:40 +02:00
#if defined(__aarch64__)
Use ISB as a spin-delay instruction on ARM64. This seems beneficial on high-core-count machines, and not harmful on lesser hardware. However, older ARM32 gear doesn't have this instruction, so restrict the patch to ARM64. Geoffrey Blake Discussion: https://postgr.es/m/78338F29-9D7F-4DC8-BD71-E9674CE71425@amazon.com
2022-04-07 00:57:57 +02:00
#define SPIN_DELAY() spin_delay()
static __inline__ void
spin_delay(void)
{
__asm__ __volatile__(
" isb; \n");
}
Remove unused symbol __aarch64 This was added as a possible variant of __aarch64__ back when 64-bit ARM was vaporware. It hasn't shown up in the wild since then, so remove. Nathan Bossart Discussion: https://www.postgresql.org/message-id/CAFBsxsEN5nW3uRh%3Djrs-QexDrC1btu0ZfriD3FFfb%3D3J6tAngg%40mail.gmail.com
2022-08-25 08:37:40 +02:00
#endif /* __aarch64__ */
Add a basic atomic ops API abstracting away platform/architecture details. Several upcoming performance/scalability improvements require atomic operations. This new API avoids the need to splatter compiler and architecture dependent code over all the locations employing atomic ops. For several of the potential usages it'd be problematic to maintain both, a atomics using implementation and one using spinlocks or similar. In all likelihood one of the implementations would not get tested regularly under concurrency. To avoid that scenario the new API provides a automatic fallback of atomic operations to spinlocks. All properties of atomic operations are maintained. This fallback - obviously - isn't as fast as just using atomic ops, but it's not bad either. For one of the future users the atomics ontop spinlocks implementation was actually slightly faster than the old purely spinlock using implementation. That's important because it reduces the fear of regressing older platforms when improving the scalability for new ones. The API, loosely modeled after the C11 atomics support, currently provides 'atomic flags' and 32 bit unsigned integers. If the platform efficiently supports atomic 64 bit unsigned integers those are also provided. To implement atomics support for a platform/architecture/compiler for a type of atomics 32bit compare and exchange needs to be implemented. If available and more efficient native support for flags, 32 bit atomic addition, and corresponding 64 bit operations may also be provided. Additional useful atomic operations are implemented generically ontop of these. The implementation for various versions of gcc, msvc and sun studio have been tested. Additional existing stub implementations for * Intel icc * HUPX acc * IBM xlc are included but have never been tested. These will likely require fixes based on buildfarm and user feedback. As atomic operations also require barriers for some operations the existing barrier support has been moved into the atomics code. Author: Andres Freund with contributions from Oskari Saarenmaa Reviewed-By: Amit Kapila, Robert Haas, Heikki Linnakangas and Álvaro Herrera Discussion: CA+TgmoYBW+ux5-8Ja=Mcyuy8=VXAnVRHp3Kess6Pn3DMXAPAEA@mail.gmail.com, 20131015123303.GH5300@awork2.anarazel.de, 20131028205522.GI20248@awork2.anarazel.de
2014-09-25 23:49:05 +02:00
#endif /* HAVE_GCC__SYNC_INT32_TAS */
Remove unused symbol __aarch64 This was added as a possible variant of __aarch64__ back when 64-bit ARM was vaporware. It hasn't shown up in the wild since then, so remove. Nathan Bossart Discussion: https://www.postgresql.org/message-id/CAFBsxsEN5nW3uRh%3Djrs-QexDrC1btu0ZfriD3FFfb%3D3J6tAngg%40mail.gmail.com
2022-08-25 08:37:40 +02:00
#endif /* __arm__ || __arm || __aarch64__ */
Add ARM64 (aarch64) support to s_lock.h. Use the same gcc atomic functions as we do on newer ARM chips. (Basically this is a copy and paste of the __arm__ code block, but omitting the SWPB option since that definitely won't work.) Back-patch to 9.2. The patch would work further back, but we'd also need to update config.guess/config.sub in older branches to make them build out-of-the-box, and there hasn't been demand for it. Mark Salter
2013-06-04 21:42:02 +02:00
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
/* S/390 and S/390x Linux (32- and 64-bit zSeries) */
Update ASM comments.
2005-12-17 21:15:43 +01:00
#if defined(__s390__) || defined(__s390x__)
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
#define HAS_TEST_AND_SET
typedef unsigned int slock_t;
TAS code originally written for s390 (32-bit) does not work for s390x (64-bit). Fix it. Per report from Permaine Cheung.
2002-11-22 02:13:16 +01:00
#define TAS(lock) tas(lock)
static __inline__ int
tas(volatile slock_t *lock)
{
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
int _res = 0;
TAS code originally written for s390 (32-bit) does not work for s390x (64-bit). Fix it. Per report from Permaine Cheung.
2002-11-22 02:13:16 +01:00
__asm__ __volatile__(
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
" cs %0,%3,0(%2) \n"
: "+d"(_res), "+m"(*lock)
: "a"(lock), "d"(1)
: "memory", "cc");
return _res;
TAS code originally written for s390 (32-bit) does not work for s390x (64-bit). Fix it. Per report from Permaine Cheung.
2002-11-22 02:13:16 +01:00
}
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
#endif /* __s390__ || __s390x__ */
TAS code originally written for s390 (32-bit) does not work for s390x (64-bit). Fix it. Per report from Permaine Cheung.
2002-11-22 02:13:16 +01:00
Add ARM32 support by Andrew McMurry
1999-04-13 19:42:26 +02:00
Update ASM comments.
2005-12-17 21:15:43 +01:00
#if defined(__sparc__) /* Sparc */
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
/*
* Solaris has always run sparc processors in TSO (total store) mode, but
* linux didn't use to and the *BSDs still don't. So, be careful about
Fix some typos and some incorrectly duplicated words Author: Justin Pryzby Reviewed-by: David Rowley Discussion: https://postgr.es/m/ZD3D1QxoccnN8A1V@telsasoft.com
2023-04-18 04:03:30 +02:00
* acquire/release semantics. The CPU will treat superfluous members as
Fix various typos and grammar errors in comments. Author: Dmitriy Olshevskiy Discussion: 553D00A6.4090205@bk.ru
2015-04-26 18:42:31 +02:00
* NOPs, so it's just code space.
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
*/
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
#define HAS_TEST_AND_SET
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
typedef unsigned char slock_t;
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
#define TAS(lock) tas(lock)
static __inline__ int
tas(volatile slock_t *lock)
{
Remove uses of register due to incompatibility with C++17 and up The use in regexec.c could remain, since we only try to keep headers C++ clean. But there really doesn't seem to be a good reason to use register in that spot. Discussion: https://postgr.es/m/20220308185902.ibdqmasoaunzjrfc@alap3.anarazel.de
2022-09-24 21:01:06 +02:00
slock_t _res;
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
Add comments that Solaris Sun compiler only supports sparc9 ASM,
2006-05-18 01:57:03 +02:00
/*
Update stale comments, and fix comment typos.
2019-06-08 19:12:26 +02:00
* See comment in src/backend/port/tas/sunstudio_sparc.s for why this
Mention that gcc/sparc generates sparcv7 binaries.
2006-05-18 18:02:30 +02:00
* uses "ldstub", and that file uses "cas". gcc currently generates
* sparcv7-targeted binaries, so "cas" use isn't possible.
Add comments that Solaris Sun compiler only supports sparc9 ASM,
2006-05-18 01:57:03 +02:00
*/
Add __volatile__ to all __asm__ and make consistent indenting
2001-01-19 04:58:35 +01:00
__asm__ __volatile__(
Still further tweaking of s_lock assembler: do not assume that leading whitespace is unimportant in assembly code. Also, move VAX definition of typedef slock_t to port header files to be like all the other ports. Note that netbsd.h and openbsd.h are now identical, and I rather think that freebsd.h is broken in the places where it doesn't agree --- but I'll leave it to the freebsders to look at that.
2001-01-20 01:03:55 +01:00
" ldstub [%2], %0 \n"
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
: "=r"(_res), "+m"(*lock)
: "r"(lock)
: "memory");
Accept alternate spellings of __sparcv7 and __sparcv8. Apparently some versions of gcc prefer __sparc_v7__ and __sparc_v8__. Per report from Waldemar Brodkorb.
2015-08-10 23:34:51 +02:00
#if defined(__sparcv7) || defined(__sparc_v7__)
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
/*
* No stbar or membar available, luckily no actually produced hardware
* requires a barrier.
*/
Accept alternate spellings of __sparcv7 and __sparcv8. Apparently some versions of gcc prefer __sparc_v7__ and __sparc_v8__. Per report from Waldemar Brodkorb.
2015-08-10 23:34:51 +02:00
#elif defined(__sparcv8) || defined(__sparc_v8__)
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
/* stbar is available (and required for both PSO, RMO), membar isn't */
__asm__ __volatile__ ("stbar \n":::"memory");
#else
/*
* #LoadStore (RMO) | #LoadLoad (RMO) together are the appropriate acquire
* barrier for sparcv8+ upwards.
*/
__asm__ __volatile__ ("membar #LoadStore | #LoadLoad \n":::"memory");
#endif
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
return (int) _res;
}
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
Accept alternate spellings of __sparcv7 and __sparcv8. Apparently some versions of gcc prefer __sparc_v7__ and __sparc_v8__. Per report from Waldemar Brodkorb.
2015-08-10 23:34:51 +02:00
#if defined(__sparcv7) || defined(__sparc_v7__)
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
/*
* No stbar or membar available, luckily no actually produced hardware
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
* requires a barrier. We fall through to the default gcc definition of
* S_UNLOCK in this case.
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
*/
Accept alternate spellings of __sparcv7 and __sparcv8. Apparently some versions of gcc prefer __sparc_v7__ and __sparc_v8__. Per report from Waldemar Brodkorb.
2015-08-10 23:34:51 +02:00
#elif defined(__sparcv8) || defined(__sparc_v8__)
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
/* stbar is available (and required for both PSO, RMO), membar isn't */
#define S_UNLOCK(lock) \
do \
{ \
__asm__ __volatile__ ("stbar \n":::"memory"); \
*((volatile slock_t *) (lock)) = 0; \
} while (0)
#else
/*
* #LoadStore (RMO) | #StoreStore (RMO, PSO) together are the appropriate
* release barrier for sparcv8+ upwards.
*/
Fix typo in solaris spinlock fix. 07968dbfaad03 missed part of the S_UNLOCK define when building for sparcv8+.
2014-09-09 13:57:38 +02:00
#define S_UNLOCK(lock) \
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
do \
{ \
__asm__ __volatile__ ("membar #LoadStore | #StoreStore \n":::"memory"); \
*((volatile slock_t *) (lock)) = 0; \
} while (0)
#endif
Redefine cpu's as __cpu__. Only for 6.6 branch.
1999-07-13 22:00:37 +02:00
#endif /* __sparc__ */
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
Update ASM comments.
2005-12-17 21:15:43 +01:00
/* PowerPC */
Recognize __ppc64__, which seems to be Apple's spelling of the predefined symbol for PPC64 hardware. I hadn't known that Apple supported PPC64 at all, but darn if there aren't 64-bit variant libraries in OS X as well as support in their gcc.
2006-04-20 01:11:15 +02:00
#if defined(__ppc__) || defined(__powerpc__) || defined(__ppc64__) || defined(__powerpc64__)
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
#define HAS_TEST_AND_SET
typedef unsigned int slock_t;
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Apple's assembler likes the inlined TAS syntax too, so no reason to maintain a separate out-of-line version of PPC tas() anymore. Also fix S_UNLOCK for __powerpc64__ platforms.
2003-04-20 23:54:34 +02:00
#define TAS(lock) tas(lock)
Use 4-byte slock_t on both PPC and PPC64. Previously we defined slock_t as 8 bytes on PPC64, but the TAS assembly code uses word-wide operations regardless, so that the second word was just wasted space. There doesn't appear to be any performance benefit in adding the second word, so get rid of it to simplify the code.
2012-01-02 04:44:01 +01:00
Use a non-locking initial test in TAS_SPIN on PPC. Further testing convinces me that this is helpful at sufficiently high contention levels, though it's still worrisome that it loses slightly at lower contention levels. Per Manabu Ori.
2012-01-03 22:00:06 +01:00
/* On PPC, it's a win to use a non-locking test before the lwarx */
#define TAS_SPIN(lock) (*(lock) ? 1 : TAS(lock))
Apple's assembler likes the inlined TAS syntax too, so no reason to maintain a separate out-of-line version of PPC tas() anymore. Also fix S_UNLOCK for __powerpc64__ platforms.
2003-04-20 23:54:34 +02:00
/*
For PowerPC instruction "addi", use constraint "b". Without "b", a variant of the tas() code miscompiles on macOS 10.4. This may also fix a compilation failure involving macOS 10.1. Today's compilers have been allocating acceptable registers with or without this change, but this future-proofs the code by precisely conveying the acceptable registers. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. Discussion: https://postgr.es/m/20191009063900.GA4066266@rfd.leadboat.com
2019-10-19 05:20:28 +02:00
* The second operand of addi can hold a constant zero or a register number,
* hence constraint "=&b" to avoid allocating r0. "b" stands for "address
* base register"; most operands having this register-or-zero property are
* address bases, e.g. the second operand of lwax.
*
Apple's assembler likes the inlined TAS syntax too, so no reason to maintain a separate out-of-line version of PPC tas() anymore. Also fix S_UNLOCK for __powerpc64__ platforms.
2003-04-20 23:54:34 +02:00
* NOTE: per the Enhanced PowerPC Architecture manual, v1.0 dated 7-May-2002,
* an isync is a sufficient synchronization barrier after a lwarx/stwcx loop.
Remove configurability of PPC spinlock assembly code. Assume that we can use LWARX hint flags and the LWSYNC instruction on any PPC machine. The check on the assembler's behavior was only needed for Apple's old assembler, which is no longer of interest now that we've de-supported all PPC-era versions of macOS (thanks to them not having clock_gettime()). Also, given an up-to-date assembler these instructions work even on Apple's old hardware. It seems quite unlikely that anyone would be interested in running current Postgres on PPC hardware that's so old as to not have these instructions. Hence, rip out associated configure test and manual configuration options, and just use the modernized instructions all the time. Also, update atomics/arch-ppc.h to use these instructions as well. (It was already using LWSYNC unconditionally in another place, providing further proof that nobody is using PG on hardware old enough to have a problem with that.) Discussion: https://postgr.es/m/166622.1660323391@sss.pgh.pa.us
2022-08-13 19:36:39 +02:00
* But if the spinlock is in ordinary memory, we can use lwsync instead for
* better performance.
Fix s_lock.h PPC assembly code to be compatible with native AIX assembler. On recent AIX it's necessary to configure gcc to use the native assembler (because the GNU assembler hasn't been updated to handle AIX 6+). This caused PG builds to fail with assembler syntax errors, because we'd try to compile s_lock.h's gcc asm fragment for PPC, and that assembly code relied on GNU-style local labels. We can't substitute normal labels because it would fail in any file containing more than one inlined use of tas(). Fortunately, that code is stable enough, and the PPC ISA is simple enough, that it doesn't seem like too much of a maintenance burden to just hand-code the branch offsets, removing the need for any labels. Note that the AIX assembler only accepts "$" for the location counter pseudo-symbol. The usual GNU convention is "."; but it appears that all versions of gas for PPC also accept "$", so in theory this patch will not break any other PPC platforms. This has been reported by a few people, but Steve Underwood gets the credit for being the first to pursue the problem far enough to understand why it was failing. Thanks also to Noah Misch for additional testing.
2015-08-29 22:09:25 +02:00
*
* Ordinarily, we'd code the branches here using GNU-style local symbols, that
* is "1f" referencing "1:" and so on. But some people run gcc on AIX with
* IBM's assembler as backend, and IBM's assembler doesn't do local symbols.
* So hand-code the branch offsets; fortunately, all PPC instructions are
* exactly 4 bytes each, so it's not too hard to count.
Apple's assembler likes the inlined TAS syntax too, so no reason to maintain a separate out-of-line version of PPC tas() anymore. Also fix S_UNLOCK for __powerpc64__ platforms.
2003-04-20 23:54:34 +02:00
*/
> > I'll re-check that with the ppc architecture guy here. > > ... he is now about to write an inlined version that can go into > s_lock.h . I'll send the new patch later on... OK, here it comes: An inlined version of tas(), that works for both, powerpc and powerpc64. The patch is against 7.3b5 and passes the test suite on both architectures. Reinhard Max
2002-11-10 01:33:43 +01:00
static __inline__ int
tas(volatile slock_t *lock)
{
slock_t _t;
int _res;
__asm__ __volatile__(
Use mutex hint bit in PPC LWARX instructions, where possible. The hint bit makes for a small but measurable performance improvement in access to contended spinlocks. On the other hand, some PPC chips give an illegal-instruction failure. There doesn't seem to be a completely bulletproof way to tell whether the hint bit will cause an illegal-instruction failure other than by trying it; but most if not all 64-bit PPC machines should accept it, so follow the Linux kernel's lead and assume it's okay to use it in 64-bit builds. Of course we must also check whether the assembler accepts the command, since even with a recent CPU the toolchain could be old. Patch by Manabu Ori, significantly modified by me.
2012-01-02 04:39:59 +01:00
" lwarx %0,0,%3,1 \n"
> > I'll re-check that with the ppc architecture guy here. > > ... he is now about to write an inlined version that can go into > s_lock.h . I'll send the new patch later on... OK, here it comes: An inlined version of tas(), that works for both, powerpc and powerpc64. The patch is against 7.3b5 and passes the test suite on both architectures. Reinhard Max
2002-11-10 01:33:43 +01:00
" cmpwi %0,0 \n"
Fix s_lock.h PPC assembly code to be compatible with native AIX assembler. On recent AIX it's necessary to configure gcc to use the native assembler (because the GNU assembler hasn't been updated to handle AIX 6+). This caused PG builds to fail with assembler syntax errors, because we'd try to compile s_lock.h's gcc asm fragment for PPC, and that assembly code relied on GNU-style local labels. We can't substitute normal labels because it would fail in any file containing more than one inlined use of tas(). Fortunately, that code is stable enough, and the PPC ISA is simple enough, that it doesn't seem like too much of a maintenance burden to just hand-code the branch offsets, removing the need for any labels. Note that the AIX assembler only accepts "$" for the location counter pseudo-symbol. The usual GNU convention is "."; but it appears that all versions of gas for PPC also accept "$", so in theory this patch will not break any other PPC platforms. This has been reported by a few people, but Steve Underwood gets the credit for being the first to pursue the problem far enough to understand why it was failing. Thanks also to Noah Misch for additional testing.
2015-08-29 22:09:25 +02:00
" bne $+16 \n" /* branch to li %1,1 */
> > I'll re-check that with the ppc architecture guy here. > > ... he is now about to write an inlined version that can go into > s_lock.h . I'll send the new patch later on... OK, here it comes: An inlined version of tas(), that works for both, powerpc and powerpc64. The patch is against 7.3b5 and passes the test suite on both architectures. Reinhard Max
2002-11-10 01:33:43 +01:00
" addi %0,%0,1 \n"
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
" stwcx. %0,0,%3 \n"
Remove configurability of PPC spinlock assembly code. Assume that we can use LWARX hint flags and the LWSYNC instruction on any PPC machine. The check on the assembler's behavior was only needed for Apple's old assembler, which is no longer of interest now that we've de-supported all PPC-era versions of macOS (thanks to them not having clock_gettime()). Also, given an up-to-date assembler these instructions work even on Apple's old hardware. It seems quite unlikely that anyone would be interested in running current Postgres on PPC hardware that's so old as to not have these instructions. Hence, rip out associated configure test and manual configuration options, and just use the modernized instructions all the time. Also, update atomics/arch-ppc.h to use these instructions as well. (It was already using LWSYNC unconditionally in another place, providing further proof that nobody is using PG on hardware old enough to have a problem with that.) Discussion: https://postgr.es/m/166622.1660323391@sss.pgh.pa.us
2022-08-13 19:36:39 +02:00
" beq $+12 \n" /* branch to lwsync */
Fix s_lock.h PPC assembly code to be compatible with native AIX assembler. On recent AIX it's necessary to configure gcc to use the native assembler (because the GNU assembler hasn't been updated to handle AIX 6+). This caused PG builds to fail with assembler syntax errors, because we'd try to compile s_lock.h's gcc asm fragment for PPC, and that assembly code relied on GNU-style local labels. We can't substitute normal labels because it would fail in any file containing more than one inlined use of tas(). Fortunately, that code is stable enough, and the PPC ISA is simple enough, that it doesn't seem like too much of a maintenance burden to just hand-code the branch offsets, removing the need for any labels. Note that the AIX assembler only accepts "$" for the location counter pseudo-symbol. The usual GNU convention is "."; but it appears that all versions of gas for PPC also accept "$", so in theory this patch will not break any other PPC platforms. This has been reported by a few people, but Steve Underwood gets the credit for being the first to pursue the problem far enough to understand why it was failing. Thanks also to Noah Misch for additional testing.
2015-08-29 22:09:25 +02:00
" li %1,1 \n"
" b $+12 \n" /* branch to end of asm sequence */
Use LWSYNC in place of SYNC/ISYNC in PPC spinlocks, where possible. This is allegedly a win, at least on some PPC implementations, according to the PPC ISA documents. However, as with LWARX hints, some PPC platforms give an illegal-instruction failure. Use the same trick as before of assuming that PPC64 platforms will accept it; we might need to refine that based on experience, but there are other projects doing likewise according to google. I did not add an assembler compatibility test because LWSYNC has been around much longer than hint bits, and it seems unlikely that any toolchains currently in use don't recognize it.
2012-01-02 06:01:33 +01:00
" lwsync \n"
Tighten up register usage for inline PPC version of tas().
2003-04-04 08:57:39 +02:00
" li %1,0 \n"
> > I'll re-check that with the ppc architecture guy here. > > ... he is now about to write an inlined version that can go into > s_lock.h . I'll send the new patch later on... OK, here it comes: An inlined version of tas(), that works for both, powerpc and powerpc64. The patch is against 7.3b5 and passes the test suite on both architectures. Reinhard Max
2002-11-10 01:33:43 +01:00
For PowerPC instruction "addi", use constraint "b". Without "b", a variant of the tas() code miscompiles on macOS 10.4. This may also fix a compilation failure involving macOS 10.1. Today's compilers have been allocating acceptable registers with or without this change, but this future-proofs the code by precisely conveying the acceptable registers. Back-patch to 9.4 (all supported versions). Reviewed by Tom Lane. Discussion: https://postgr.es/m/20191009063900.GA4066266@rfd.leadboat.com
2019-10-19 05:20:28 +02:00
: "=&b"(_t), "=r"(_res), "+m"(*lock)
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
: "r"(lock)
: "memory", "cc");
> > I'll re-check that with the ppc architecture guy here. > > ... he is now about to write an inlined version that can go into > s_lock.h . I'll send the new patch later on... OK, here it comes: An inlined version of tas(), that works for both, powerpc and powerpc64. The patch is against 7.3b5 and passes the test suite on both architectures. Reinhard Max
2002-11-10 01:33:43 +01:00
return _res;
}
Apple's assembler likes the inlined TAS syntax too, so no reason to maintain a separate out-of-line version of PPC tas() anymore. Also fix S_UNLOCK for __powerpc64__ platforms.
2003-04-20 23:54:34 +02:00
Use LWSYNC in place of SYNC/ISYNC in PPC spinlocks, where possible. This is allegedly a win, at least on some PPC implementations, according to the PPC ISA documents. However, as with LWARX hints, some PPC platforms give an illegal-instruction failure. Use the same trick as before of assuming that PPC64 platforms will accept it; we might need to refine that based on experience, but there are other projects doing likewise according to google. I did not add an assembler compatibility test because LWSYNC has been around much longer than hint bits, and it seems unlikely that any toolchains currently in use don't recognize it.
2012-01-02 06:01:33 +01:00
/*
* PowerPC S_UNLOCK is almost standard but requires a "sync" instruction.
Remove configurability of PPC spinlock assembly code. Assume that we can use LWARX hint flags and the LWSYNC instruction on any PPC machine. The check on the assembler's behavior was only needed for Apple's old assembler, which is no longer of interest now that we've de-supported all PPC-era versions of macOS (thanks to them not having clock_gettime()). Also, given an up-to-date assembler these instructions work even on Apple's old hardware. It seems quite unlikely that anyone would be interested in running current Postgres on PPC hardware that's so old as to not have these instructions. Hence, rip out associated configure test and manual configuration options, and just use the modernized instructions all the time. Also, update atomics/arch-ppc.h to use these instructions as well. (It was already using LWSYNC unconditionally in another place, providing further proof that nobody is using PG on hardware old enough to have a problem with that.) Discussion: https://postgr.es/m/166622.1660323391@sss.pgh.pa.us
2022-08-13 19:36:39 +02:00
* But we can use lwsync instead for better performance.
Use LWSYNC in place of SYNC/ISYNC in PPC spinlocks, where possible. This is allegedly a win, at least on some PPC implementations, according to the PPC ISA documents. However, as with LWARX hints, some PPC platforms give an illegal-instruction failure. Use the same trick as before of assuming that PPC64 platforms will accept it; we might need to refine that based on experience, but there are other projects doing likewise according to google. I did not add an assembler compatibility test because LWSYNC has been around much longer than hint bits, and it seems unlikely that any toolchains currently in use don't recognize it.
2012-01-02 06:01:33 +01:00
*/
#define S_UNLOCK(lock) \
do \
{ \
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
__asm__ __volatile__ (" lwsync \n" ::: "memory"); \
Use LWSYNC in place of SYNC/ISYNC in PPC spinlocks, where possible. This is allegedly a win, at least on some PPC implementations, according to the PPC ISA documents. However, as with LWARX hints, some PPC platforms give an illegal-instruction failure. Use the same trick as before of assuming that PPC64 platforms will accept it; we might need to refine that based on experience, but there are other projects doing likewise according to google. I did not add an assembler compatibility test because LWSYNC has been around much longer than hint bits, and it seems unlikely that any toolchains currently in use don't recognize it.
2012-01-02 06:01:33 +01:00
*((volatile slock_t *) (lock)) = 0; \
} while (0)
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Apple's assembler likes the inlined TAS syntax too, so no reason to maintain a separate out-of-line version of PPC tas() anymore. Also fix S_UNLOCK for __powerpc64__ platforms.
2003-04-20 23:54:34 +02:00
#endif /* powerpc */
> > I'll re-check that with the ppc architecture guy here. > > ... he is now about to write an inlined version that can go into > s_lock.h . I'll send the new patch later on... OK, here it comes: An inlined version of tas(), that works for both, powerpc and powerpc64. The patch is against 7.3b5 and passes the test suite on both architectures. Reinhard Max
2002-11-10 01:33:43 +01:00
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
Update ASM comments.
2005-12-17 21:15:43 +01:00
#if defined(__mips__) && !defined(__sgi) /* non-SGI MIPS */
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
#define HAS_TEST_AND_SET
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
typedef unsigned int slock_t;
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
#define TAS(lock) tas(lock)
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Fix spinlock assembly code for MIPS so it works on MIPS r6. Original MIPS-I processors didn't have the LL/SC instructions (nor any other userland synchronization primitive). If the build toolchain targets that ISA variant by default, as an astonishingly large fraction of MIPS platforms still do, the assembler won't take LL/SC without coercion in the form of a ".set mips2" instruction. But we issued that unconditionally, making it an ISA downgrade for chips later than MIPS2. That breaks things for the latest MIPS r6 ISA, which encodes these instructions differently. Adjust the code so we don't change ISA level if it's >= 2. Note that this patch doesn't change what happens on an actual MIPS-I processor: either the kernel will emulate these instructions transparently, or you'll get a SIGILL failure. That tradeoff seemed fine in 2002 when this code was added (cf 3cbe6b247), and it's even more so today when MIPS-I is basically extinct. But let's add a comment about that. YunQiang Su (with cosmetic adjustments by me). Back-patch to all supported branches. Discussion: https://postgr.es/m/15844-8f62fe7e163939b3@postgresql.org
2019-06-23 02:31:50 +02:00
/*
* Original MIPS-I processors lacked the LL/SC instructions, but if we are
* so unfortunate as to be running on one of those, we expect that the kernel
* will handle the illegal-instruction traps and emulate them for us. On
* anything newer (and really, MIPS-I is extinct) LL/SC is the only sane
* choice because any other synchronization method must involve a kernel
* call. Unfortunately, many toolchains still default to MIPS-I as the
* codegen target; if the symbol __mips shows that that's the case, we
* have to force the assembler to accept LL/SC.
*
* R10000 and up processors require a separate SYNC, which has the same
* issues as LL/SC.
*/
#if __mips < 2
#define MIPS_SET_MIPS2 " .set mips2 \n"
#else
#define MIPS_SET_MIPS2
#endif
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
static __inline__ int
tas(volatile slock_t *lock)
{
Remove uses of register due to incompatibility with C++17 and up The use in regexec.c could remain, since we only try to keep headers C++ clean. But there really doesn't seem to be a good reason to use register in that spot. Discussion: https://postgr.es/m/20220308185902.ibdqmasoaunzjrfc@alap3.anarazel.de
2022-09-24 21:01:06 +02:00
volatile slock_t *_l = lock;
int _res;
int _tmp;
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
__asm__ __volatile__(
" .set push \n"
Fix spinlock assembly code for MIPS so it works on MIPS r6. Original MIPS-I processors didn't have the LL/SC instructions (nor any other userland synchronization primitive). If the build toolchain targets that ISA variant by default, as an astonishingly large fraction of MIPS platforms still do, the assembler won't take LL/SC without coercion in the form of a ".set mips2" instruction. But we issued that unconditionally, making it an ISA downgrade for chips later than MIPS2. That breaks things for the latest MIPS r6 ISA, which encodes these instructions differently. Adjust the code so we don't change ISA level if it's >= 2. Note that this patch doesn't change what happens on an actual MIPS-I processor: either the kernel will emulate these instructions transparently, or you'll get a SIGILL failure. That tradeoff seemed fine in 2002 when this code was added (cf 3cbe6b247), and it's even more so today when MIPS-I is basically extinct. But let's add a comment about that. YunQiang Su (with cosmetic adjustments by me). Back-patch to all supported branches. Discussion: https://postgr.es/m/15844-8f62fe7e163939b3@postgresql.org
2019-06-23 02:31:50 +02:00
MIPS_SET_MIPS2
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
" .set noreorder \n"
" .set nomacro \n"
Another try at the inlined MIPS spinlock code. Can't test this myself, but for sure it's not any more broken than the prior version.
2005-08-27 00:04:42 +02:00
" ll %0, %2 \n"
Get the MIPS assembler syntax right. Also add a separate sync command; the reference I consulted yesterday said SC does a SYNC, but apparently this is not true on newer MIPS processors, so be safe.
2005-08-27 18:22:48 +02:00
" or %1, %0, 1 \n"
Another try at the inlined MIPS spinlock code. Can't test this myself, but for sure it's not any more broken than the prior version.
2005-08-27 00:04:42 +02:00
" sc %1, %2 \n"
Get the MIPS assembler syntax right. Also add a separate sync command; the reference I consulted yesterday said SC does a SYNC, but apparently this is not true on newer MIPS processors, so be safe.
2005-08-27 18:22:48 +02:00
" xori %1, 1 \n"
Another try at the inlined MIPS spinlock code. Can't test this myself, but for sure it's not any more broken than the prior version.
2005-08-27 00:04:42 +02:00
" or %0, %0, %1 \n"
Get the MIPS assembler syntax right. Also add a separate sync command; the reference I consulted yesterday said SC does a SYNC, but apparently this is not true on newer MIPS processors, so be safe.
2005-08-27 18:22:48 +02:00
" sync \n"
" .set pop "
Another try at the inlined MIPS spinlock code. Can't test this myself, but for sure it's not any more broken than the prior version.
2005-08-27 00:04:42 +02:00
: "=&r" (_res), "=&r" (_tmp), "+R" (*_l)
Really fix compilation failure on MIPS. I missed an additional colon in previous patch. Oops. to make that mistake less likely in the future, add comments as placeholders for unused inputs and outputs in inline assembly.
2014-11-11 09:25:22 +01:00
: /* no inputs */
Another try at the inlined MIPS spinlock code. Can't test this myself, but for sure it's not any more broken than the prior version.
2005-08-27 00:04:42 +02:00
: "memory");
return _res;
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
}
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Add a SYNC instruction to the S_UNLOCK sequence for MIPS.
2005-08-28 20:26:01 +02:00
/* MIPS S_UNLOCK is almost standard but requires a "sync" instruction */
#define S_UNLOCK(lock) \
do \
Sigh, looks like you need '.set mips2' before you can access MIPS SYNC instruction.
2005-08-29 02:41:34 +02:00
{ \
__asm__ __volatile__( \
" .set push \n" \
Fix spinlock assembly code for MIPS so it works on MIPS r6. Original MIPS-I processors didn't have the LL/SC instructions (nor any other userland synchronization primitive). If the build toolchain targets that ISA variant by default, as an astonishingly large fraction of MIPS platforms still do, the assembler won't take LL/SC without coercion in the form of a ".set mips2" instruction. But we issued that unconditionally, making it an ISA downgrade for chips later than MIPS2. That breaks things for the latest MIPS r6 ISA, which encodes these instructions differently. Adjust the code so we don't change ISA level if it's >= 2. Note that this patch doesn't change what happens on an actual MIPS-I processor: either the kernel will emulate these instructions transparently, or you'll get a SIGILL failure. That tradeoff seemed fine in 2002 when this code was added (cf 3cbe6b247), and it's even more so today when MIPS-I is basically extinct. But let's add a comment about that. YunQiang Su (with cosmetic adjustments by me). Back-patch to all supported branches. Discussion: https://postgr.es/m/15844-8f62fe7e163939b3@postgresql.org
2019-06-23 02:31:50 +02:00
MIPS_SET_MIPS2 \
Sigh, looks like you need '.set mips2' before you can access MIPS SYNC instruction.
2005-08-29 02:41:34 +02:00
" .set noreorder \n" \
" .set nomacro \n" \
" sync \n" \
Fix compilation failure on MIPS. Rémi Zara
2014-11-11 00:04:57 +01:00
" .set pop " \
Really fix compilation failure on MIPS. I missed an additional colon in previous patch. Oops. to make that mistake less likely in the future, add comments as placeholders for unused inputs and outputs in inline assembly.
2014-11-11 09:25:22 +01:00
: /* no outputs */ \
: /* no inputs */ \
Fix compilation failure on MIPS. Rémi Zara
2014-11-11 00:04:57 +01:00
: "memory"); \
Add a SYNC instruction to the S_UNLOCK sequence for MIPS.
2005-08-28 20:26:01 +02:00
*((volatile slock_t *) (lock)) = 0; \
} while (0)
Replace out-of-line tas() assembly code for MIPS with a properly constrained GCC inline version. Thiemo Seufer, by way of Martin Pitt.
2005-08-25 19:17:10 +02:00
#endif /* __mips__ && !__sgi */
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
#if defined(__hppa) || defined(__hppa__) /* HP PA-RISC */
On architectures where we don't have any special inline code for GCC, the inner "#if defined(__GNUC__)" can just be omitted in that architecture's block. The existing arrangement with an outer "#if defined(__GNUC__)" doesn't have any obvious benefit, and it encourages missed cases like this one. BTW, I'd suggest making the definition of clear_lock for HPUX be static const slock_t clear_lock = {{-1, -1, -1, -1}}; The extra braces are needed to suppress warnings from gcc, and declaring it const just seems like good practice. regards, tom lane
1998-07-18 16:51:10 +02:00
/*
Since HPUX now exists for Itanium, we should decouple the assumption that OS=hpux is the same as CPU=hppa. First steps at doing this. With these patches, we still work on hppa with either gcc or HP's cc. We might work on hpux/itanium with gcc, but I can't test it. Definitely will not work on hpux/itanium with non-gcc compiler, for lack of spinlock code.
2003-08-01 21:12:52 +02:00
* HP's PA-RISC
On architectures where we don't have any special inline code for GCC, the inner "#if defined(__GNUC__)" can just be omitted in that architecture's block. The existing arrangement with an outer "#if defined(__GNUC__)" doesn't have any obvious benefit, and it encourages missed cases like this one. BTW, I'd suggest making the definition of clear_lock for HPUX be static const slock_t clear_lock = {{-1, -1, -1, -1}}; The extra braces are needed to suppress warnings from gcc, and declaring it const just seems like good practice. regards, tom lane
1998-07-18 16:51:10 +02:00
*
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
* Because LDCWX requires a 16-byte-aligned address, we declare slock_t as a
* 16-byte struct. The active word in the struct is whichever has the aligned
* address; the other three words just sit at -1.
On architectures where we don't have any special inline code for GCC, the inner "#if defined(__GNUC__)" can just be omitted in that architecture's block. The existing arrangement with an outer "#if defined(__GNUC__)" doesn't have any obvious benefit, and it encourages missed cases like this one. BTW, I'd suggest making the definition of clear_lock for HPUX be static const slock_t clear_lock = {{-1, -1, -1, -1}}; The extra braces are needed to suppress warnings from gcc, and declaring it const just seems like good practice. regards, tom lane
1998-07-18 16:51:10 +02:00
*/
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
#define HAS_TEST_AND_SET
typedef struct
{
int sema[4];
} slock_t;
From: Tom I Helbekkmo <tih@Hamartun.Priv.NO> Apart from this Makefile hack, all I've done is to make dynamically loaded code modules fail properly (as was already done for __mips__, although I think this is too loose: I believe NetBSD for the pmax can do dynamic linking), and to add test-and-set lock handling. As Bruce suggested, this is done in a maximally efficient inlined way: I was not aware that this code was so important, speed-wise.
1998-02-13 06:10:06 +01:00
Redefine Datum as uintptr_t, instead of unsigned long. This is more in keeping with modern practice, and is a first step towards porting to Win64 (which has sizeof(pointer) > sizeof(long)). Tsutomu Yamada, Magnus Hagander, Tom Lane
2009-12-31 20:41:37 +01:00
#define TAS_ACTIVE_WORD(lock) ((volatile int *) (((uintptr_t) (lock) + 15) & ~15))
Use inlined TAS() on PA-RISC, if we are compiling with gcc. Patch inspired by original submission from ViSolve.
2003-12-23 23:15:07 +01:00
static __inline__ int
tas(volatile slock_t *lock)
{
volatile int *lockword = TAS_ACTIVE_WORD(lock);
Remove uses of register due to incompatibility with C++17 and up The use in regexec.c could remain, since we only try to keep headers C++ clean. But there really doesn't seem to be a good reason to use register in that spot. Discussion: https://postgr.es/m/20220308185902.ibdqmasoaunzjrfc@alap3.anarazel.de
2022-09-24 21:01:06 +02:00
int lockval;
Use inlined TAS() on PA-RISC, if we are compiling with gcc. Patch inspired by original submission from ViSolve.
2003-12-23 23:15:07 +01:00
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
/*
* The LDCWX instruction atomically clears the target word and
* returns the previous value. Hence, if the instruction returns
* 0, someone else has already acquired the lock before we tested
* it (i.e., we have failed).
*
* Notice that this means that we actually clear the word to set
* the lock and set the word to clear the lock. This is the
* opposite behavior from the SPARC LDSTUB instruction. For some
* reason everything that H-P does is rather baroque...
*
* For details about the LDCWX instruction, see the "Precision
* Architecture and Instruction Reference Manual" (09740-90014 of June
* 1987), p. 5-38.
*/
Use inlined TAS() on PA-RISC, if we are compiling with gcc. Patch inspired by original submission from ViSolve.
2003-12-23 23:15:07 +01:00
__asm__ __volatile__(
" ldcwx 0(0,%2),%0 \n"
Adjust TAS assembly as per recent discussions: use "+m"(*lock) everywhere to reference the spinlock variable, and specify "memory" as a clobber operand to be sure gcc does not try to keep shared-memory values in registers across a spinlock acquisition. Also tighten the S/390 asm sequence, which was apparently written with only minimal study of the gcc asm documentation. I have personally tested i386, ia64, ppc, hppa, and s390 variants --- there is some small chance that I broke the others, but I doubt it.
2004-06-20 01:02:32 +02:00
: "=r"(lockval), "+m"(*lockword)
: "r"(lockword)
: "memory");
Use inlined TAS() on PA-RISC, if we are compiling with gcc. Patch inspired by original submission from ViSolve.
2003-12-23 23:15:07 +01:00
return (lockval == 0);
}
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
#define S_UNLOCK(lock) \
do { \
__asm__ __volatile__("" : : : "memory"); \
*TAS_ACTIVE_WORD(lock) = -1; \
} while (0)
Use inlined TAS() on PA-RISC, if we are compiling with gcc. Patch inspired by original submission from ViSolve.
2003-12-23 23:15:07 +01:00
#define S_INIT_LOCK(lock) \
Fix alignment
2001-01-19 08:03:53 +01:00
do { \
Use inlined TAS() on PA-RISC, if we are compiling with gcc. Patch inspired by original submission from ViSolve.
2003-12-23 23:15:07 +01:00
volatile slock_t *lock_ = (lock); \
Just noticed that use of 'volatile' in HPPA S_UNLOCK() was causing gcc to generate unnecessarily stupid code. Tweak macro to describe a series of store-constant ops, not store/load/store/load/store/load/store.
2001-02-17 00:50:40 +01:00
lock_->sema[0] = -1; \
lock_->sema[1] = -1; \
lock_->sema[2] = -1; \
lock_->sema[3] = -1; \
Fix alignment
2001-01-19 08:03:53 +01:00
} while (0)
Make HP-PA S_UNLOCK a little faster and less dependent on unspecified compiler codegen details. Make default S_LOCK macro more robust against syntax mistakes. (I cleared these changes with David Gould a few days ago.)
1998-10-01 03:53:50 +02:00
Use inlined TAS() on PA-RISC, if we are compiling with gcc. Patch inspired by original submission from ViSolve.
2003-12-23 23:15:07 +01:00
#define S_LOCK_FREE(lock) (*TAS_ACTIVE_WORD(lock) != 0)
Make HP-PA S_UNLOCK a little faster and less dependent on unspecified compiler codegen details. Make default S_LOCK macro more robust against syntax mistakes. (I cleared these changes with David Gould a few days ago.)
1998-10-01 03:53:50 +02:00
Must test for __hppa__ as well as __hppa to make linux-hppa happy.
2004-01-03 06:47:44 +01:00
#endif /* __hppa || __hppa__ */
From: Tom I Helbekkmo <tih@Hamartun.Priv.NO> Apart from this Makefile hack, all I've done is to make dynamically loaded code modules fail properly (as was already done for __mips__, although I think this is too loose: I believe NetBSD for the pmax can do dynamic linking), and to add test-and-set lock handling. As Bruce suggested, this is done in a maximally efficient inlined way: I was not aware that this code was so important, speed-wise.
1998-02-13 06:10:06 +01:00
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Allow use of __sync_lock_test_and_set for spinlocks on any machine. If we have no special-case code in s_lock.h for the current platform, but the compiler has __sync_lock_test_and_set, use that instead of failing. It's unlikely that anybody's __sync_lock_test_and_set would be so awful as to be worse than our semaphore-based fallback, but if it is, they can (continue to) use --disable-spinlocks. This allows removal of the RISC-V special case installed by commit c32fcac56, which generated exactly the same code but only on that platform. Usefully, the RISC-V buildfarm animals should now test at least the int variant of this patch. I've manually tested both variants on ARM by dint of removing the ARM-specific stanza. We don't want to drop that, because it already has some special knowledge and is likely to grow more over time. Likewise, this is not meant to preclude installing special cases for other arches if that proves worthwhile. Per discussion of a request to install the same code for loongarch64. Like the previous patch, we might as well back-patch to supported branches. Discussion: https://postgr.es/m/761ac43d44b84d679ba803c2bd947cc0@HSMAILSVR04.hs.handsome.com.cn
2022-11-02 22:37:26 +01:00
/*
* If we have no platform-specific knowledge, but we found that the compiler
* provides __sync_lock_test_and_set(), use that. Prefer the int-width
* version over the char-width version if we have both, on the rather dubious
* grounds that that's known to be more likely to work in the ARM ecosystem.
* (But we dealt with ARM above.)
*/
#if !defined(HAS_TEST_AND_SET)
#if defined(HAVE_GCC__SYNC_INT32_TAS)
#define HAS_TEST_AND_SET
#define TAS(lock) tas(lock)
typedef int slock_t;
static __inline__ int
tas(volatile slock_t *lock)
{
return __sync_lock_test_and_set(lock, 1);
}
#define S_UNLOCK(lock) __sync_lock_release(lock)
#elif defined(HAVE_GCC__SYNC_CHAR_TAS)
#define HAS_TEST_AND_SET
#define TAS(lock) tas(lock)
typedef char slock_t;
static __inline__ int
tas(volatile slock_t *lock)
{
return __sync_lock_test_and_set(lock, 1);
}
#define S_UNLOCK(lock) __sync_lock_release(lock)
#endif /* HAVE_GCC__SYNC_INT32_TAS */
#endif /* !defined(HAS_TEST_AND_SET) */
Add comments explaining why our Itanium spinlock implementation is safe.
2012-03-16 09:14:45 +01:00
/*
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
* Default implementation of S_UNLOCK() for gcc/icc.
Add comments explaining why our Itanium spinlock implementation is safe.
2012-03-16 09:14:45 +01:00
*
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
* Note that this implementation is unsafe for any platform that can reorder
* a memory access (either load or store) after a following store. That
* happens not to be possible on x86 and most legacy architectures (some are
* single-processor!), but many modern systems have weaker memory ordering.
* Those that do must define their own version of S_UNLOCK() rather than
* relying on this one.
Add comments explaining why our Itanium spinlock implementation is safe.
2012-03-16 09:14:45 +01:00
*/
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
#if !defined(S_UNLOCK)
#define S_UNLOCK(lock) \
do { __asm__ __volatile__("" : : : "memory"); *(lock) = 0; } while (0)
#endif
Add s_lock support for HPUX on IA64, per Shinji Teragaito.
2004-09-02 19:10:58 +02:00
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
#endif /* defined(__GNUC__) || defined(__INTEL_COMPILER) */
Add s_lock support for HPUX on IA64, per Shinji Teragaito.
2004-09-02 19:10:58 +02:00
Remove HP-UX port. HP-UX hardware is no longer produced, build farm coverage recently ended, and there are no known active maintainers targeting this OS. Since there is a major rewrite of the build system in the pipeline for PostgreSQL 16, and that requires development, testing and maintainance for each OS and tool chain, it seems like a good time to drop support for: * HP-UX, the operating system. * HP aCC, the HP-UX native compiler. Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Reviewed-by: Andres Freund <andres@anarazel.de> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Discussion: https://postgr.es/m/1415825.1656893299%40sss.pgh.pa.us
2022-07-08 01:17:47 +02:00
/*
* ---------------------------------------------------------------------
* Platforms that use non-gcc inline assembly:
* ---------------------------------------------------------------------
*/
#if !defined(HAS_TEST_AND_SET) /* We didn't trigger above, let's try here */
Add s_lock support for HPUX on IA64, per Shinji Teragaito.
2004-09-02 19:10:58 +02:00
Update s_lock.c comments.
2005-12-17 21:39:16 +01:00
#if defined(_AIX) /* AIX */
Move common lock code to their own section.
1998-07-18 16:58:58 +02:00
/*
* AIX (POWER)
*/
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
#define HAS_TEST_AND_SET
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
On AIX, include <sys/atomic_op.h> so that the functions we use for TAS support are properly declared.
2007-07-16 04:03:14 +02:00
#include <sys/atomic_op.h>
Hmm, so evidently _check_lock and _clear_lock take an argument of type int not unsigned int. Third try to get grebe building without warnings...
2007-07-16 16:02:22 +02:00
typedef int slock_t;
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
So our reward for including <sys/atomic_op.h> seems to be a bunch of nattering about casting away volatile. Losers.
2007-07-16 06:57:57 +02:00
#define TAS(lock) _check_lock((slock_t *) (lock), 0, 1)
#define S_UNLOCK(lock) _clear_lock((slock_t *) (lock), 0)
Move common lock code to their own section.
1998-07-18 16:58:58 +02:00
#endif /* _AIX */
Fix spinlock implementation for some !solaris sparc platforms. Some Sparc CPUs can be run in various coherence models, ranging from RMO (relaxed) over PSO (partial) to TSO (total). Solaris has always run CPUs in TSO mode while in userland, but linux didn't use to and the various *BSDs still don't. Unfortunately the sparc TAS/S_UNLOCK were only correct under TSO. Fix that by adding the necessary memory barrier instructions. On sparcv8+, which should be all relevant CPUs, these are treated as NOPs if the current consistency model doesn't require the barriers. Discussion: 20140630222854.GW26930@awork2.anarazel.de Will be backpatched to all released branches once a few buildfarm cycles haven't shown up problems. As I've no access to sparc, this is blindly written.
2014-09-09 00:47:32 +02:00
/* These are in sunstudio_(sparc|x86).s */
Move common lock code to their own section.
1998-07-18 16:58:58 +02:00
Support for Sun Studio compiler on Linux This basically takes some build system code that was previously labeled "Solaris" and ties it to the compiler rather than the operating system. Author: Julius Stroffek <Julius.Stroffek@Sun.COM>
2008-10-29 17:06:47 +01:00
#if defined(__SUNPRO_C) && (defined(__i386) || defined(__x86_64__) || defined(__sparc__) || defined(__sparc))
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
#define HAS_TEST_AND_SET
Use unsigned into for slock_t for pre-sparcv8plus.
2006-05-18 23:18:40 +02:00
Fix Solaris/ASM test for x86.
2006-05-19 15:10:11 +02:00
#if defined(__i386) || defined(__x86_64__) || defined(__sparcv9) || defined(__sparcv8plus)
Solaris tas() uses 'int' now. Theo Schlossnagle
2006-04-29 13:55:19 +02:00
typedef unsigned int slock_t;
Use unsigned into for slock_t for pre-sparcv8plus.
2006-05-18 23:18:40 +02:00
#else
typedef unsigned char slock_t;
#endif
Fix TAS assembly stuff for Solaris/386. (I'm not in a position to actually test this, but it couldn't be broken any worse than it was...)
2004-09-24 02:21:32 +02:00
Remove "volatile" from tas function, per TOm.
2006-04-28 05:43:19 +02:00
extern slock_t pg_atomic_cas(volatile slock_t *lock, slock_t with,
Rewrite Solaris compiler tas() assembly routines, merge i386 and x86_64 assembler files, renamed as solaris_x86.s. Theo Schlossnagle
2006-04-28 00:28:42 +02:00
slock_t cmp);
Fix TAS assembly stuff for Solaris/386. (I'm not in a position to actually test this, but it couldn't be broken any worse than it was...)
2004-09-24 02:21:32 +02:00
Rewrite Solaris compiler tas() assembly routines, merge i386 and x86_64 assembler files, renamed as solaris_x86.s. Theo Schlossnagle
2006-04-28 00:28:42 +02:00
#define TAS(a) (pg_atomic_cas((a), 1, 0) != 0)
Fix TAS assembly stuff for Solaris/386. (I'm not in a position to actually test this, but it couldn't be broken any worse than it was...)
2004-09-24 02:21:32 +02:00
#endif
Remove symbol WIN32_ONLY_COMPILER This used to mean "Visual C++ except in those parts where Borland C++ was supported where it meant one of those". Now that we don't support Borland C++ anymore, simplify by using _MSC_VER which is the normal way to detect Visual C++.
2017-04-11 15:21:25 +02:00
#ifdef _MSC_VER
Prepare code to be built by MSVC: o remove many WIN32_CLIENT_ONLY defines o add WIN32_ONLY_COMPILER define o add 3rd argument to open() for portability o add include/port/win32_msvc directory for system includes Magnus Hagander
2006-06-08 00:24:46 +02:00
typedef LONG slock_t;
#define HAS_TEST_AND_SET
#define TAS(lock) (InterlockedCompareExchange(lock, 1, 0))
#define SPIN_DELAY() spin_delay()
Add a Win64-specific spin_delay() function. We can't use the same as before, since MSVC on Win64 doesn't support inline assembly.
2010-01-04 18:10:24 +01:00
/* If using Visual C++ on Win64, inline assembly is unavailable.
Fix typos in comments. Backpatch to all supported versions, where applicable, to make backpatching of future fixes go more smoothly. Josh Soref Discussion: https://www.postgresql.org/message-id/CACZqfqCf+5qRztLPgmmosr-B0Ye4srWzzw_mo4c_8_B_mtjmJQ@mail.gmail.com
2017-02-06 10:33:58 +01:00
* Use a _mm_pause intrinsic instead of rep nop.
Add a Win64-specific spin_delay() function. We can't use the same as before, since MSVC on Win64 doesn't support inline assembly.
2010-01-04 18:10:24 +01:00
*/
#if defined(_WIN64)
static __forceinline void
spin_delay(void)
{
Use _mm_pause() for win64 spin_delay(), per note from Tsutomu Yamada.
2010-01-05 12:06:28 +01:00
_mm_pause();
Add a Win64-specific spin_delay() function. We can't use the same as before, since MSVC on Win64 doesn't support inline assembly.
2010-01-04 18:10:24 +01:00
}
#else
Prepare code to be built by MSVC: o remove many WIN32_CLIENT_ONLY defines o add WIN32_ONLY_COMPILER define o add 3rd argument to open() for portability o add include/port/win32_msvc directory for system includes Magnus Hagander
2006-06-08 00:24:46 +02:00
static __forceinline void
spin_delay(void)
{
/* See comment for gcc code. Same code, MASM syntax */
__asm rep nop;
}
Add a Win64-specific spin_delay() function. We can't use the same as before, since MSVC on Win64 doesn't support inline assembly.
2010-01-04 18:10:24 +01:00
#endif
Prepare code to be built by MSVC: o remove many WIN32_CLIENT_ONLY defines o add WIN32_ONLY_COMPILER define o add 3rd argument to open() for portability o add include/port/win32_msvc directory for system includes Magnus Hagander
2006-06-08 00:24:46 +02:00
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
#include <intrin.h>
#pragma intrinsic(_ReadWriteBarrier)
#define S_UNLOCK(lock) \
Fix typo in 07c8651dd91d5 causing WIN32_ONLY_COMPILER builds to fail.
2014-09-10 02:43:18 +02:00
do { _ReadWriteBarrier(); (*(lock)) = 0; } while (0)
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
Prepare code to be built by MSVC: o remove many WIN32_CLIENT_ONLY defines o add WIN32_ONLY_COMPILER define o add 3rd argument to open() for portability o add include/port/win32_msvc directory for system includes Magnus Hagander
2006-06-08 00:24:46 +02:00
#endif
Remove useless whitespace at end of lines
2010-11-23 21:27:50 +01:00
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
#endif /* !defined(HAS_TEST_AND_SET) */
Implement compiler #error if spinlock code not found, add configure flag to bypass the error, --without-spinlocks.
2003-09-12 18:10:27 +02:00
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
/* Blow up if we didn't have any way to do spinlocks */
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
#ifndef HAS_TEST_AND_SET
Replace @postgresql.org with @lists.postgresql.org for mailinglists Commit c0d0e54084 replaced the ones in the documentation, but missed out on the ones in the code. Replace those as well, but unlike c0d0e54084, don't backpatch the code changes to avoid breaking translations.
2019-01-19 19:06:35 +01:00
#error PostgreSQL does not have native spinlock support on this platform. To continue the compilation, rerun configure using --disable-spinlocks. However, performance will be poor. Please report this to pgsql-bugs@lists.postgresql.org.
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
#endif
#else /* !HAVE_SPINLOCKS */
Move slock_t typdefs into s_lock.h from include/port files for centralization and easier maintanence.
2003-12-23 04:31:30 +01:00
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
/*
Create an internal semaphore API that is not tied to SysV semaphores. As proof of concept, provide an alternate implementation based on POSIX semaphores. Also push the SysV shared-memory implementation into a separate file so that it can be replaced conveniently.
2002-05-05 02:03:29 +02:00
* Fake spinlock implementation using semaphores --- slow and prone
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
* to fall foul of kernel limits on number of semaphores, so don't use this
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
* unless you must! The subroutines appear in spin.c.
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
*/
Reduce the number of semaphores used under --disable-spinlocks. Instead of allocating a semaphore from the operating system for every spinlock, allocate a fixed number of semaphores (by default, 1024) from the operating system and multiplex all the spinlocks that get created onto them. This could self-deadlock if a process attempted to acquire more than one spinlock at a time, but since processes aren't supposed to execute anything other than short stretches of straight-line code while holding a spinlock, that shouldn't happen. One motivation for this change is that, with the introduction of dynamic shared memory, it may be desirable to create spinlocks that last for less than the lifetime of the server. Without this change, attempting to use such facilities under --disable-spinlocks would quickly exhaust any supply of available semaphores. Quite apart from that, it's desirable to contain the quantity of semaphores needed to run the server simply on convenience grounds, since using too many may make it harder to get PostgreSQL running on a new platform, which is mostly the point of --disable-spinlocks in the first place. Patch by me; review by Tom Lane.
2014-01-09 00:49:14 +01:00
typedef int slock_t;
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
extern bool s_lock_free_sema(volatile slock_t *lock);
extern void s_unlock_sema(volatile slock_t *lock);
Add a basic atomic ops API abstracting away platform/architecture details. Several upcoming performance/scalability improvements require atomic operations. This new API avoids the need to splatter compiler and architecture dependent code over all the locations employing atomic ops. For several of the potential usages it'd be problematic to maintain both, a atomics using implementation and one using spinlocks or similar. In all likelihood one of the implementations would not get tested regularly under concurrency. To avoid that scenario the new API provides a automatic fallback of atomic operations to spinlocks. All properties of atomic operations are maintained. This fallback - obviously - isn't as fast as just using atomic ops, but it's not bad either. For one of the future users the atomics ontop spinlocks implementation was actually slightly faster than the old purely spinlock using implementation. That's important because it reduces the fear of regressing older platforms when improving the scalability for new ones. The API, loosely modeled after the C11 atomics support, currently provides 'atomic flags' and 32 bit unsigned integers. If the platform efficiently supports atomic 64 bit unsigned integers those are also provided. To implement atomics support for a platform/architecture/compiler for a type of atomics 32bit compare and exchange needs to be implemented. If available and more efficient native support for flags, 32 bit atomic addition, and corresponding 64 bit operations may also be provided. Additional useful atomic operations are implemented generically ontop of these. The implementation for various versions of gcc, msvc and sun studio have been tested. Additional existing stub implementations for * Intel icc * HUPX acc * IBM xlc are included but have never been tested. These will likely require fixes based on buildfarm and user feedback. As atomic operations also require barriers for some operations the existing barrier support has been moved into the atomics code. Author: Andres Freund with contributions from Oskari Saarenmaa Reviewed-By: Amit Kapila, Robert Haas, Heikki Linnakangas and Álvaro Herrera Discussion: CA+TgmoYBW+ux5-8Ja=Mcyuy8=VXAnVRHp3Kess6Pn3DMXAPAEA@mail.gmail.com, 20131015123303.GH5300@awork2.anarazel.de, 20131028205522.GI20248@awork2.anarazel.de
2014-09-25 23:49:05 +02:00
extern void s_init_lock_sema(volatile slock_t *lock, bool nested);
New ASM format: /* * Standard __asm__ format: * * __asm__( * "command;" * "command;" * "command;" * : "=r"(_res) return value, in register * : "r"(lock) argument, 'lock pointer', in register * : "r0"); inline code uses this register */
2001-01-19 03:58:59 +01:00
extern int tas_sema(volatile slock_t *lock);
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
New ASM format: /* * Standard __asm__ format: * * __asm__( * "command;" * "command;" * "command;" * : "=r"(_res) return value, in register * : "r"(lock) argument, 'lock pointer', in register * : "r0"); inline code uses this register */
2001-01-19 03:58:59 +01:00
#define S_LOCK_FREE(lock) s_lock_free_sema(lock)
#define S_UNLOCK(lock) s_unlock_sema(lock)
Add a basic atomic ops API abstracting away platform/architecture details. Several upcoming performance/scalability improvements require atomic operations. This new API avoids the need to splatter compiler and architecture dependent code over all the locations employing atomic ops. For several of the potential usages it'd be problematic to maintain both, a atomics using implementation and one using spinlocks or similar. In all likelihood one of the implementations would not get tested regularly under concurrency. To avoid that scenario the new API provides a automatic fallback of atomic operations to spinlocks. All properties of atomic operations are maintained. This fallback - obviously - isn't as fast as just using atomic ops, but it's not bad either. For one of the future users the atomics ontop spinlocks implementation was actually slightly faster than the old purely spinlock using implementation. That's important because it reduces the fear of regressing older platforms when improving the scalability for new ones. The API, loosely modeled after the C11 atomics support, currently provides 'atomic flags' and 32 bit unsigned integers. If the platform efficiently supports atomic 64 bit unsigned integers those are also provided. To implement atomics support for a platform/architecture/compiler for a type of atomics 32bit compare and exchange needs to be implemented. If available and more efficient native support for flags, 32 bit atomic addition, and corresponding 64 bit operations may also be provided. Additional useful atomic operations are implemented generically ontop of these. The implementation for various versions of gcc, msvc and sun studio have been tested. Additional existing stub implementations for * Intel icc * HUPX acc * IBM xlc are included but have never been tested. These will likely require fixes based on buildfarm and user feedback. As atomic operations also require barriers for some operations the existing barrier support has been moved into the atomics code. Author: Andres Freund with contributions from Oskari Saarenmaa Reviewed-By: Amit Kapila, Robert Haas, Heikki Linnakangas and Álvaro Herrera Discussion: CA+TgmoYBW+ux5-8Ja=Mcyuy8=VXAnVRHp3Kess6Pn3DMXAPAEA@mail.gmail.com, 20131015123303.GH5300@awork2.anarazel.de, 20131028205522.GI20248@awork2.anarazel.de
2014-09-25 23:49:05 +02:00
#define S_INIT_LOCK(lock) s_init_lock_sema(lock, false)
New ASM format: /* * Standard __asm__ format: * * __asm__( * "command;" * "command;" * "command;" * : "=r"(_res) return value, in register * : "r"(lock) argument, 'lock pointer', in register * : "r0"); inline code uses this register */
2001-01-19 03:58:59 +01:00
#define TAS(lock) tas_sema(lock)
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
Mop-up for HAS_TEST_AND_SET refactoring. Un-break two or three platforms that were broken, try to make layout of s_lock.h entries consistent, use HAVE_SPINLOCKS in preference to HAS_TEST_AND_SET everywhere outside s_lock.h itself.
2003-12-23 19:13:17 +01:00
#endif /* HAVE_SPINLOCKS */
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
Move common lock code to their own section.
1998-07-18 16:58:58 +02:00
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
/*
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
* Default Definitions - override these above as needed.
*/
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
#if !defined(S_LOCK)
Fix macros that were not properly surrounded by parens or braces.
1998-06-15 20:40:05 +02:00
#define S_LOCK(lock) \
Rely on __func__ being supported Previously we fell back to __FUNCTION__ and then NULL. As __func__ is in C99 that shouldn't be necessary anymore. Solution.pm defined HAVE_FUNCNAME__FUNCTION instead of HAVE_FUNCNAME__FUNC (originating in 4164e6636e2), as at some point in the past MSVC only supported __FUNCTION__. Our minimum version supports __func__. Reviewed-By: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/20220807012914.ydz73yte6j3coulo@awork3.anarazel.de
2022-08-07 18:36:01 +02:00
(TAS(lock) ? s_lock((lock), __FILE__, __LINE__, __func__) : 0)
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#endif /* S_LOCK */
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#if !defined(S_LOCK_FREE)
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
#define S_LOCK_FREE(lock) (*(lock) == 0)
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#endif /* S_LOCK_FREE */
bring in alpha/linux s_lock changes From: Ryan Kirkpatrick <rkirkpat@nag.cs.colorado.edu>
1997-12-30 05:01:28 +01:00
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#if !defined(S_UNLOCK)
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
/*
* Our default implementation of S_UNLOCK is essentially *(lock) = 0. This
Clean up icc + ia64 situation. Some googling turned up multiple sources saying that older versions of icc do not accept gcc-compatible asm blocks on IA64, though asm does work on x86[_64]. This is apparently fixed as of icc version 12.0 or so, but that doesn't help us much; if we have to carry the extra implementation anyway, we may as well just use it for icc rather than add a compiler version test. Hence, revert commit 2c713d6ea29c91cd2cbd92fa801a61e55ea2a3c4 (though I separated the icc code from the gcc code completely, producing what seems cleaner code). Document the state of affairs more explicitly, both in s_lock.h and postgres.c, and make some cosmetic adjustments around the IA64 code in s_lock.h.
2015-09-01 00:10:04 +02:00
* is unsafe if the platform can reorder a memory access (either load or
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
* store) after a following store; platforms where this is possible must
* define their own S_UNLOCK. But CPU reordering is not the only concern:
* if we simply defined S_UNLOCK() as an inline macro, the compiler might
* reorder instructions from inside the critical section to occur after the
* lock release. Since the compiler probably can't know what the external
* function s_unlock is doing, putting the same logic there should be adequate.
* A sufficiently-smart globally optimizing compiler could break that
* assumption, though, and the cost of a function call for every spinlock
* release may hurt performance significantly, so we use this implementation
* only for platforms where we don't know of a suitable intrinsic. For the
* most part, those are relatively obscure platform/compiler combinations to
* which the PostgreSQL project does not have access.
*/
#define USE_DEFAULT_S_UNLOCK
Fix thinko in 0709b7ee72e4bc71ad07b7120acd117265ab51d0. Buildfarm member castoroides is unhappy with this, for entirely understandable reasons.
2014-09-10 20:40:21 +02:00
extern void s_unlock(volatile slock_t *lock);
Change the spinlock primitives to function as compiler barriers. Previously, they functioned as barriers against CPU reordering but not compiler reordering, an odd API that required extensive use of volatile everywhere that spinlocks are used. That's error-prone and has negative implications for performance, so change it. In theory, this makes it safe to remove many of the uses of volatile that we currently have in our code base, but we may find that there are some bugs in this effort when we do. In the long run, though, this should make for much more maintainable code. Patch by me. Review by Andres Freund.
2014-09-09 23:45:20 +02:00
#define S_UNLOCK(lock) s_unlock(lock)
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#endif /* S_UNLOCK */
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#if !defined(S_INIT_LOCK)
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
#define S_INIT_LOCK(lock) S_UNLOCK(lock)
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#endif /* S_INIT_LOCK */
Inline frequently called functions.
1997-09-18 16:21:02 +02:00
Improve spinlock code for recent x86 processors: insert a PAUSE instruction in the s_lock() wait loop, and use test before test-and-set in TAS() macro to avoid unnecessary bus traffic. Patch from Manfred Spraul, reworked a bit by Tom.
2003-12-27 21:58:58 +01:00
#if !defined(SPIN_DELAY)
#define SPIN_DELAY() ((void) 0)
#endif /* SPIN_DELAY */
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#if !defined(TAS)
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
extern int tas(volatile slock_t *lock); /* in port/.../tas.s, or
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
* s_lock.c */
Fix for S_FREE_LOCK.
1997-09-25 01:37:26 +02:00
Fix failure in CreateCheckPoint on some Alpha boxes --- it's not OK to assume that TAS() will always succeed the first time, even if the lock is known to be free. Also, make sure that code will eventually time out and report a stuck spinlock, rather than looping forever. Small cleanups in s_lock.h, too.
2000-12-29 22:31:21 +01:00
#define TAS(lock) tas(lock)
Okay...this should fix the s_lock() patches from David to what he currently has... From: David Gould <dg@illustra.com>
1998-05-04 18:58:59 +02:00
#endif /* TAS */
Here is the long awaited optimized version of the S_LOCK patch. This incorporates all the precedeing patches and emailed suggestions and the results of the performance testing I posted last week. I would like to get this tested on as many platforms as possible so I can verify it went in correctly (as opposed to the horrorshow last time I sent in a patch). Once this is confirmed, I will make a tarball of files that can be dropped into a 6.3.2 source tree as a few people have asked for this in 6.3.2 as well. David Gould
1998-06-16 09:18:16 +02:00
Improve spinlock performance for HP-UX, ia64, non-gcc. At least on this architecture, it's very important to spin on a non-atomic instruction and only retry the atomic once it appears that it will succeed. To fix this, split TAS() into two macros: TAS(), for trying to grab the lock the first time, and TAS_SPIN(), for spinning until we get it. TAS_SPIN() defaults to same as TAS(), but we can override it when we know there's a better way. It's likely that some of the other cases in s_lock.h require similar treatment, but this is the only one we've got conclusive evidence for at present.
2011-08-29 16:05:48 +02:00
#if !defined(TAS_SPIN)
#define TAS_SPIN(lock) TAS(lock)
#endif /* TAS_SPIN */
Apply PGDLLIMPORT markings broadly. Up until now, we've had a policy of only marking certain variables in the PostgreSQL header files with PGDLLIMPORT, but now we've decided to mark them all. This means that extensions running on Windows should no longer operate at a disadvantage as compared to extensions running on Linux: if the variable is present in a header file, it should be accessible. Discussion: http://postgr.es/m/CA+TgmoYanc1_FSfimhgiWSqVyP5KKmh5NP2BWNwDhO8Pg2vGYQ@mail.gmail.com
2022-04-08 14:16:38 +02:00
extern PGDLLIMPORT slock_t dummy_spinlock;
Significant cleanups in SysV IPC handling (shared mem and semaphores). IPC key assignment will now work correctly even when multiple postmasters are using same logical port number (which is possible given -k switch). There is only one shared-mem segment per postmaster now, not 3. Rip out broken code for non-TAS case in bufmgr and xlog, substitute a complete S_LOCK emulation using semaphores in spin.c. TAS and non-TAS logic is now exactly the same. When deadlock is detected, "Deadlock detected" is now the elog(ERROR) message, rather than a NOTICE that comes out before an unhelpful ERROR.
2000-11-29 00:27:57 +01:00
Implement new 'lightweight lock manager' that's intermediate between existing lock manager and spinlocks: it understands exclusive vs shared lock but has few other fancy features. Replace most uses of spinlocks with lightweight locks. All remaining uses of spinlocks have very short lock hold times (a few dozen instructions), so tweak spinlock backoff code to work efficiently given this assumption. All per my proposal on pghackers 26-Sep-01.
2001-09-29 06:02:27 +02:00
/*
Re-order declarations to un-break the non-HAS_TEST_AND_SET case.
2001-03-25 19:52:46 +02:00
* Platform-independent out-of-line support routines
*/
Make init_spin_delay() C89 compliant and change stuck spinlock reporting. The current definition of init_spin_delay (introduced recently in 48354581a) wasn't C89 compliant. It's not legal to refer to refer to non-constant expressions, and the ptr argument was one. This, as reported by Tom, lead to a failure on buildfarm animal pademelon. The pointer, especially on system systems with ASLR, isn't super helpful anyway, though. So instead of making init_spin_delay into an inline function, make s_lock_stuck() report the function name in addition to file:line and change init_spin_delay() accordingly. While not a direct replacement, the function name is likely more useful anyway (line numbers are often hard to interpret in third party reports). This also fixes what file/line number is reported for waits via s_lock(). As PG_FUNCNAME_MACRO is now used outside of elog.h, move it to c.h. Reported-By: Tom Lane Discussion: 4369.1460435533@sss.pgh.pa.us
2016-04-14 01:42:01 +02:00
extern int s_lock(volatile slock_t *lock, const char *file, int line, const char *func);
Re-order declarations to un-break the non-HAS_TEST_AND_SET case.
2001-03-25 19:52:46 +02:00
Add infrastructure for making spins_per_delay variable depending on whether we seem to be running in a uniprocessor or multiprocessor. The adjustment rules could probably still use further tweaking, but I'm convinced this should be a win overall.
2005-10-11 22:41:32 +02:00
/* Support for dynamic adjustment of spins_per_delay */
#define DEFAULT_SPINS_PER_DELAY 100
extern void set_spins_per_delay(int shared_spins_per_delay);
extern int update_spins_per_delay(int shared_spins_per_delay);
Allow Pin/UnpinBuffer to operate in a lockfree manner. Pinning/Unpinning a buffer is a very frequent operation; especially in read-mostly cache resident workloads. Benchmarking shows that in various scenarios the spinlock protecting a buffer header's state becomes a significant bottleneck. The problem can be reproduced with pgbench -S on larger machines, but can be considerably worse for queries which touch the same buffers over and over at a high frequency (e.g. nested loops over a small inner table). To allow atomic operations to be used, cram BufferDesc's flags, usage_count, buf_hdr_lock, refcount into a single 32bit atomic variable; that allows to manipulate them together using 32bit compare-and-swap operations. This requires reducing MAX_BACKENDS to 2^18-1 (which could be lifted by using a 64bit field, but it's not a realistic configuration atm). As not all operations can easily implemented in a lockfree manner, implement the previous buf_hdr_lock via a flag bit in the atomic variable. That way we can continue to lock the header in places where it's needed, but can get away without acquiring it in the more frequent hot-paths. There's some additional operations which can be done without the lock, but aren't in this patch; but the most important places are covered. As bufmgr.c now essentially re-implements spinlocks, abstract the delay logic from s_lock.c into something more generic. It now has already two users, and more are coming up; there's a follupw patch for lwlock.c at least. This patch is based on a proof-of-concept written by me, which Alexander Korotkov made into a fully working patch; the committed version is again revised by me. Benchmarking and testing has, amongst others, been provided by Dilip Kumar, Alexander Korotkov, Robert Haas. On a large x86 system improvements for readonly pgbench, with a high client count, of a factor of 8 have been observed. Author: Alexander Korotkov and Andres Freund Discussion: 2400449.GjM57CE0Yg@dinodell
2016-04-11 05:12:32 +02:00
/*
* Support for spin delay which is useful in various places where
* spinlock-like procedures take place.
*/
typedef struct
{
int spins;
int delays;
int cur_delay;
const char *file;
int line;
Make init_spin_delay() C89 compliant and change stuck spinlock reporting. The current definition of init_spin_delay (introduced recently in 48354581a) wasn't C89 compliant. It's not legal to refer to refer to non-constant expressions, and the ptr argument was one. This, as reported by Tom, lead to a failure on buildfarm animal pademelon. The pointer, especially on system systems with ASLR, isn't super helpful anyway, though. So instead of making init_spin_delay into an inline function, make s_lock_stuck() report the function name in addition to file:line and change init_spin_delay() accordingly. While not a direct replacement, the function name is likely more useful anyway (line numbers are often hard to interpret in third party reports). This also fixes what file/line number is reported for waits via s_lock(). As PG_FUNCNAME_MACRO is now used outside of elog.h, move it to c.h. Reported-By: Tom Lane Discussion: 4369.1460435533@sss.pgh.pa.us
2016-04-14 01:42:01 +02:00
const char *func;
Allow Pin/UnpinBuffer to operate in a lockfree manner. Pinning/Unpinning a buffer is a very frequent operation; especially in read-mostly cache resident workloads. Benchmarking shows that in various scenarios the spinlock protecting a buffer header's state becomes a significant bottleneck. The problem can be reproduced with pgbench -S on larger machines, but can be considerably worse for queries which touch the same buffers over and over at a high frequency (e.g. nested loops over a small inner table). To allow atomic operations to be used, cram BufferDesc's flags, usage_count, buf_hdr_lock, refcount into a single 32bit atomic variable; that allows to manipulate them together using 32bit compare-and-swap operations. This requires reducing MAX_BACKENDS to 2^18-1 (which could be lifted by using a 64bit field, but it's not a realistic configuration atm). As not all operations can easily implemented in a lockfree manner, implement the previous buf_hdr_lock via a flag bit in the atomic variable. That way we can continue to lock the header in places where it's needed, but can get away without acquiring it in the more frequent hot-paths. There's some additional operations which can be done without the lock, but aren't in this patch; but the most important places are covered. As bufmgr.c now essentially re-implements spinlocks, abstract the delay logic from s_lock.c into something more generic. It now has already two users, and more are coming up; there's a follupw patch for lwlock.c at least. This patch is based on a proof-of-concept written by me, which Alexander Korotkov made into a fully working patch; the committed version is again revised by me. Benchmarking and testing has, amongst others, been provided by Dilip Kumar, Alexander Korotkov, Robert Haas. On a large x86 system improvements for readonly pgbench, with a high client count, of a factor of 8 have been observed. Author: Alexander Korotkov and Andres Freund Discussion: 2400449.GjM57CE0Yg@dinodell
2016-04-11 05:12:32 +02:00
} SpinDelayStatus;
Make init_spin_delay() C89 compliant #2. My previous attempt at doing so, in 80abbeba23, was not sufficient. While that fixed the problem for bufmgr.c and lwlock.c , s_lock.c still has non-constant expressions in the struct initializer, because the file/line/function information comes from the caller of s_lock(). Give up on using a macro, and use a static inline instead. Discussion: 4369.1460435533@sss.pgh.pa.us
2016-04-15 04:26:13 +02:00
static inline void
init_spin_delay(SpinDelayStatus *status,
const char *file, int line, const char *func)
{
status->spins = 0;
status->delays = 0;
status->cur_delay = 0;
status->file = file;
status->line = line;
status->func = func;
}
Rely on __func__ being supported Previously we fell back to __FUNCTION__ and then NULL. As __func__ is in C99 that shouldn't be necessary anymore. Solution.pm defined HAVE_FUNCNAME__FUNCTION instead of HAVE_FUNCNAME__FUNC (originating in 4164e6636e2), as at some point in the past MSVC only supported __FUNCTION__. Our minimum version supports __func__. Reviewed-By: Thomas Munro <thomas.munro@gmail.com> Discussion: https://postgr.es/m/20220807012914.ydz73yte6j3coulo@awork3.anarazel.de
2022-08-07 18:36:01 +02:00
#define init_local_spin_delay(status) init_spin_delay(status, __FILE__, __LINE__, __func__)
Add missing 'extern' to function prototypes. Postgres style is to spell out extern. Noticed while scripting adding PGDLLIMPORT markers to functions. Discussion: https://postgr.es/m/20220512164513.vaheofqp2q24l65r@alap3.anarazel.de
2022-05-12 18:17:14 +02:00
extern void perform_spin_delay(SpinDelayStatus *status);
extern void finish_spin_delay(SpinDelayStatus *status);
Allow Pin/UnpinBuffer to operate in a lockfree manner. Pinning/Unpinning a buffer is a very frequent operation; especially in read-mostly cache resident workloads. Benchmarking shows that in various scenarios the spinlock protecting a buffer header's state becomes a significant bottleneck. The problem can be reproduced with pgbench -S on larger machines, but can be considerably worse for queries which touch the same buffers over and over at a high frequency (e.g. nested loops over a small inner table). To allow atomic operations to be used, cram BufferDesc's flags, usage_count, buf_hdr_lock, refcount into a single 32bit atomic variable; that allows to manipulate them together using 32bit compare-and-swap operations. This requires reducing MAX_BACKENDS to 2^18-1 (which could be lifted by using a 64bit field, but it's not a realistic configuration atm). As not all operations can easily implemented in a lockfree manner, implement the previous buf_hdr_lock via a flag bit in the atomic variable. That way we can continue to lock the header in places where it's needed, but can get away without acquiring it in the more frequent hot-paths. There's some additional operations which can be done without the lock, but aren't in this patch; but the most important places are covered. As bufmgr.c now essentially re-implements spinlocks, abstract the delay logic from s_lock.c into something more generic. It now has already two users, and more are coming up; there's a follupw patch for lwlock.c at least. This patch is based on a proof-of-concept written by me, which Alexander Korotkov made into a fully working patch; the committed version is again revised by me. Benchmarking and testing has, amongst others, been provided by Dilip Kumar, Alexander Korotkov, Robert Haas. On a large x86 system improvements for readonly pgbench, with a high client count, of a factor of 8 have been observed. Author: Alexander Korotkov and Andres Freund Discussion: 2400449.GjM57CE0Yg@dinodell
2016-04-11 05:12:32 +02:00
Significant cleanups in SysV IPC handling (shared mem and semaphores). IPC key assignment will now work correctly even when multiple postmasters are using same logical port number (which is possible given -k switch). There is only one shared-mem segment per postmaster now, not 3. Rip out broken code for non-TAS case in bufmgr and xlog, substitute a complete S_LOCK emulation using semaphores in spin.c. TAS and non-TAS logic is now exactly the same. When deadlock is detected, "Deadlock detected" is now the elog(ERROR) message, rather than a NOTICE that comes out before an unhelpful ERROR.
2000-11-29 00:27:57 +01:00
#endif /* S_LOCK_H */