2018-03-22 03:28:28 +01:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
|
|
|
* llvmjit.c
|
|
|
|
* Core part of the LLVM JIT provider.
|
|
|
|
*
|
2024-01-04 02:49:05 +01:00
|
|
|
* Copyright (c) 2016-2024, PostgreSQL Global Development Group
|
2018-03-22 03:28:28 +01:00
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
|
|
|
* src/backend/jit/llvm/llvmjit.c
|
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include "postgres.h"
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
#include <llvm-c/Analysis.h>
|
|
|
|
#include <llvm-c/BitReader.h>
|
|
|
|
#include <llvm-c/BitWriter.h>
|
|
|
|
#include <llvm-c/Core.h>
|
2018-07-23 06:13:20 +02:00
|
|
|
#include <llvm-c/ExecutionEngine.h>
|
2023-10-18 11:15:54 +02:00
|
|
|
#if LLVM_VERSION_MAJOR > 16
|
|
|
|
#include <llvm-c/Transforms/PassBuilder.h>
|
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
#include <llvm-c/Orc.h>
|
|
|
|
#include <llvm-c/OrcEE.h>
|
|
|
|
#include <llvm-c/LLJIT.h>
|
|
|
|
#else
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
#include <llvm-c/OrcBindings.h>
|
2020-11-10 05:01:33 +01:00
|
|
|
#endif
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
#include <llvm-c/Support.h>
|
2018-03-22 03:28:28 +01:00
|
|
|
#include <llvm-c/Target.h>
|
2023-10-18 11:15:54 +02:00
|
|
|
#if LLVM_VERSION_MAJOR < 17
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
#include <llvm-c/Transforms/IPO.h>
|
|
|
|
#include <llvm-c/Transforms/PassManagerBuilder.h>
|
|
|
|
#include <llvm-c/Transforms/Scalar.h>
|
2018-07-23 06:05:06 +02:00
|
|
|
#include <llvm-c/Transforms/Utils.h>
|
|
|
|
#endif
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
2019-11-12 04:00:16 +01:00
|
|
|
#include "jit/llvmjit.h"
|
|
|
|
#include "jit/llvmjit_emit.h"
|
|
|
|
#include "miscadmin.h"
|
|
|
|
#include "portability/instr_time.h"
|
|
|
|
#include "storage/ipc.h"
|
|
|
|
#include "utils/memutils.h"
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 12:30:50 +01:00
|
|
|
#include "utils/resowner.h"
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
#define LLVMJIT_LLVM_CONTEXT_REUSE_MAX 100
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
/* Handle of a module emitted via ORC JIT */
|
|
|
|
typedef struct LLVMJitHandle
|
|
|
|
{
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
LLVMOrcLLJITRef lljit;
|
|
|
|
LLVMOrcResourceTrackerRef resource_tracker;
|
|
|
|
#else
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMOrcJITStackRef stack;
|
|
|
|
LLVMOrcModuleHandle orc_handle;
|
2020-11-10 05:01:33 +01:00
|
|
|
#endif
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
} LLVMJitHandle;
|
|
|
|
|
|
|
|
|
|
|
|
/* types & functions commonly needed for JITing */
|
|
|
|
LLVMTypeRef TypeSizeT;
|
2018-03-23 06:15:51 +01:00
|
|
|
LLVMTypeRef TypeParamBool;
|
|
|
|
LLVMTypeRef TypeStorageBool;
|
2018-02-05 18:09:28 +01:00
|
|
|
LLVMTypeRef TypePGFunction;
|
Change function call information to be variable length.
Before this change FunctionCallInfoData, the struct arguments etc for
V1 function calls are stored in, always had space for
FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two
arrays. For nearly every function call 100 arguments is far more than
needed, therefore wasting memory. Arg and argnull being two separate
arrays also guarantees that to access a single argument, two
cachelines have to be touched.
Change the layout so there's a single variable-length array with pairs
of value / isnull. That drastically reduces memory consumption for
most function calls (on x86-64 a two argument function now uses
64bytes, previously 936 bytes), and makes it very likely that argument
value and its nullness are on the same cacheline.
Arguments are stored in a new NullableDatum struct, which, due to
padding, needs more memory per argument than before. But as usually
far fewer arguments are stored, and individual arguments are cheaper
to access, that's still a clear win. It's likely that there's other
places where conversion to NullableDatum arrays would make sense,
e.g. TupleTableSlots, but that's for another commit.
Because the function call information is now variable-length
allocations have to take the number of arguments into account. For
heap allocations that can be done with SizeForFunctionCallInfoData(),
for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro
that helps to allocate an appropriately sized and aligned variable.
Some places with stack allocation function call information don't know
the number of arguments at compile time, and currently variably sized
stack allocations aren't allowed in postgres. Therefore allow for
FUNC_MAX_ARGS space in these cases. They're not that common, so for
now that seems acceptable.
Because of the need to allocate FunctionCallInfo of the appropriate
size, older extensions may need to update their code. To avoid subtle
breakages, the FunctionCallInfoData struct has been renamed to
FunctionCallInfoBaseData. Most code only references FunctionCallInfo,
so that shouldn't cause much collateral damage.
This change is also a prerequisite for more efficient expression JIT
compilation (by allocating the function call information on the stack,
allowing LLVM to optimize it away); previously the size of the call
information caused problems inside LLVM's optimizer.
Author: Andres Freund
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
|
|
|
LLVMTypeRef StructNullableDatum;
|
2018-02-05 18:09:28 +01:00
|
|
|
LLVMTypeRef StructHeapTupleData;
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
LLVMTypeRef StructMinimalTupleData;
|
2019-01-15 01:15:20 +01:00
|
|
|
LLVMTypeRef StructTupleDescData;
|
2018-02-05 18:09:28 +01:00
|
|
|
LLVMTypeRef StructTupleTableSlot;
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
LLVMTypeRef StructHeapTupleHeaderData;
|
Make TupleTableSlots extensible, finish split of existing slot type.
This commit completes the work prepared in 1a0586de36, splitting the
old TupleTableSlot implementation (which could store buffer, heap,
minimal and virtual slots) into four different slot types. As
described in the aforementioned commit, this is done with the goal of
making tuple table slots extensible, to allow for pluggable table
access methods.
To achieve runtime extensibility for TupleTableSlots, operations on
slots that can differ between types of slots are performed using the
TupleTableSlotOps struct provided at slot creation time. That
includes information from the size of TupleTableSlot struct to be
allocated, initialization, deforming etc. See the struct's definition
for more detailed information about callbacks TupleTableSlotOps.
I decided to rename TTSOpsBufferTuple to TTSOpsBufferHeapTuple and
ExecCopySlotTuple to ExecCopySlotHeapTuple, as that seems more
consistent with other naming introduced in recent patches.
There's plenty optimization potential in the slot implementation, but
according to benchmarking the state after this commit has similar
performance characteristics to before this set of changes, which seems
sufficient.
There's a few changes in execReplication.c that currently need to poke
through the slot abstraction, that'll be repaired once the pluggable
storage patchset provides the necessary infrastructure.
Author: Andres Freund and Ashutosh Bapat, with changes by Amit Khandekar
Discussion: https://postgr.es/m/20181105210039.hh4vvi4vwoq5ba2q@alap3.anarazel.de
2018-11-17 01:35:11 +01:00
|
|
|
LLVMTypeRef StructHeapTupleTableSlot;
|
|
|
|
LLVMTypeRef StructMinimalTupleTableSlot;
|
2018-02-05 18:09:28 +01:00
|
|
|
LLVMTypeRef StructMemoryContextData;
|
|
|
|
LLVMTypeRef StructFunctionCallInfoData;
|
|
|
|
LLVMTypeRef StructExprContext;
|
|
|
|
LLVMTypeRef StructExprEvalStep;
|
|
|
|
LLVMTypeRef StructExprState;
|
|
|
|
LLVMTypeRef StructAggState;
|
|
|
|
LLVMTypeRef StructAggStatePerGroupData;
|
|
|
|
LLVMTypeRef StructAggStatePerTransData;
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
LLVMTypeRef StructPlanState;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
|
|
|
LLVMValueRef AttributeTemplate;
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
LLVMValueRef ExecEvalSubroutineTemplate;
|
|
|
|
LLVMValueRef ExecEvalBoolSubroutineTemplate;
|
2018-03-22 03:28:28 +01:00
|
|
|
|
2023-09-27 13:02:14 +02:00
|
|
|
static LLVMModuleRef llvm_types_module = NULL;
|
2018-03-22 03:28:28 +01:00
|
|
|
|
|
|
|
static bool llvm_session_initialized = false;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
static size_t llvm_generation = 0;
|
2023-09-27 13:02:21 +02:00
|
|
|
|
|
|
|
/* number of LLVMJitContexts that currently are in use */
|
|
|
|
static size_t llvm_jit_context_in_use_count = 0;
|
|
|
|
|
|
|
|
/* how many times has the current LLVMContextRef been used */
|
|
|
|
static size_t llvm_llvm_context_reuse_count = 0;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
static const char *llvm_triple = NULL;
|
|
|
|
static const char *llvm_layout = NULL;
|
2023-09-27 13:02:21 +02:00
|
|
|
static LLVMContextRef llvm_context;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
|
|
|
|
|
|
|
static LLVMTargetRef llvm_targetref;
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
static LLVMOrcThreadSafeContextRef llvm_ts_context;
|
|
|
|
static LLVMOrcLLJITRef llvm_opt0_orc;
|
|
|
|
static LLVMOrcLLJITRef llvm_opt3_orc;
|
|
|
|
#else /* LLVM_VERSION_MAJOR > 11 */
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
static LLVMOrcJITStackRef llvm_opt0_orc;
|
|
|
|
static LLVMOrcJITStackRef llvm_opt3_orc;
|
2020-11-10 05:01:33 +01:00
|
|
|
#endif /* LLVM_VERSION_MAJOR > 11 */
|
2018-03-22 03:28:28 +01:00
|
|
|
|
|
|
|
|
|
|
|
static void llvm_release_context(JitContext *context);
|
|
|
|
static void llvm_session_initialize(void);
|
|
|
|
static void llvm_shutdown(int code, Datum arg);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
static void llvm_compile_module(LLVMJitContext *context);
|
|
|
|
static void llvm_optimize_module(LLVMJitContext *context, LLVMModuleRef module);
|
|
|
|
|
|
|
|
static void llvm_create_types(void);
|
2023-09-27 13:02:21 +02:00
|
|
|
static void llvm_set_target(void);
|
|
|
|
static void llvm_recreate_llvm_context(void);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
static uint64_t llvm_resolve_symbol(const char *name, void *ctx);
|
2018-03-22 03:28:28 +01:00
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
static LLVMOrcLLJITRef llvm_create_jit_instance(LLVMTargetMachineRef tm);
|
|
|
|
static char *llvm_error_message(LLVMErrorRef error);
|
|
|
|
#endif /* LLVM_VERSION_MAJOR > 11 */
|
2018-03-22 03:28:28 +01:00
|
|
|
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 12:30:50 +01:00
|
|
|
/* ResourceOwner callbacks to hold JitContexts */
|
|
|
|
static void ResOwnerReleaseJitContext(Datum res);
|
|
|
|
|
|
|
|
static const ResourceOwnerDesc jit_resowner_desc =
|
|
|
|
{
|
|
|
|
.name = "LLVM JIT context",
|
|
|
|
.release_phase = RESOURCE_RELEASE_BEFORE_LOCKS,
|
|
|
|
.release_priority = RELEASE_PRIO_JIT_CONTEXTS,
|
|
|
|
.ReleaseResource = ResOwnerReleaseJitContext,
|
|
|
|
.DebugPrint = NULL /* the default message is fine */
|
|
|
|
};
|
|
|
|
|
|
|
|
/* Convenience wrappers over ResourceOwnerRemember/Forget */
|
|
|
|
static inline void
|
|
|
|
ResourceOwnerRememberJIT(ResourceOwner owner, LLVMJitContext *handle)
|
|
|
|
{
|
|
|
|
ResourceOwnerRemember(owner, PointerGetDatum(handle), &jit_resowner_desc);
|
|
|
|
}
|
|
|
|
static inline void
|
|
|
|
ResourceOwnerForgetJIT(ResourceOwner owner, LLVMJitContext *handle)
|
|
|
|
{
|
|
|
|
ResourceOwnerForget(owner, PointerGetDatum(handle), &jit_resowner_desc);
|
|
|
|
}
|
|
|
|
|
2018-03-22 03:28:28 +01:00
|
|
|
PG_MODULE_MAGIC;
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Initialize LLVM JIT provider.
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
_PG_jit_provider_init(JitProviderCallbacks *cb)
|
|
|
|
{
|
|
|
|
cb->reset_after_error = llvm_reset_after_error;
|
|
|
|
cb->release_context = llvm_release_context;
|
2018-03-20 10:20:46 +01:00
|
|
|
cb->compile_expr = llvm_compile_expr;
|
2018-03-22 03:28:28 +01:00
|
|
|
}
|
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Every now and then create a new LLVMContextRef. Unfortunately, during every
|
|
|
|
* round of inlining, types may "leak" (they can still be found/used via the
|
|
|
|
* context, but new types will be created the next time in inlining is
|
|
|
|
* performed). To prevent that from slowly accumulating problematic amounts of
|
|
|
|
* memory, recreate the LLVMContextRef we use. We don't want to do so too
|
|
|
|
* often, as that implies some overhead (particularly re-loading the module
|
|
|
|
* summaries / modules is fairly expensive). A future TODO would be to make
|
|
|
|
* this more finegrained and only drop/recreate the LLVMContextRef when we know
|
|
|
|
* there has been inlining. If we can get the size of the context from LLVM
|
|
|
|
* then that might be a better way to determine when to drop/recreate rather
|
|
|
|
* then the usagecount heuristic currently employed.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_recreate_llvm_context(void)
|
|
|
|
{
|
|
|
|
if (!llvm_context)
|
|
|
|
elog(ERROR, "Trying to recreate a non-existing context");
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We can only safely recreate the LLVM context if no other code is being
|
|
|
|
* JITed, otherwise we'd release the types in use for that.
|
|
|
|
*/
|
|
|
|
if (llvm_jit_context_in_use_count > 0)
|
|
|
|
{
|
|
|
|
llvm_llvm_context_reuse_count++;
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (llvm_llvm_context_reuse_count <= LLVMJIT_LLVM_CONTEXT_REUSE_MAX)
|
|
|
|
{
|
|
|
|
llvm_llvm_context_reuse_count++;
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Need to reset the modules that the inlining code caches before
|
|
|
|
* disposing of the context. LLVM modules exist within a specific LLVM
|
|
|
|
* context, therefore disposing of the context before resetting the cache
|
|
|
|
* would lead to dangling pointers to modules.
|
|
|
|
*/
|
|
|
|
llvm_inline_reset_caches();
|
|
|
|
|
|
|
|
LLVMContextDispose(llvm_context);
|
|
|
|
llvm_context = LLVMContextCreate();
|
|
|
|
llvm_llvm_context_reuse_count = 0;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Re-build cached type information, so code generation code can rely on
|
|
|
|
* that information to be present (also prevents the variables to be
|
|
|
|
* dangling references).
|
|
|
|
*/
|
|
|
|
llvm_create_types();
|
|
|
|
}
|
|
|
|
|
|
|
|
|
2018-03-22 03:28:28 +01:00
|
|
|
/*
|
|
|
|
* Create a context for JITing work.
|
|
|
|
*
|
|
|
|
* The context, including subsidiary resources, will be cleaned up either when
|
|
|
|
* the context is explicitly released, or when the lifetime of
|
|
|
|
* CurrentResourceOwner ends (usually the end of the current [sub]xact).
|
|
|
|
*/
|
|
|
|
LLVMJitContext *
|
|
|
|
llvm_create_context(int jitFlags)
|
|
|
|
{
|
|
|
|
LLVMJitContext *context;
|
|
|
|
|
|
|
|
llvm_assert_in_fatal_section();
|
|
|
|
|
|
|
|
llvm_session_initialize();
|
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
llvm_recreate_llvm_context();
|
|
|
|
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 12:30:50 +01:00
|
|
|
ResourceOwnerEnlarge(CurrentResourceOwner);
|
2018-03-22 03:28:28 +01:00
|
|
|
|
|
|
|
context = MemoryContextAllocZero(TopMemoryContext,
|
|
|
|
sizeof(LLVMJitContext));
|
|
|
|
context->base.flags = jitFlags;
|
|
|
|
|
|
|
|
/* ensure cleanup */
|
|
|
|
context->base.resowner = CurrentResourceOwner;
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 12:30:50 +01:00
|
|
|
ResourceOwnerRememberJIT(CurrentResourceOwner, context);
|
2018-03-22 03:28:28 +01:00
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
llvm_jit_context_in_use_count++;
|
|
|
|
|
2018-03-22 03:28:28 +01:00
|
|
|
return context;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Release resources required by one llvm context.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_release_context(JitContext *context)
|
|
|
|
{
|
2023-09-27 13:02:21 +02:00
|
|
|
LLVMJitContext *llvm_jit_context = (LLVMJitContext *) context;
|
2021-11-01 21:24:39 +01:00
|
|
|
ListCell *lc;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
/*
|
|
|
|
* Consider as cleaned up even if we skip doing so below, that way we can
|
|
|
|
* verify the tracking is correct (see llvm_shutdown()).
|
|
|
|
*/
|
|
|
|
llvm_jit_context_in_use_count--;
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
/*
|
|
|
|
* When this backend is exiting, don't clean up LLVM. As an error might
|
|
|
|
* have occurred from within LLVM, we do not want to risk reentering. All
|
|
|
|
* resource cleanup is going to happen through process exit.
|
|
|
|
*/
|
2020-11-10 05:01:33 +01:00
|
|
|
if (proc_exit_inprogress)
|
|
|
|
return;
|
|
|
|
|
2021-09-14 03:07:19 +02:00
|
|
|
llvm_enter_fatal_on_oom();
|
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
if (llvm_jit_context->module)
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
{
|
2023-09-27 13:02:21 +02:00
|
|
|
LLVMDisposeModule(llvm_jit_context->module);
|
|
|
|
llvm_jit_context->module = NULL;
|
2020-11-10 05:01:33 +01:00
|
|
|
}
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
foreach(lc, llvm_jit_context->handles)
|
2020-11-10 05:01:33 +01:00
|
|
|
{
|
2021-11-01 21:24:39 +01:00
|
|
|
LLVMJitHandle *jit_handle = (LLVMJitHandle *) lfirst(lc);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
{
|
|
|
|
LLVMOrcExecutionSessionRef ee;
|
|
|
|
LLVMOrcSymbolStringPoolRef sp;
|
|
|
|
|
|
|
|
LLVMOrcResourceTrackerRemove(jit_handle->resource_tracker);
|
|
|
|
LLVMOrcReleaseResourceTracker(jit_handle->resource_tracker);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Without triggering cleanup of the string pool, we'd leak
|
|
|
|
* memory. It'd be sufficient to do this far less often, but in
|
|
|
|
* experiments the required time was small enough to just always
|
|
|
|
* do it.
|
|
|
|
*/
|
|
|
|
ee = LLVMOrcLLJITGetExecutionSession(jit_handle->lljit);
|
|
|
|
sp = LLVMOrcExecutionSessionGetSymbolStringPool(ee);
|
|
|
|
LLVMOrcSymbolStringPoolClearDeadEntries(sp);
|
|
|
|
}
|
|
|
|
#else /* LLVM_VERSION_MAJOR > 11 */
|
|
|
|
{
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMOrcRemoveModule(jit_handle->stack, jit_handle->orc_handle);
|
|
|
|
}
|
2020-11-10 05:01:33 +01:00
|
|
|
#endif /* LLVM_VERSION_MAJOR > 11 */
|
|
|
|
|
|
|
|
pfree(jit_handle);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
2023-09-27 13:02:21 +02:00
|
|
|
list_free(llvm_jit_context->handles);
|
|
|
|
llvm_jit_context->handles = NIL;
|
2023-07-05 12:13:13 +02:00
|
|
|
|
|
|
|
llvm_leave_fatal_on_oom();
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 12:30:50 +01:00
|
|
|
|
|
|
|
if (context->resowner)
|
|
|
|
ResourceOwnerForgetJIT(context->resowner, llvm_jit_context);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Return module which may be modified, e.g. by creating new functions.
|
|
|
|
*/
|
|
|
|
LLVMModuleRef
|
|
|
|
llvm_mutable_module(LLVMJitContext *context)
|
|
|
|
{
|
|
|
|
llvm_assert_in_fatal_section();
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If there's no in-progress module, create a new one.
|
|
|
|
*/
|
|
|
|
if (!context->module)
|
|
|
|
{
|
|
|
|
context->compiled = false;
|
|
|
|
context->module_generation = llvm_generation++;
|
2023-09-27 13:02:21 +02:00
|
|
|
context->module = LLVMModuleCreateWithNameInContext("pg", llvm_context);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMSetTarget(context->module, llvm_triple);
|
|
|
|
LLVMSetDataLayout(context->module, llvm_layout);
|
|
|
|
}
|
|
|
|
|
|
|
|
return context->module;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Expand function name to be non-conflicting. This should be used by code
|
|
|
|
* generating code, when adding new externally visible function definitions to
|
|
|
|
* a Module.
|
|
|
|
*/
|
|
|
|
char *
|
|
|
|
llvm_expand_funcname(struct LLVMJitContext *context, const char *basename)
|
|
|
|
{
|
|
|
|
Assert(context->module != NULL);
|
|
|
|
|
2018-09-25 21:54:29 +02:00
|
|
|
context->base.instr.created_functions++;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Previously we used dots to separate, but turns out some tools, e.g.
|
|
|
|
* GDB, don't like that and truncate name.
|
|
|
|
*/
|
|
|
|
return psprintf("%s_%zu_%d",
|
|
|
|
basename,
|
|
|
|
context->module_generation,
|
|
|
|
context->counter++);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Return pointer to function funcname, which has to exist. If there's pending
|
|
|
|
* code to be optimized and emitted, do so first.
|
|
|
|
*/
|
|
|
|
void *
|
|
|
|
llvm_get_function(LLVMJitContext *context, const char *funcname)
|
|
|
|
{
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
llvm_assert_in_fatal_section();
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If there is a pending / not emitted module, compile and emit now.
|
2018-03-26 21:58:17 +02:00
|
|
|
* Otherwise we might not find the [correct] function.
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
*/
|
|
|
|
if (!context->compiled)
|
|
|
|
{
|
|
|
|
llvm_compile_module(context);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* ORC's symbol table is of *unmangled* symbols. Therefore we don't need
|
|
|
|
* to mangle here.
|
|
|
|
*/
|
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
foreach(lc, context->handles)
|
|
|
|
{
|
|
|
|
LLVMJitHandle *handle = (LLVMJitHandle *) lfirst(lc);
|
|
|
|
instr_time starttime;
|
|
|
|
instr_time endtime;
|
|
|
|
LLVMErrorRef error;
|
|
|
|
LLVMOrcJITTargetAddress addr;
|
|
|
|
|
|
|
|
INSTR_TIME_SET_CURRENT(starttime);
|
|
|
|
|
|
|
|
addr = 0;
|
|
|
|
error = LLVMOrcLLJITLookup(handle->lljit, &addr, funcname);
|
|
|
|
if (error)
|
|
|
|
elog(ERROR, "failed to look up symbol \"%s\": %s",
|
|
|
|
funcname, llvm_error_message(error));
|
|
|
|
|
|
|
|
/*
|
|
|
|
* LLJIT only actually emits code the first time a symbol is
|
|
|
|
* referenced. Thus add lookup time to emission time. That's counting
|
|
|
|
* a bit more than with older LLVM versions, but unlikely to ever
|
|
|
|
* matter.
|
|
|
|
*/
|
|
|
|
INSTR_TIME_SET_CURRENT(endtime);
|
|
|
|
INSTR_TIME_ACCUM_DIFF(context->base.instr.emission_counter,
|
|
|
|
endtime, starttime);
|
|
|
|
|
|
|
|
if (addr)
|
|
|
|
return (void *) (uintptr_t) addr;
|
|
|
|
}
|
2024-01-25 03:23:04 +01:00
|
|
|
#else
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
foreach(lc, context->handles)
|
|
|
|
{
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMOrcTargetAddress addr;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMJitHandle *handle = (LLVMJitHandle *) lfirst(lc);
|
|
|
|
|
|
|
|
addr = 0;
|
|
|
|
if (LLVMOrcGetSymbolAddressIn(handle->stack, &addr, handle->orc_handle, funcname))
|
2018-03-26 21:58:17 +02:00
|
|
|
elog(ERROR, "failed to look up symbol \"%s\"", funcname);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
if (addr)
|
|
|
|
return (void *) (uintptr_t) addr;
|
|
|
|
}
|
2020-11-10 05:01:33 +01:00
|
|
|
#endif
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
|
|
|
elog(ERROR, "failed to JIT: %s", funcname);
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
2020-12-07 22:16:55 +01:00
|
|
|
/*
|
|
|
|
* Return type of a variable in llvmjit_types.c. This is useful to keep types
|
|
|
|
* in sync between plain C and JIT related code.
|
|
|
|
*/
|
|
|
|
LLVMTypeRef
|
|
|
|
llvm_pg_var_type(const char *varname)
|
|
|
|
{
|
|
|
|
LLVMValueRef v_srcvar;
|
|
|
|
LLVMTypeRef typ;
|
|
|
|
|
|
|
|
/* this'll return a *pointer* to the global */
|
|
|
|
v_srcvar = LLVMGetNamedGlobal(llvm_types_module, varname);
|
|
|
|
if (!v_srcvar)
|
|
|
|
elog(ERROR, "variable %s not in llvmjit_types.c", varname);
|
|
|
|
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
typ = LLVMGlobalGetValueType(v_srcvar);
|
2020-12-07 22:16:55 +01:00
|
|
|
|
|
|
|
return typ;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Return function type of a variable in llvmjit_types.c. This is useful to
|
|
|
|
* keep function types in sync between C and JITed code.
|
|
|
|
*/
|
|
|
|
LLVMTypeRef
|
|
|
|
llvm_pg_var_func_type(const char *varname)
|
|
|
|
{
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
LLVMValueRef v_srcvar;
|
|
|
|
LLVMTypeRef typ;
|
2020-12-07 22:16:55 +01:00
|
|
|
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
v_srcvar = LLVMGetNamedFunction(llvm_types_module, varname);
|
|
|
|
if (!v_srcvar)
|
|
|
|
elog(ERROR, "function %s not in llvmjit_types.c", varname);
|
|
|
|
|
|
|
|
typ = LLVMGetFunctionType(v_srcvar);
|
2020-12-07 22:16:55 +01:00
|
|
|
|
|
|
|
return typ;
|
|
|
|
}
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
/*
|
2020-02-07 07:13:52 +01:00
|
|
|
* Return declaration for a function referenced in llvmjit_types.c, adding it
|
|
|
|
* to the module if necessary.
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
*
|
2020-02-07 07:13:52 +01:00
|
|
|
* This is used to make functions discovered via llvm_create_types() known to
|
|
|
|
* the module that's currently being worked on.
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
*/
|
|
|
|
LLVMValueRef
|
2020-02-07 07:13:52 +01:00
|
|
|
llvm_pg_func(LLVMModuleRef mod, const char *funcname)
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
{
|
2020-02-07 07:13:52 +01:00
|
|
|
LLVMValueRef v_srcfn;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMValueRef v_fn;
|
|
|
|
|
|
|
|
/* don't repeatedly add function */
|
2020-02-07 07:13:52 +01:00
|
|
|
v_fn = LLVMGetNamedFunction(mod, funcname);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
if (v_fn)
|
|
|
|
return v_fn;
|
|
|
|
|
2020-02-07 07:13:52 +01:00
|
|
|
v_srcfn = LLVMGetNamedFunction(llvm_types_module, funcname);
|
|
|
|
|
|
|
|
if (!v_srcfn)
|
|
|
|
elog(ERROR, "function %s not in llvmjit_types.c", funcname);
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
v_fn = LLVMAddFunction(mod,
|
2020-02-07 07:13:52 +01:00
|
|
|
funcname,
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
LLVMGetFunctionType(v_srcfn));
|
2020-02-07 07:13:52 +01:00
|
|
|
llvm_copy_attributes(v_srcfn, v_fn);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
|
|
|
return v_fn;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2020-10-15 22:39:41 +02:00
|
|
|
* Copy attributes from one function to another, for a specific index (an
|
|
|
|
* index can reference return value, function and parameter attributes).
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
*/
|
2020-10-15 22:39:41 +02:00
|
|
|
static void
|
|
|
|
llvm_copy_attributes_at_index(LLVMValueRef v_from, LLVMValueRef v_to, uint32 index)
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
{
|
|
|
|
int num_attributes;
|
|
|
|
LLVMAttributeRef *attrs;
|
|
|
|
|
2024-01-25 03:23:04 +01:00
|
|
|
num_attributes = LLVMGetAttributeCountAtIndex(v_from, index);
|
2020-10-16 02:38:00 +02:00
|
|
|
|
|
|
|
if (num_attributes == 0)
|
|
|
|
return;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
|
|
|
attrs = palloc(sizeof(LLVMAttributeRef) * num_attributes);
|
2020-10-15 22:39:41 +02:00
|
|
|
LLVMGetAttributesAtIndex(v_from, index, attrs);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
2020-10-15 22:39:41 +02:00
|
|
|
for (int attno = 0; attno < num_attributes; attno++)
|
|
|
|
LLVMAddAttributeAtIndex(v_to, index, attrs[attno]);
|
|
|
|
|
|
|
|
pfree(attrs);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Copy all attributes from one function to another. I.e. function, return and
|
|
|
|
* parameters will be copied.
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
llvm_copy_attributes(LLVMValueRef v_from, LLVMValueRef v_to)
|
|
|
|
{
|
|
|
|
uint32 param_count;
|
|
|
|
|
|
|
|
/* copy function attributes */
|
|
|
|
llvm_copy_attributes_at_index(v_from, v_to, LLVMAttributeFunctionIndex);
|
|
|
|
|
|
|
|
/* and the return value attributes */
|
|
|
|
llvm_copy_attributes_at_index(v_from, v_to, LLVMAttributeReturnIndex);
|
|
|
|
|
|
|
|
/* and each function parameter's attribute */
|
|
|
|
param_count = LLVMCountParams(v_from);
|
|
|
|
|
|
|
|
for (int paramidx = 1; paramidx <= param_count; paramidx++)
|
|
|
|
llvm_copy_attributes_at_index(v_from, v_to, paramidx);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
|
|
|
|
2018-03-20 10:20:46 +01:00
|
|
|
/*
|
|
|
|
* Return a callable LLVMValueRef for fcinfo.
|
|
|
|
*/
|
|
|
|
LLVMValueRef
|
|
|
|
llvm_function_reference(LLVMJitContext *context,
|
|
|
|
LLVMBuilderRef builder,
|
|
|
|
LLVMModuleRef mod,
|
|
|
|
FunctionCallInfo fcinfo)
|
|
|
|
{
|
|
|
|
char *modname;
|
|
|
|
char *basename;
|
|
|
|
char *funcname;
|
|
|
|
|
|
|
|
LLVMValueRef v_fn;
|
|
|
|
|
|
|
|
fmgr_symbol(fcinfo->flinfo->fn_oid, &modname, &basename);
|
|
|
|
|
|
|
|
if (modname != NULL && basename != NULL)
|
|
|
|
{
|
|
|
|
/* external function in loadable library */
|
|
|
|
funcname = psprintf("pgextern.%s.%s", modname, basename);
|
|
|
|
}
|
|
|
|
else if (basename != NULL)
|
|
|
|
{
|
|
|
|
/* internal function */
|
2022-09-06 03:19:44 +02:00
|
|
|
funcname = pstrdup(basename);
|
2018-03-20 10:20:46 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Function we don't know to handle, return pointer. We do so by
|
|
|
|
* creating a global constant containing a pointer to the function.
|
|
|
|
* Makes IR more readable.
|
|
|
|
*/
|
|
|
|
LLVMValueRef v_fn_addr;
|
|
|
|
|
|
|
|
funcname = psprintf("pgoidextern.%u",
|
|
|
|
fcinfo->flinfo->fn_oid);
|
|
|
|
v_fn = LLVMGetNamedGlobal(mod, funcname);
|
|
|
|
if (v_fn != 0)
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
return l_load(builder, TypePGFunction, v_fn, "");
|
2018-03-20 10:20:46 +01:00
|
|
|
|
|
|
|
v_fn_addr = l_ptr_const(fcinfo->flinfo->fn_addr, TypePGFunction);
|
|
|
|
|
|
|
|
v_fn = LLVMAddGlobal(mod, TypePGFunction, funcname);
|
|
|
|
LLVMSetInitializer(v_fn, v_fn_addr);
|
|
|
|
LLVMSetGlobalConstant(v_fn, true);
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMSetLinkage(v_fn, LLVMPrivateLinkage);
|
|
|
|
LLVMSetUnnamedAddr(v_fn, true);
|
2018-03-20 10:20:46 +01:00
|
|
|
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
return l_load(builder, TypePGFunction, v_fn, "");
|
2018-03-20 10:20:46 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/* check if function already has been added */
|
|
|
|
v_fn = LLVMGetNamedFunction(mod, funcname);
|
|
|
|
if (v_fn != 0)
|
|
|
|
return v_fn;
|
|
|
|
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
v_fn = LLVMAddFunction(mod, funcname, LLVMGetFunctionType(AttributeTemplate));
|
2018-03-20 10:20:46 +01:00
|
|
|
|
|
|
|
return v_fn;
|
|
|
|
}
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
/*
|
|
|
|
* Optimize code in module using the flags set in context.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_optimize_module(LLVMJitContext *context, LLVMModuleRef module)
|
|
|
|
{
|
2023-10-18 11:15:54 +02:00
|
|
|
#if LLVM_VERSION_MAJOR < 17
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMPassManagerBuilderRef llvm_pmb;
|
|
|
|
LLVMPassManagerRef llvm_mpm;
|
|
|
|
LLVMPassManagerRef llvm_fpm;
|
|
|
|
LLVMValueRef func;
|
|
|
|
int compile_optlevel;
|
|
|
|
|
|
|
|
if (context->base.flags & PGJIT_OPT3)
|
|
|
|
compile_optlevel = 3;
|
|
|
|
else
|
|
|
|
compile_optlevel = 0;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Have to create a new pass manager builder every pass through, as the
|
|
|
|
* inliner has some per-builder state. Otherwise one ends up only inlining
|
|
|
|
* a function the first time though.
|
|
|
|
*/
|
|
|
|
llvm_pmb = LLVMPassManagerBuilderCreate();
|
|
|
|
LLVMPassManagerBuilderSetOptLevel(llvm_pmb, compile_optlevel);
|
|
|
|
llvm_fpm = LLVMCreateFunctionPassManagerForModule(module);
|
|
|
|
|
|
|
|
if (context->base.flags & PGJIT_OPT3)
|
|
|
|
{
|
2018-11-02 13:56:16 +01:00
|
|
|
/* TODO: Unscientifically determined threshold */
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMPassManagerBuilderUseInlinerWithThreshold(llvm_pmb, 512);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* we rely on mem2reg heavily, so emit even in the O0 case */
|
|
|
|
LLVMAddPromoteMemoryToRegisterPass(llvm_fpm);
|
|
|
|
}
|
|
|
|
|
|
|
|
LLVMPassManagerBuilderPopulateFunctionPassManager(llvm_pmb, llvm_fpm);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Do function level optimization. This could be moved to the point where
|
|
|
|
* functions are emitted, to reduce memory usage a bit.
|
|
|
|
*/
|
|
|
|
LLVMInitializeFunctionPassManager(llvm_fpm);
|
|
|
|
for (func = LLVMGetFirstFunction(context->module);
|
|
|
|
func != NULL;
|
|
|
|
func = LLVMGetNextFunction(func))
|
|
|
|
LLVMRunFunctionPassManager(llvm_fpm, func);
|
|
|
|
LLVMFinalizeFunctionPassManager(llvm_fpm);
|
|
|
|
LLVMDisposePassManager(llvm_fpm);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Perform module level optimization. We do so even in the non-optimized
|
|
|
|
* case, so always-inline functions etc get inlined. It's cheap enough.
|
|
|
|
*/
|
|
|
|
llvm_mpm = LLVMCreatePassManager();
|
|
|
|
LLVMPassManagerBuilderPopulateModulePassManager(llvm_pmb,
|
|
|
|
llvm_mpm);
|
|
|
|
/* always use always-inliner pass */
|
|
|
|
if (!(context->base.flags & PGJIT_OPT3))
|
|
|
|
LLVMAddAlwaysInlinerPass(llvm_mpm);
|
2018-03-28 22:19:08 +02:00
|
|
|
/* if doing inlining, but no expensive optimization, add inlining pass */
|
|
|
|
if (context->base.flags & PGJIT_INLINE
|
|
|
|
&& !(context->base.flags & PGJIT_OPT3))
|
|
|
|
LLVMAddFunctionInliningPass(llvm_mpm);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMRunPassManager(llvm_mpm, context->module);
|
|
|
|
LLVMDisposePassManager(llvm_mpm);
|
|
|
|
|
|
|
|
LLVMPassManagerBuilderDispose(llvm_pmb);
|
2023-10-18 11:15:54 +02:00
|
|
|
#else
|
|
|
|
LLVMPassBuilderOptionsRef options;
|
|
|
|
LLVMErrorRef err;
|
|
|
|
const char *passes;
|
|
|
|
|
|
|
|
if (context->base.flags & PGJIT_OPT3)
|
|
|
|
passes = "default<O3>";
|
|
|
|
else
|
|
|
|
passes = "default<O0>,mem2reg";
|
|
|
|
|
|
|
|
options = LLVMCreatePassBuilderOptions();
|
|
|
|
|
|
|
|
#ifdef LLVM_PASS_DEBUG
|
|
|
|
LLVMPassBuilderOptionsSetDebugLogging(options, 1);
|
|
|
|
#endif
|
|
|
|
|
|
|
|
LLVMPassBuilderOptionsSetInlinerThreshold(options, 512);
|
|
|
|
|
|
|
|
err = LLVMRunPasses(module, passes, NULL, options);
|
|
|
|
|
|
|
|
if (err)
|
|
|
|
elog(ERROR, "failed to JIT module: %s", llvm_error_message(err));
|
|
|
|
|
|
|
|
LLVMDisposePassBuilderOptions(options);
|
|
|
|
#endif
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Emit code for the currently pending module.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_compile_module(LLVMJitContext *context)
|
|
|
|
{
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMJitHandle *handle;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
MemoryContext oldcontext;
|
|
|
|
instr_time starttime;
|
|
|
|
instr_time endtime;
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
LLVMOrcLLJITRef compile_orc;
|
|
|
|
#else
|
|
|
|
LLVMOrcJITStackRef compile_orc;
|
|
|
|
#endif
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
|
|
|
if (context->base.flags & PGJIT_OPT3)
|
|
|
|
compile_orc = llvm_opt3_orc;
|
|
|
|
else
|
|
|
|
compile_orc = llvm_opt0_orc;
|
|
|
|
|
2018-03-28 22:19:08 +02:00
|
|
|
/* perform inlining */
|
|
|
|
if (context->base.flags & PGJIT_INLINE)
|
|
|
|
{
|
|
|
|
INSTR_TIME_SET_CURRENT(starttime);
|
|
|
|
llvm_inline(context->module);
|
|
|
|
INSTR_TIME_SET_CURRENT(endtime);
|
2018-09-25 21:54:29 +02:00
|
|
|
INSTR_TIME_ACCUM_DIFF(context->base.instr.inlining_counter,
|
2018-03-28 22:19:08 +02:00
|
|
|
endtime, starttime);
|
|
|
|
}
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
if (jit_dump_bitcode)
|
|
|
|
{
|
|
|
|
char *filename;
|
|
|
|
|
2023-03-10 07:10:43 +01:00
|
|
|
filename = psprintf("%d.%zu.bc",
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
MyProcPid,
|
|
|
|
context->module_generation);
|
|
|
|
LLVMWriteBitcodeToFile(context->module, filename);
|
|
|
|
pfree(filename);
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
/* optimize according to the chosen optimization settings */
|
|
|
|
INSTR_TIME_SET_CURRENT(starttime);
|
|
|
|
llvm_optimize_module(context, context->module);
|
|
|
|
INSTR_TIME_SET_CURRENT(endtime);
|
2018-09-25 21:54:29 +02:00
|
|
|
INSTR_TIME_ACCUM_DIFF(context->base.instr.optimization_counter,
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
endtime, starttime);
|
|
|
|
|
|
|
|
if (jit_dump_bitcode)
|
|
|
|
{
|
|
|
|
char *filename;
|
|
|
|
|
2023-03-10 07:10:43 +01:00
|
|
|
filename = psprintf("%d.%zu.optimized.bc",
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
MyProcPid,
|
|
|
|
context->module_generation);
|
|
|
|
LLVMWriteBitcodeToFile(context->module, filename);
|
|
|
|
pfree(filename);
|
|
|
|
}
|
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
handle = (LLVMJitHandle *)
|
|
|
|
MemoryContextAlloc(TopMemoryContext, sizeof(LLVMJitHandle));
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
/*
|
|
|
|
* Emit the code. Note that this can, depending on the optimization
|
|
|
|
* settings, take noticeable resources as code emission executes low-level
|
|
|
|
* instruction combining/selection passes etc. Without optimization a
|
|
|
|
* faster instruction selection mechanism is used.
|
|
|
|
*/
|
|
|
|
INSTR_TIME_SET_CURRENT(starttime);
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
{
|
|
|
|
LLVMOrcThreadSafeModuleRef ts_module;
|
|
|
|
LLVMErrorRef error;
|
|
|
|
LLVMOrcJITDylibRef jd = LLVMOrcLLJITGetMainJITDylib(compile_orc);
|
|
|
|
|
|
|
|
ts_module = LLVMOrcCreateNewThreadSafeModule(context->module, llvm_ts_context);
|
|
|
|
|
|
|
|
handle->lljit = compile_orc;
|
|
|
|
handle->resource_tracker = LLVMOrcJITDylibCreateResourceTracker(jd);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* NB: This doesn't actually emit code. That happens lazily the first
|
|
|
|
* time a symbol defined in the module is requested. Due to that
|
|
|
|
* llvm_get_function() also accounts for emission time.
|
|
|
|
*/
|
|
|
|
|
|
|
|
context->module = NULL; /* will be owned by LLJIT */
|
|
|
|
error = LLVMOrcLLJITAddLLVMIRModuleWithRT(compile_orc,
|
|
|
|
handle->resource_tracker,
|
|
|
|
ts_module);
|
|
|
|
|
|
|
|
if (error)
|
|
|
|
elog(ERROR, "failed to JIT module: %s",
|
|
|
|
llvm_error_message(error));
|
|
|
|
|
|
|
|
/* LLVMOrcLLJITAddLLVMIRModuleWithRT takes ownership of the module */
|
|
|
|
}
|
2024-01-25 03:23:04 +01:00
|
|
|
#else
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
{
|
2020-11-10 05:01:33 +01:00
|
|
|
handle->stack = compile_orc;
|
|
|
|
if (LLVMOrcAddEagerlyCompiledIR(compile_orc, &handle->orc_handle, context->module,
|
2018-03-27 00:55:16 +02:00
|
|
|
llvm_resolve_symbol, NULL))
|
|
|
|
elog(ERROR, "failed to JIT module");
|
|
|
|
|
|
|
|
/* LLVMOrcAddEagerlyCompiledIR takes ownership of the module */
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
INSTR_TIME_SET_CURRENT(endtime);
|
2018-09-25 21:54:29 +02:00
|
|
|
INSTR_TIME_ACCUM_DIFF(context->base.instr.emission_counter,
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
endtime, starttime);
|
|
|
|
|
|
|
|
context->module = NULL;
|
|
|
|
context->compiled = true;
|
|
|
|
|
|
|
|
/* remember emitted code for cleanup and lookups */
|
|
|
|
oldcontext = MemoryContextSwitchTo(TopMemoryContext);
|
2020-11-10 05:01:33 +01:00
|
|
|
context->handles = lappend(context->handles, handle);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
MemoryContextSwitchTo(oldcontext);
|
|
|
|
|
|
|
|
ereport(DEBUG1,
|
2021-02-17 11:24:46 +01:00
|
|
|
(errmsg_internal("time to inline: %.3fs, opt: %.3fs, emit: %.3fs",
|
2018-09-25 21:54:29 +02:00
|
|
|
INSTR_TIME_GET_DOUBLE(context->base.instr.inlining_counter),
|
|
|
|
INSTR_TIME_GET_DOUBLE(context->base.instr.optimization_counter),
|
|
|
|
INSTR_TIME_GET_DOUBLE(context->base.instr.emission_counter)),
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
errhidestmt(true),
|
|
|
|
errhidecontext(true)));
|
2018-03-22 03:28:28 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Per session initialization.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_session_initialize(void)
|
|
|
|
{
|
|
|
|
MemoryContext oldcontext;
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
char *error = NULL;
|
|
|
|
char *cpu = NULL;
|
|
|
|
char *features = NULL;
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMTargetMachineRef opt0_tm;
|
|
|
|
LLVMTargetMachineRef opt3_tm;
|
2018-03-22 03:28:28 +01:00
|
|
|
|
|
|
|
if (llvm_session_initialized)
|
|
|
|
return;
|
|
|
|
|
|
|
|
oldcontext = MemoryContextSwitchTo(TopMemoryContext);
|
|
|
|
|
|
|
|
LLVMInitializeNativeTarget();
|
|
|
|
LLVMInitializeNativeAsmPrinter();
|
|
|
|
LLVMInitializeNativeAsmParser();
|
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
if (llvm_context == NULL)
|
|
|
|
{
|
|
|
|
llvm_context = LLVMContextCreate();
|
|
|
|
|
|
|
|
llvm_jit_context_in_use_count = 0;
|
|
|
|
llvm_llvm_context_reuse_count = 0;
|
|
|
|
}
|
|
|
|
|
2022-10-19 11:18:26 +02:00
|
|
|
/*
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
* When targeting LLVM 15, turn off opaque pointers for the context we
|
|
|
|
* build our code in. We don't need to do so for other contexts (e.g.
|
|
|
|
* llvm_ts_context). Once the IR is generated, it carries the necessary
|
|
|
|
* information.
|
|
|
|
*
|
|
|
|
* For 16 and above, opaque pointers must be used, and we have special
|
|
|
|
* code for that.
|
2022-10-19 11:18:26 +02:00
|
|
|
*/
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
#if LLVM_VERSION_MAJOR == 15
|
2022-10-19 11:18:26 +02:00
|
|
|
LLVMContextSetOpaquePointers(LLVMGetGlobalContext(), false);
|
|
|
|
#endif
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
/*
|
|
|
|
* Synchronize types early, as that also includes inferring the target
|
|
|
|
* triple.
|
|
|
|
*/
|
|
|
|
llvm_create_types();
|
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
/*
|
|
|
|
* Extract target information from loaded module.
|
|
|
|
*/
|
|
|
|
llvm_set_target();
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
if (LLVMGetTargetFromTriple(llvm_triple, &llvm_targetref, &error) != 0)
|
|
|
|
{
|
2023-01-03 08:26:14 +01:00
|
|
|
elog(FATAL, "failed to query triple %s", error);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We want the generated code to use all available features. Therefore
|
|
|
|
* grab the host CPU string and detect features of the current CPU. The
|
|
|
|
* latter is needed because some CPU architectures default to enabling
|
|
|
|
* features not all CPUs have (weird, huh).
|
|
|
|
*/
|
|
|
|
cpu = LLVMGetHostCPUName();
|
|
|
|
features = LLVMGetHostCPUFeatures();
|
|
|
|
elog(DEBUG2, "LLVMJIT detected CPU \"%s\", with features \"%s\"",
|
|
|
|
cpu, features);
|
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
opt0_tm =
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMCreateTargetMachine(llvm_targetref, llvm_triple, cpu, features,
|
|
|
|
LLVMCodeGenLevelNone,
|
|
|
|
LLVMRelocDefault,
|
|
|
|
LLVMCodeModelJITDefault);
|
2020-11-10 05:01:33 +01:00
|
|
|
opt3_tm =
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
LLVMCreateTargetMachine(llvm_targetref, llvm_triple, cpu, features,
|
|
|
|
LLVMCodeGenLevelAggressive,
|
|
|
|
LLVMRelocDefault,
|
|
|
|
LLVMCodeModelJITDefault);
|
|
|
|
|
|
|
|
LLVMDisposeMessage(cpu);
|
|
|
|
cpu = NULL;
|
|
|
|
LLVMDisposeMessage(features);
|
|
|
|
features = NULL;
|
|
|
|
|
|
|
|
/* force symbols in main binary to be loaded */
|
|
|
|
LLVMLoadLibraryPermanently(NULL);
|
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
2018-03-22 19:07:55 +01:00
|
|
|
{
|
2020-11-10 05:01:33 +01:00
|
|
|
llvm_ts_context = LLVMOrcCreateNewThreadSafeContext();
|
|
|
|
|
|
|
|
llvm_opt0_orc = llvm_create_jit_instance(opt0_tm);
|
|
|
|
opt0_tm = 0;
|
2018-07-23 06:13:20 +02:00
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
llvm_opt3_orc = llvm_create_jit_instance(opt3_tm);
|
|
|
|
opt3_tm = 0;
|
2018-03-22 19:07:55 +01:00
|
|
|
}
|
2020-11-10 05:01:33 +01:00
|
|
|
#else /* LLVM_VERSION_MAJOR > 11 */
|
|
|
|
{
|
|
|
|
llvm_opt0_orc = LLVMOrcCreateInstance(opt0_tm);
|
|
|
|
llvm_opt3_orc = LLVMOrcCreateInstance(opt3_tm);
|
|
|
|
|
|
|
|
#if defined(HAVE_DECL_LLVMCREATEGDBREGISTRATIONLISTENER) && HAVE_DECL_LLVMCREATEGDBREGISTRATIONLISTENER
|
|
|
|
if (jit_debugging_support)
|
|
|
|
{
|
|
|
|
LLVMJITEventListenerRef l = LLVMCreateGDBRegistrationListener();
|
|
|
|
|
|
|
|
LLVMOrcRegisterJITEventListener(llvm_opt0_orc, l);
|
|
|
|
LLVMOrcRegisterJITEventListener(llvm_opt3_orc, l);
|
|
|
|
}
|
2018-03-22 19:07:55 +01:00
|
|
|
#endif
|
2018-07-23 06:13:20 +02:00
|
|
|
#if defined(HAVE_DECL_LLVMCREATEPERFJITEVENTLISTENER) && HAVE_DECL_LLVMCREATEPERFJITEVENTLISTENER
|
2020-11-10 05:01:33 +01:00
|
|
|
if (jit_profiling_support)
|
|
|
|
{
|
|
|
|
LLVMJITEventListenerRef l = LLVMCreatePerfJITEventListener();
|
2018-07-23 06:13:20 +02:00
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMOrcRegisterJITEventListener(llvm_opt0_orc, l);
|
|
|
|
LLVMOrcRegisterJITEventListener(llvm_opt3_orc, l);
|
|
|
|
}
|
2018-03-22 19:07:55 +01:00
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
}
|
|
|
|
#endif /* LLVM_VERSION_MAJOR > 11 */
|
2018-03-22 19:07:55 +01:00
|
|
|
|
2020-08-06 20:13:03 +02:00
|
|
|
on_proc_exit(llvm_shutdown, 0);
|
2018-03-22 03:28:28 +01:00
|
|
|
|
|
|
|
llvm_session_initialized = true;
|
|
|
|
|
|
|
|
MemoryContextSwitchTo(oldcontext);
|
|
|
|
}
|
|
|
|
|
|
|
|
static void
|
|
|
|
llvm_shutdown(int code, Datum arg)
|
|
|
|
{
|
2021-09-14 03:07:19 +02:00
|
|
|
/*
|
|
|
|
* If llvm_shutdown() is reached while in a fatal-on-oom section an error
|
|
|
|
* has occurred in the middle of LLVM code. It is not safe to call back
|
|
|
|
* into LLVM (which is why a FATAL error was thrown).
|
|
|
|
*
|
|
|
|
* We do need to shutdown LLVM in other shutdown cases, otherwise e.g.
|
|
|
|
* profiling data won't be written out.
|
|
|
|
*/
|
|
|
|
if (llvm_in_fatal_on_oom())
|
|
|
|
{
|
|
|
|
Assert(proc_exit_inprogress);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
if (llvm_jit_context_in_use_count != 0)
|
|
|
|
elog(PANIC, "LLVMJitContext in use count not 0 at exit (is %zu)",
|
|
|
|
llvm_jit_context_in_use_count);
|
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
{
|
|
|
|
if (llvm_opt3_orc)
|
|
|
|
{
|
|
|
|
LLVMOrcDisposeLLJIT(llvm_opt3_orc);
|
|
|
|
llvm_opt3_orc = NULL;
|
|
|
|
}
|
|
|
|
if (llvm_opt0_orc)
|
|
|
|
{
|
|
|
|
LLVMOrcDisposeLLJIT(llvm_opt0_orc);
|
|
|
|
llvm_opt0_orc = NULL;
|
|
|
|
}
|
|
|
|
if (llvm_ts_context)
|
|
|
|
{
|
|
|
|
LLVMOrcDisposeThreadSafeContext(llvm_ts_context);
|
|
|
|
llvm_ts_context = NULL;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
#else /* LLVM_VERSION_MAJOR > 11 */
|
2018-03-22 19:07:55 +01:00
|
|
|
{
|
2020-11-10 05:01:33 +01:00
|
|
|
/* unregister profiling support, needs to be flushed to be useful */
|
|
|
|
|
|
|
|
if (llvm_opt3_orc)
|
|
|
|
{
|
2018-03-22 19:07:55 +01:00
|
|
|
#if defined(HAVE_DECL_LLVMORCREGISTERPERF) && HAVE_DECL_LLVMORCREGISTERPERF
|
2020-11-10 05:01:33 +01:00
|
|
|
if (jit_profiling_support)
|
|
|
|
LLVMOrcUnregisterPerf(llvm_opt3_orc);
|
2018-03-22 19:07:55 +01:00
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMOrcDisposeInstance(llvm_opt3_orc);
|
|
|
|
llvm_opt3_orc = NULL;
|
|
|
|
}
|
2018-03-22 19:07:55 +01:00
|
|
|
|
2020-11-10 05:01:33 +01:00
|
|
|
if (llvm_opt0_orc)
|
|
|
|
{
|
2018-03-22 19:07:55 +01:00
|
|
|
#if defined(HAVE_DECL_LLVMORCREGISTERPERF) && HAVE_DECL_LLVMORCREGISTERPERF
|
2020-11-10 05:01:33 +01:00
|
|
|
if (jit_profiling_support)
|
|
|
|
LLVMOrcUnregisterPerf(llvm_opt0_orc);
|
2018-03-22 19:07:55 +01:00
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMOrcDisposeInstance(llvm_opt0_orc);
|
|
|
|
llvm_opt0_orc = NULL;
|
|
|
|
}
|
2018-03-22 19:07:55 +01:00
|
|
|
}
|
2020-11-10 05:01:33 +01:00
|
|
|
#endif /* LLVM_VERSION_MAJOR > 11 */
|
2018-03-22 03:28:28 +01:00
|
|
|
}
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
|
2018-03-23 06:15:51 +01:00
|
|
|
/* helper for llvm_create_types, returning a function's return type */
|
|
|
|
static LLVMTypeRef
|
|
|
|
load_return_type(LLVMModuleRef mod, const char *name)
|
|
|
|
{
|
|
|
|
LLVMValueRef value;
|
|
|
|
LLVMTypeRef typ;
|
|
|
|
|
|
|
|
/* this'll return a *pointer* to the function */
|
|
|
|
value = LLVMGetNamedFunction(mod, name);
|
|
|
|
if (!value)
|
|
|
|
elog(ERROR, "function %s is unknown", name);
|
|
|
|
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
typ = LLVMGetFunctionReturnType(value); /* in llvmjit_wrap.cpp */
|
2018-03-23 06:15:51 +01:00
|
|
|
|
|
|
|
return typ;
|
|
|
|
}
|
|
|
|
|
2023-09-27 13:02:21 +02:00
|
|
|
/*
|
|
|
|
* Load triple & layout from clang emitted file so we're guaranteed to be
|
|
|
|
* compatible.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_set_target(void)
|
|
|
|
{
|
|
|
|
if (!llvm_types_module)
|
|
|
|
elog(ERROR, "failed to extract target information, llvmjit_types.c not loaded");
|
|
|
|
|
|
|
|
if (llvm_triple == NULL)
|
|
|
|
llvm_triple = pstrdup(LLVMGetTarget(llvm_types_module));
|
|
|
|
|
|
|
|
if (llvm_layout == NULL)
|
|
|
|
llvm_layout = pstrdup(LLVMGetDataLayoutStr(llvm_types_module));
|
|
|
|
}
|
|
|
|
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
/*
|
|
|
|
* Load required information, types, function signatures from llvmjit_types.c
|
|
|
|
* and make them available in global variables.
|
|
|
|
*
|
|
|
|
* Those global variables are then used while emitting code.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_create_types(void)
|
|
|
|
{
|
|
|
|
char path[MAXPGPATH];
|
|
|
|
LLVMMemoryBufferRef buf;
|
|
|
|
char *msg;
|
|
|
|
|
|
|
|
snprintf(path, MAXPGPATH, "%s/%s", pkglib_path, "llvmjit_types.bc");
|
|
|
|
|
|
|
|
/* open file */
|
|
|
|
if (LLVMCreateMemoryBufferWithContentsOfFile(path, &buf, &msg))
|
|
|
|
{
|
|
|
|
elog(ERROR, "LLVMCreateMemoryBufferWithContentsOfFile(%s) failed: %s",
|
|
|
|
path, msg);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* eagerly load contents, going to need it all */
|
2023-09-27 13:02:21 +02:00
|
|
|
if (LLVMParseBitcodeInContext2(llvm_context, buf, &llvm_types_module))
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
{
|
2023-09-27 13:02:21 +02:00
|
|
|
elog(ERROR, "LLVMParseBitcodeInContext2 of %s failed", path);
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
|
|
|
LLVMDisposeMemoryBuffer(buf);
|
|
|
|
|
2020-12-07 22:16:55 +01:00
|
|
|
TypeSizeT = llvm_pg_var_type("TypeSizeT");
|
2020-02-07 07:13:52 +01:00
|
|
|
TypeParamBool = load_return_type(llvm_types_module, "FunctionReturningBool");
|
2020-12-07 22:16:55 +01:00
|
|
|
TypeStorageBool = llvm_pg_var_type("TypeStorageBool");
|
|
|
|
TypePGFunction = llvm_pg_var_type("TypePGFunction");
|
|
|
|
StructNullableDatum = llvm_pg_var_type("StructNullableDatum");
|
|
|
|
StructExprContext = llvm_pg_var_type("StructExprContext");
|
|
|
|
StructExprEvalStep = llvm_pg_var_type("StructExprEvalStep");
|
|
|
|
StructExprState = llvm_pg_var_type("StructExprState");
|
|
|
|
StructFunctionCallInfoData = llvm_pg_var_type("StructFunctionCallInfoData");
|
|
|
|
StructMemoryContextData = llvm_pg_var_type("StructMemoryContextData");
|
|
|
|
StructTupleTableSlot = llvm_pg_var_type("StructTupleTableSlot");
|
|
|
|
StructHeapTupleTableSlot = llvm_pg_var_type("StructHeapTupleTableSlot");
|
|
|
|
StructMinimalTupleTableSlot = llvm_pg_var_type("StructMinimalTupleTableSlot");
|
|
|
|
StructHeapTupleData = llvm_pg_var_type("StructHeapTupleData");
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
StructHeapTupleHeaderData = llvm_pg_var_type("StructHeapTupleHeaderData");
|
2020-12-07 22:16:55 +01:00
|
|
|
StructTupleDescData = llvm_pg_var_type("StructTupleDescData");
|
|
|
|
StructAggState = llvm_pg_var_type("StructAggState");
|
|
|
|
StructAggStatePerGroupData = llvm_pg_var_type("StructAggStatePerGroupData");
|
|
|
|
StructAggStatePerTransData = llvm_pg_var_type("StructAggStatePerTransData");
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
StructPlanState = llvm_pg_var_type("StructPlanState");
|
|
|
|
StructMinimalTupleData = llvm_pg_var_type("StructMinimalTupleData");
|
2020-02-07 07:13:52 +01:00
|
|
|
|
|
|
|
AttributeTemplate = LLVMGetNamedFunction(llvm_types_module, "AttributeTemplate");
|
jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].
* For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
* For LLVM == 15, we'll continue to do the same, explicitly opting
out of opaque pointer mode.
* For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
the extra type argument.
The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc. The change is mostly mechanical,
except that at each site the correct type had to be provided.
In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.
Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.
[1] https://llvm.org/docs/OpaquePointers.html
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 11:09:05 +02:00
|
|
|
ExecEvalSubroutineTemplate = LLVMGetNamedFunction(llvm_types_module, "ExecEvalSubroutineTemplate");
|
|
|
|
ExecEvalBoolSubroutineTemplate = LLVMGetNamedFunction(llvm_types_module, "ExecEvalBoolSubroutineTemplate");
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Split a symbol into module / function parts. If the function is in the
|
|
|
|
* main binary (or an external library) *modname will be NULL.
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
llvm_split_symbol_name(const char *name, char **modname, char **funcname)
|
|
|
|
{
|
|
|
|
*modname = NULL;
|
|
|
|
*funcname = NULL;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Module function names are pgextern.$module.$funcname
|
|
|
|
*/
|
|
|
|
if (strncmp(name, "pgextern.", strlen("pgextern.")) == 0)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Symbol names cannot contain a ., therefore we can split based on
|
2018-11-02 13:56:16 +01:00
|
|
|
* first and last occurrence of one.
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
*/
|
|
|
|
*funcname = rindex(name, '.');
|
|
|
|
(*funcname)++; /* jump over . */
|
|
|
|
|
|
|
|
*modname = pnstrdup(name + strlen("pgextern."),
|
|
|
|
*funcname - name - strlen("pgextern.") - 1);
|
|
|
|
Assert(funcname);
|
|
|
|
|
|
|
|
*funcname = pstrdup(*funcname);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
*modname = NULL;
|
|
|
|
*funcname = pstrdup(name);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Attempt to resolve symbol, so LLVM can emit a reference to it.
|
|
|
|
*/
|
|
|
|
static uint64_t
|
|
|
|
llvm_resolve_symbol(const char *symname, void *ctx)
|
|
|
|
{
|
|
|
|
uintptr_t addr;
|
|
|
|
char *funcname;
|
|
|
|
char *modname;
|
|
|
|
|
|
|
|
/*
|
2018-03-26 21:58:17 +02:00
|
|
|
* macOS prefixes all object level symbols with an underscore. But neither
|
Support for optimizing and emitting code in LLVM JIT provider.
This commit introduces the ability to actually generate code using
LLVM. In particular, this adds:
- Ability to emit code both in heavily optimized and largely
unoptimized fashion
- Batching facility to allow functions to be defined in small
increments, but optimized and emitted in executable form in larger
batches (for performance and memory efficiency)
- Type and function declaration synchronization between runtime
generated code and normal postgres code. This is critical to be able
to access struct fields etc.
- Developer oriented jit_dump_bitcode GUC, for inspecting / debugging
the generated code.
- per JitContext statistics of number of functions, time spent
generating code, optimizing, and emitting it. This will later be
employed for EXPLAIN support.
This commit doesn't yet contain any code actually generating
functions. That'll follow in later commits.
Documentation for GUCs added, and for JIT in general, will be added in
later commits.
Author: Andres Freund, with contributions by Pierre Ducroquet
Testing-By: Thomas Munro, Peter Eisentraut
Discussion: https://postgr.es/m/20170901064131.tazjxwus3k2w3ybh@alap3.anarazel.de
2018-03-22 19:05:22 +01:00
|
|
|
* dlsym() nor PG's inliner expect that. So undo.
|
|
|
|
*/
|
|
|
|
#if defined(__darwin__)
|
|
|
|
if (symname[0] != '_')
|
|
|
|
elog(ERROR, "expected prefixed symbol name, but got \"%s\"", symname);
|
|
|
|
symname++;
|
|
|
|
#endif
|
|
|
|
|
|
|
|
llvm_split_symbol_name(symname, &modname, &funcname);
|
|
|
|
|
|
|
|
/* functions that aren't resolved to names shouldn't ever get here */
|
|
|
|
Assert(funcname);
|
|
|
|
|
|
|
|
if (modname)
|
|
|
|
addr = (uintptr_t) load_external_function(modname, funcname,
|
|
|
|
true, NULL);
|
|
|
|
else
|
|
|
|
addr = (uintptr_t) LLVMSearchForAddressOfSymbol(symname);
|
|
|
|
|
|
|
|
pfree(funcname);
|
|
|
|
if (modname)
|
|
|
|
pfree(modname);
|
|
|
|
|
|
|
|
/* let LLVM will error out - should never happen */
|
|
|
|
if (!addr)
|
|
|
|
elog(WARNING, "failed to resolve name %s", symname);
|
|
|
|
|
|
|
|
return (uint64_t) addr;
|
|
|
|
}
|
2020-11-10 05:01:33 +01:00
|
|
|
|
|
|
|
#if LLVM_VERSION_MAJOR > 11
|
|
|
|
|
|
|
|
static LLVMErrorRef
|
|
|
|
llvm_resolve_symbols(LLVMOrcDefinitionGeneratorRef GeneratorObj, void *Ctx,
|
|
|
|
LLVMOrcLookupStateRef *LookupState, LLVMOrcLookupKind Kind,
|
|
|
|
LLVMOrcJITDylibRef JD, LLVMOrcJITDylibLookupFlags JDLookupFlags,
|
|
|
|
LLVMOrcCLookupSet LookupSet, size_t LookupSetSize)
|
|
|
|
{
|
2022-10-19 11:18:26 +02:00
|
|
|
#if LLVM_VERSION_MAJOR > 14
|
|
|
|
LLVMOrcCSymbolMapPairs symbols = palloc0(sizeof(LLVMOrcCSymbolMapPair) * LookupSetSize);
|
|
|
|
#else
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMOrcCSymbolMapPairs symbols = palloc0(sizeof(LLVMJITCSymbolMapPair) * LookupSetSize);
|
2022-10-19 11:18:26 +02:00
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMErrorRef error;
|
|
|
|
LLVMOrcMaterializationUnitRef mu;
|
|
|
|
|
|
|
|
for (int i = 0; i < LookupSetSize; i++)
|
|
|
|
{
|
|
|
|
const char *name = LLVMOrcSymbolStringPoolEntryStr(LookupSet[i].Name);
|
|
|
|
|
2021-06-24 23:55:26 +02:00
|
|
|
#if LLVM_VERSION_MAJOR > 12
|
|
|
|
LLVMOrcRetainSymbolStringPoolEntry(LookupSet[i].Name);
|
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
symbols[i].Name = LookupSet[i].Name;
|
|
|
|
symbols[i].Sym.Address = llvm_resolve_symbol(name, NULL);
|
|
|
|
symbols[i].Sym.Flags.GenericFlags = LLVMJITSymbolGenericFlagsExported;
|
|
|
|
}
|
|
|
|
|
|
|
|
mu = LLVMOrcAbsoluteSymbols(symbols, LookupSetSize);
|
|
|
|
error = LLVMOrcJITDylibDefine(JD, mu);
|
|
|
|
if (error != LLVMErrorSuccess)
|
|
|
|
LLVMOrcDisposeMaterializationUnit(mu);
|
|
|
|
|
|
|
|
pfree(symbols);
|
|
|
|
|
|
|
|
return error;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We cannot throw errors through LLVM (without causing a FATAL at least), so
|
|
|
|
* just use WARNING here. That's OK anyway, as the error is also reported at
|
|
|
|
* the top level action (with less detail) and there might be multiple
|
|
|
|
* invocations of errors with details.
|
|
|
|
*
|
|
|
|
* This doesn't really happen during normal operation, but in cases like
|
|
|
|
* symbol resolution breakage. So just using elog(WARNING) is fine.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
llvm_log_jit_error(void *ctx, LLVMErrorRef error)
|
|
|
|
{
|
|
|
|
elog(WARNING, "error during JITing: %s",
|
|
|
|
llvm_error_message(error));
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Create our own object layer, so we can add event listeners.
|
|
|
|
*/
|
|
|
|
static LLVMOrcObjectLayerRef
|
|
|
|
llvm_create_object_layer(void *Ctx, LLVMOrcExecutionSessionRef ES, const char *Triple)
|
|
|
|
{
|
|
|
|
LLVMOrcObjectLayerRef objlayer =
|
|
|
|
LLVMOrcCreateRTDyldObjectLinkingLayerWithSectionMemoryManager(ES);
|
|
|
|
|
|
|
|
#if defined(HAVE_DECL_LLVMCREATEGDBREGISTRATIONLISTENER) && HAVE_DECL_LLVMCREATEGDBREGISTRATIONLISTENER
|
|
|
|
if (jit_debugging_support)
|
|
|
|
{
|
|
|
|
LLVMJITEventListenerRef l = LLVMCreateGDBRegistrationListener();
|
|
|
|
|
|
|
|
LLVMOrcRTDyldObjectLinkingLayerRegisterJITEventListener(objlayer, l);
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
|
|
|
#if defined(HAVE_DECL_LLVMCREATEPERFJITEVENTLISTENER) && HAVE_DECL_LLVMCREATEPERFJITEVENTLISTENER
|
|
|
|
if (jit_profiling_support)
|
|
|
|
{
|
|
|
|
LLVMJITEventListenerRef l = LLVMCreatePerfJITEventListener();
|
|
|
|
|
|
|
|
LLVMOrcRTDyldObjectLinkingLayerRegisterJITEventListener(objlayer, l);
|
|
|
|
}
|
|
|
|
#endif
|
|
|
|
|
|
|
|
return objlayer;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Create LLJIT instance, using the passed in target machine. Note that the
|
|
|
|
* target machine afterwards is owned by the LLJIT instance.
|
|
|
|
*/
|
|
|
|
static LLVMOrcLLJITRef
|
|
|
|
llvm_create_jit_instance(LLVMTargetMachineRef tm)
|
|
|
|
{
|
|
|
|
LLVMOrcLLJITRef lljit;
|
|
|
|
LLVMOrcJITTargetMachineBuilderRef tm_builder;
|
|
|
|
LLVMOrcLLJITBuilderRef lljit_builder;
|
|
|
|
LLVMErrorRef error;
|
|
|
|
LLVMOrcDefinitionGeneratorRef main_gen;
|
|
|
|
LLVMOrcDefinitionGeneratorRef ref_gen;
|
|
|
|
|
|
|
|
lljit_builder = LLVMOrcCreateLLJITBuilder();
|
|
|
|
tm_builder = LLVMOrcJITTargetMachineBuilderCreateFromTargetMachine(tm);
|
|
|
|
LLVMOrcLLJITBuilderSetJITTargetMachineBuilder(lljit_builder, tm_builder);
|
|
|
|
|
|
|
|
LLVMOrcLLJITBuilderSetObjectLinkingLayerCreator(lljit_builder,
|
|
|
|
llvm_create_object_layer,
|
|
|
|
NULL);
|
|
|
|
|
|
|
|
error = LLVMOrcCreateLLJIT(&lljit, lljit_builder);
|
|
|
|
if (error)
|
|
|
|
elog(ERROR, "failed to create lljit instance: %s",
|
|
|
|
llvm_error_message(error));
|
|
|
|
|
|
|
|
LLVMOrcExecutionSessionSetErrorReporter(LLVMOrcLLJITGetExecutionSession(lljit),
|
|
|
|
llvm_log_jit_error, NULL);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Symbol resolution support for symbols in the postgres binary /
|
|
|
|
* libraries already loaded.
|
|
|
|
*/
|
|
|
|
error = LLVMOrcCreateDynamicLibrarySearchGeneratorForProcess(&main_gen,
|
|
|
|
LLVMOrcLLJITGetGlobalPrefix(lljit),
|
|
|
|
0, NULL);
|
|
|
|
if (error)
|
|
|
|
elog(ERROR, "failed to create generator: %s",
|
|
|
|
llvm_error_message(error));
|
|
|
|
LLVMOrcJITDylibAddGenerator(LLVMOrcLLJITGetMainJITDylib(lljit), main_gen);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Symbol resolution support for "special" functions, e.g. a call into an
|
|
|
|
* SQL callable function.
|
|
|
|
*/
|
2022-10-19 11:18:26 +02:00
|
|
|
#if LLVM_VERSION_MAJOR > 14
|
|
|
|
ref_gen = LLVMOrcCreateCustomCAPIDefinitionGenerator(llvm_resolve_symbols, NULL, NULL);
|
|
|
|
#else
|
2020-11-10 05:01:33 +01:00
|
|
|
ref_gen = LLVMOrcCreateCustomCAPIDefinitionGenerator(llvm_resolve_symbols, NULL);
|
2022-10-19 11:18:26 +02:00
|
|
|
#endif
|
2020-11-10 05:01:33 +01:00
|
|
|
LLVMOrcJITDylibAddGenerator(LLVMOrcLLJITGetMainJITDylib(lljit), ref_gen);
|
|
|
|
|
|
|
|
return lljit;
|
|
|
|
}
|
|
|
|
|
|
|
|
static char *
|
|
|
|
llvm_error_message(LLVMErrorRef error)
|
|
|
|
{
|
|
|
|
char *orig = LLVMGetErrorMessage(error);
|
|
|
|
char *msg = pstrdup(orig);
|
|
|
|
|
|
|
|
LLVMDisposeErrorMessage(orig);
|
|
|
|
|
|
|
|
return msg;
|
|
|
|
}
|
|
|
|
|
|
|
|
#endif /* LLVM_VERSION_MAJOR > 11 */
|
Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.
The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.
All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.
The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.
Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget. There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.
Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.
There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.
Add tests in src/test/modules/test_resowner.
Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 12:30:50 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* ResourceOwner callbacks
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
ResOwnerReleaseJitContext(Datum res)
|
|
|
|
{
|
|
|
|
JitContext *context = (JitContext *) DatumGetPointer(res);
|
|
|
|
|
|
|
|
context->resowner = NULL;
|
|
|
|
jit_release_context(context);
|
|
|
|
}
|