Fix failure to detoast fields in composite elements of structured types.
If we have an array of records stored on disk, the individual record fields cannot contain out-of-line TOAST pointers: the tuptoaster.c mechanisms are only prepared to deal with TOAST pointers appearing in top-level fields of a stored row. The same applies for ranges over composite types, nested composites, etc. However, the existing code only took care of expanding sub-field TOAST pointers for the case of nested composites, not for other structured types containing composites. For example, given a command such as UPDATE tab SET arraycol = ARRAY[(ROW(x,42)::mycompositetype] ... where x is a direct reference to a field of an on-disk tuple, if that field is long enough to be toasted out-of-line then the TOAST pointer would be inserted as-is into the array column. If the source record for x is later deleted, the array field value would become a dangling pointer, leading to errors along the line of "missing chunk number 0 for toast value ..." when the value is referenced. A reproducible test case for this was provided by Jan Pecek, but it seems likely that some of the "missing chunk number" reports we've heard in the past were caused by similar issues. Code-wise, the problem is that PG_DETOAST_DATUM() is not adequate to produce a self-contained Datum value if the Datum is of composite type. Seen in this light, the problem is not just confined to arrays and ranges, but could also affect some other places where detoasting is done in that way, for example form_index_tuple(). I tried teaching the array code to apply toast_flatten_tuple_attribute() along with PG_DETOAST_DATUM() when the array element type is composite, but this was messy and imposed extra cache lookup costs whether or not any TOAST pointers were present, indeed sometimes when the array element type isn't even composite (since sometimes it takes a typcache lookup to find that out). The idea of extending that approach to all the places that currently use PG_DETOAST_DATUM() wasn't attractive at all. This patch instead solves the problem by decreeing that composite Datum values must not contain any out-of-line TOAST pointers in the first place; that is, we expand out-of-line fields at the point of constructing a composite Datum, not at the point where we're about to insert it into a larger tuple. This rule is applied only to true composite Datums, not to tuples that are being passed around the system as tuples, so it's not as invasive as it might sound at first. With this approach, the amount of code that has to be touched for a full solution is greatly reduced, and added cache lookup costs are avoided except when there actually is a TOAST pointer that needs to be inlined. The main drawback of this approach is that we might sometimes dereference a TOAST pointer that will never actually be used by the query, imposing a rather large cost that wasn't there before. On the other side of the coin, if the field value is used multiple times then we'll come out ahead by avoiding repeat detoastings. Experimentation suggests that common SQL coding patterns are unaffected either way, though. Applications that are very negatively affected could be advised to modify their code to not fetch columns they won't be using. In future, we might consider reverting this solution in favor of detoasting only at the point where data is about to be stored to disk, using some method that can drill down into multiple levels of nested structured types. That will require defining new APIs for structured types, though, so it doesn't seem feasible as a back-patchable fix. Note that this patch changes HeapTupleGetDatum() from a macro to a function call; this means that any third-party code using that macro will not get protection against creating TOAST-pointer-containing Datums until it's recompiled. The same applies to any uses of PG_RETURN_HEAPTUPLEHEADER(). It seems likely that this is not a big problem in practice: most of the tuple-returning functions in core and contrib produce outputs that could not possibly be toasted anyway, and the same probably holds for third-party extensions. This bug has existed since TOAST was invented, so back-patch to all supported branches.
This commit is contained in:
Tom Lane
parent
65fb5ff130
commit
3f8c8e3c61
|
|
@ -617,6 +617,41 @@ heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest)
|
|||
memcpy((char *) dest->t_data, (char *) src->t_data, src->t_len);
|
||||
}
|
||||
|
||||
/* ----------------
|
||||
* heap_copy_tuple_as_datum
|
||||
*
|
||||
* copy a tuple as a composite-type Datum
|
||||
* ----------------
|
||||
*/
|
||||
Datum
|
||||
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
|
||||
{
|
||||
HeapTupleHeader td;
|
||||
|
||||
/*
|
||||
* If the tuple contains any external TOAST pointers, we have to inline
|
||||
* those fields to meet the conventions for composite-type Datums.
|
||||
*/
|
||||
if (HeapTupleHasExternal(tuple))
|
||||
return toast_flatten_tuple_to_datum(tuple->t_data,
|
||||
tuple->t_len,
|
||||
tupleDesc);
|
||||
|
||||
/*
|
||||
* Fast path for easy case: just make a palloc'd copy and insert the
|
||||
* correct composite-Datum header fields (since those may not be set if
|
||||
* the given tuple came from disk, rather than from heap_form_tuple).
|
||||
*/
|
||||
td = (HeapTupleHeader) palloc(tuple->t_len);
|
||||
memcpy((char *) td, (char *) tuple->t_data, tuple->t_len);
|
||||
|
||||
HeapTupleHeaderSetDatumLength(td, tuple->t_len);
|
||||
HeapTupleHeaderSetTypeId(td, tupleDesc->tdtypeid);
|
||||
HeapTupleHeaderSetTypMod(td, tupleDesc->tdtypmod);
|
||||
|
||||
return PointerGetDatum(td);
|
||||
}
|
||||
|
||||
/*
|
||||
* heap_form_tuple
|
||||
* construct a tuple from the given values[] and isnull[] arrays,
|
||||
|
|
@ -635,7 +670,6 @@ heap_form_tuple(TupleDesc tupleDescriptor,
|
|||
data_len;
|
||||
int hoff;
|
||||
bool hasnull = false;
|
||||
Form_pg_attribute *att = tupleDescriptor->attrs;
|
||||
int numberOfAttributes = tupleDescriptor->natts;
|
||||
int i;
|
||||
|
||||
|
|
@ -646,28 +680,14 @@ heap_form_tuple(TupleDesc tupleDescriptor,
|
|||
numberOfAttributes, MaxTupleAttributeNumber)));
|
||||
|
||||
/*
|
||||
* Check for nulls and embedded tuples; expand any toasted attributes in
|
||||
* embedded tuples. This preserves the invariant that toasting can only
|
||||
* go one level deep.
|
||||
*
|
||||
* We can skip calling toast_flatten_tuple_attribute() if the attribute
|
||||
* couldn't possibly be of composite type. All composite datums are
|
||||
* varlena and have alignment 'd'; furthermore they aren't arrays. Also,
|
||||
* if an attribute is already toasted, it must have been sent to disk
|
||||
* already and so cannot contain toasted attributes.
|
||||
* Check for nulls
|
||||
*/
|
||||
for (i = 0; i < numberOfAttributes; i++)
|
||||
{
|
||||
if (isnull[i])
|
||||
hasnull = true;
|
||||
else if (att[i]->attlen == -1 &&
|
||||
att[i]->attalign == 'd' &&
|
||||
att[i]->attndims == 0 &&
|
||||
!VARATT_IS_EXTENDED(DatumGetPointer(values[i])))
|
||||
{
|
||||
values[i] = toast_flatten_tuple_attribute(values[i],
|
||||
att[i]->atttypid,
|
||||
att[i]->atttypmod);
|
||||
hasnull = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -697,7 +717,8 @@ heap_form_tuple(TupleDesc tupleDescriptor,
|
|||
|
||||
/*
|
||||
* And fill in the information. Note we fill the Datum fields even though
|
||||
* this tuple may never become a Datum.
|
||||
* this tuple may never become a Datum. This lets HeapTupleHeaderGetDatum
|
||||
* identify the tuple type if needed.
|
||||
*/
|
||||
tuple->t_len = len;
|
||||
ItemPointerSetInvalid(&(tuple->t_self));
|
||||
|
|
@ -1389,7 +1410,6 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
|
|||
data_len;
|
||||
int hoff;
|
||||
bool hasnull = false;
|
||||
Form_pg_attribute *att = tupleDescriptor->attrs;
|
||||
int numberOfAttributes = tupleDescriptor->natts;
|
||||
int i;
|
||||
|
||||
|
|
@ -1400,28 +1420,14 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
|
|||
numberOfAttributes, MaxTupleAttributeNumber)));
|
||||
|
||||
/*
|
||||
* Check for nulls and embedded tuples; expand any toasted attributes in
|
||||
* embedded tuples. This preserves the invariant that toasting can only
|
||||
* go one level deep.
|
||||
*
|
||||
* We can skip calling toast_flatten_tuple_attribute() if the attribute
|
||||
* couldn't possibly be of composite type. All composite datums are
|
||||
* varlena and have alignment 'd'; furthermore they aren't arrays. Also,
|
||||
* if an attribute is already toasted, it must have been sent to disk
|
||||
* already and so cannot contain toasted attributes.
|
||||
* Check for nulls
|
||||
*/
|
||||
for (i = 0; i < numberOfAttributes; i++)
|
||||
{
|
||||
if (isnull[i])
|
||||
hasnull = true;
|
||||
else if (att[i]->attlen == -1 &&
|
||||
att[i]->attalign == 'd' &&
|
||||
att[i]->attndims == 0 &&
|
||||
!VARATT_IS_EXTENDED(values[i]))
|
||||
{
|
||||
values[i] = toast_flatten_tuple_attribute(values[i],
|
||||
att[i]->atttypid,
|
||||
att[i]->atttypmod);
|
||||
hasnull = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -158,6 +158,11 @@ index_form_tuple(TupleDesc tupleDescriptor,
|
|||
if (tupmask & HEAP_HASVARWIDTH)
|
||||
infomask |= INDEX_VAR_MASK;
|
||||
|
||||
/* Also assert we got rid of external attributes */
|
||||
#ifdef TOAST_INDEX_HACK
|
||||
Assert((tupmask & HEAP_HASEXTERNAL) == 0);
|
||||
#endif
|
||||
|
||||
/*
|
||||
* Here we make sure that the size will fit in the field reserved for it
|
||||
* in t_info.
|
||||
|
|
|
|||
|
|
@ -991,6 +991,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
|
|||
*
|
||||
* "Flatten" a tuple to contain no out-of-line toasted fields.
|
||||
* (This does not eliminate compressed or short-header datums.)
|
||||
*
|
||||
* Note: we expect the caller already checked HeapTupleHasExternal(tup),
|
||||
* so there is no need for a short-circuit path.
|
||||
* ----------
|
||||
*/
|
||||
HeapTuple
|
||||
|
|
@ -1068,59 +1071,61 @@ toast_flatten_tuple(HeapTuple tup, TupleDesc tupleDesc)
|
|||
|
||||
|
||||
/* ----------
|
||||
* toast_flatten_tuple_attribute -
|
||||
* toast_flatten_tuple_to_datum -
|
||||
*
|
||||
* If a Datum is of composite type, "flatten" it to contain no toasted fields.
|
||||
* This must be invoked on any potentially-composite field that is to be
|
||||
* inserted into a tuple. Doing this preserves the invariant that toasting
|
||||
* goes only one level deep in a tuple.
|
||||
* "Flatten" a tuple containing out-of-line toasted fields into a Datum.
|
||||
* The result is always palloc'd in the current memory context.
|
||||
*
|
||||
* Note that flattening does not mean expansion of short-header varlenas,
|
||||
* so in one sense toasting is allowed within composite datums.
|
||||
* We have a general rule that Datums of container types (rows, arrays,
|
||||
* ranges, etc) must not contain any external TOAST pointers. Without
|
||||
* this rule, we'd have to look inside each Datum when preparing a tuple
|
||||
* for storage, which would be expensive and would fail to extend cleanly
|
||||
* to new sorts of container types.
|
||||
*
|
||||
* However, we don't want to say that tuples represented as HeapTuples
|
||||
* can't contain toasted fields, so instead this routine should be called
|
||||
* when such a HeapTuple is being converted into a Datum.
|
||||
*
|
||||
* While we're at it, we decompress any compressed fields too. This is not
|
||||
* necessary for correctness, but reflects an expectation that compression
|
||||
* will be more effective if applied to the whole tuple not individual
|
||||
* fields. We are not so concerned about that that we want to deconstruct
|
||||
* and reconstruct tuples just to get rid of compressed fields, however.
|
||||
* So callers typically won't call this unless they see that the tuple has
|
||||
* at least one external field.
|
||||
*
|
||||
* On the other hand, in-line short-header varlena fields are left alone.
|
||||
* If we "untoasted" them here, they'd just get changed back to short-header
|
||||
* format anyway within heap_fill_tuple.
|
||||
* ----------
|
||||
*/
|
||||
Datum
|
||||
toast_flatten_tuple_attribute(Datum value,
|
||||
Oid typeId, int32 typeMod)
|
||||
toast_flatten_tuple_to_datum(HeapTupleHeader tup,
|
||||
uint32 tup_len,
|
||||
TupleDesc tupleDesc)
|
||||
{
|
||||
TupleDesc tupleDesc;
|
||||
HeapTupleHeader olddata;
|
||||
HeapTupleHeader new_data;
|
||||
int32 new_header_len;
|
||||
int32 new_data_len;
|
||||
int32 new_tuple_len;
|
||||
HeapTupleData tmptup;
|
||||
Form_pg_attribute *att;
|
||||
int numAttrs;
|
||||
Form_pg_attribute *att = tupleDesc->attrs;
|
||||
int numAttrs = tupleDesc->natts;
|
||||
int i;
|
||||
bool need_change = false;
|
||||
bool has_nulls = false;
|
||||
Datum toast_values[MaxTupleAttributeNumber];
|
||||
bool toast_isnull[MaxTupleAttributeNumber];
|
||||
bool toast_free[MaxTupleAttributeNumber];
|
||||
|
||||
/*
|
||||
* See if it's a composite type, and get the tupdesc if so.
|
||||
*/
|
||||
tupleDesc = lookup_rowtype_tupdesc_noerror(typeId, typeMod, true);
|
||||
if (tupleDesc == NULL)
|
||||
return value; /* not a composite type */
|
||||
|
||||
att = tupleDesc->attrs;
|
||||
numAttrs = tupleDesc->natts;
|
||||
/* Build a temporary HeapTuple control structure */
|
||||
tmptup.t_len = tup_len;
|
||||
ItemPointerSetInvalid(&(tmptup.t_self));
|
||||
tmptup.t_tableOid = InvalidOid;
|
||||
tmptup.t_data = tup;
|
||||
|
||||
/*
|
||||
* Break down the tuple into fields.
|
||||
*/
|
||||
olddata = DatumGetHeapTupleHeader(value);
|
||||
Assert(typeId == HeapTupleHeaderGetTypeId(olddata));
|
||||
Assert(typeMod == HeapTupleHeaderGetTypMod(olddata));
|
||||
/* Build a temporary HeapTuple control structure */
|
||||
tmptup.t_len = HeapTupleHeaderGetDatumLength(olddata);
|
||||
ItemPointerSetInvalid(&(tmptup.t_self));
|
||||
tmptup.t_tableOid = InvalidOid;
|
||||
tmptup.t_data = olddata;
|
||||
|
||||
Assert(numAttrs <= MaxTupleAttributeNumber);
|
||||
heap_deform_tuple(&tmptup, tupleDesc, toast_values, toast_isnull);
|
||||
|
||||
|
|
@ -1144,20 +1149,10 @@ toast_flatten_tuple_attribute(Datum value,
|
|||
new_value = heap_tuple_untoast_attr(new_value);
|
||||
toast_values[i] = PointerGetDatum(new_value);
|
||||
toast_free[i] = true;
|
||||
need_change = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* If nothing to untoast, just return the original tuple.
|
||||
*/
|
||||
if (!need_change)
|
||||
{
|
||||
ReleaseTupleDesc(tupleDesc);
|
||||
return value;
|
||||
}
|
||||
|
||||
/*
|
||||
* Calculate the new size of the tuple.
|
||||
*
|
||||
|
|
@ -1166,7 +1161,7 @@ toast_flatten_tuple_attribute(Datum value,
|
|||
new_header_len = offsetof(HeapTupleHeaderData, t_bits);
|
||||
if (has_nulls)
|
||||
new_header_len += BITMAPLEN(numAttrs);
|
||||
if (olddata->t_infomask & HEAP_HASOID)
|
||||
if (tup->t_infomask & HEAP_HASOID)
|
||||
new_header_len += sizeof(Oid);
|
||||
new_header_len = MAXALIGN(new_header_len);
|
||||
new_data_len = heap_compute_data_size(tupleDesc,
|
||||
|
|
@ -1178,14 +1173,16 @@ toast_flatten_tuple_attribute(Datum value,
|
|||
/*
|
||||
* Copy the existing tuple header, but adjust natts and t_hoff.
|
||||
*/
|
||||
memcpy(new_data, olddata, offsetof(HeapTupleHeaderData, t_bits));
|
||||
memcpy(new_data, tup, offsetof(HeapTupleHeaderData, t_bits));
|
||||
HeapTupleHeaderSetNatts(new_data, numAttrs);
|
||||
new_data->t_hoff = new_header_len;
|
||||
if (olddata->t_infomask & HEAP_HASOID)
|
||||
HeapTupleHeaderSetOid(new_data, HeapTupleHeaderGetOid(olddata));
|
||||
if (tup->t_infomask & HEAP_HASOID)
|
||||
HeapTupleHeaderSetOid(new_data, HeapTupleHeaderGetOid(tup));
|
||||
|
||||
/* Reset the datum length field, too */
|
||||
/* Set the composite-Datum header fields correctly */
|
||||
HeapTupleHeaderSetDatumLength(new_data, new_tuple_len);
|
||||
HeapTupleHeaderSetTypeId(new_data, tupleDesc->tdtypeid);
|
||||
HeapTupleHeaderSetTypMod(new_data, tupleDesc->tdtypmod);
|
||||
|
||||
/* Copy over the data, and fill the null bitmap if needed */
|
||||
heap_fill_tuple(tupleDesc,
|
||||
|
|
@ -1202,7 +1199,6 @@ toast_flatten_tuple_attribute(Datum value,
|
|||
for (i = 0; i < numAttrs; i++)
|
||||
if (toast_free[i])
|
||||
pfree(DatumGetPointer(toast_values[i]));
|
||||
ReleaseTupleDesc(tupleDesc);
|
||||
|
||||
return PointerGetDatum(new_data);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -896,8 +896,6 @@ ExecEvalWholeRowFast(WholeRowVarExprState *wrvstate, ExprContext *econtext,
|
|||
{
|
||||
Var *variable = (Var *) wrvstate->xprstate.expr;
|
||||
TupleTableSlot *slot;
|
||||
HeapTuple tuple;
|
||||
TupleDesc tupleDesc;
|
||||
HeapTupleHeader dtuple;
|
||||
|
||||
if (isDone)
|
||||
|
|
@ -927,32 +925,20 @@ ExecEvalWholeRowFast(WholeRowVarExprState *wrvstate, ExprContext *econtext,
|
|||
if (wrvstate->wrv_junkFilter != NULL)
|
||||
slot = ExecFilterJunk(wrvstate->wrv_junkFilter, slot);
|
||||
|
||||
tuple = ExecFetchSlotTuple(slot);
|
||||
tupleDesc = slot->tts_tupleDescriptor;
|
||||
|
||||
/*
|
||||
* We have to make a copy of the tuple so we can safely insert the Datum
|
||||
* overhead fields, which are not set in on-disk tuples.
|
||||
* Copy the slot tuple and make sure any toasted fields get detoasted.
|
||||
*/
|
||||
dtuple = (HeapTupleHeader) palloc(tuple->t_len);
|
||||
memcpy((char *) dtuple, (char *) tuple->t_data, tuple->t_len);
|
||||
|
||||
HeapTupleHeaderSetDatumLength(dtuple, tuple->t_len);
|
||||
dtuple = DatumGetHeapTupleHeader(ExecFetchSlotTupleDatum(slot));
|
||||
|
||||
/*
|
||||
* If the Var identifies a named composite type, label the tuple with that
|
||||
* type; otherwise use what is in the tupleDesc.
|
||||
* If the Var identifies a named composite type, label the datum with that
|
||||
* type; otherwise we'll use the slot's info.
|
||||
*/
|
||||
if (variable->vartype != RECORDOID)
|
||||
{
|
||||
HeapTupleHeaderSetTypeId(dtuple, variable->vartype);
|
||||
HeapTupleHeaderSetTypMod(dtuple, variable->vartypmod);
|
||||
}
|
||||
else
|
||||
{
|
||||
HeapTupleHeaderSetTypeId(dtuple, tupleDesc->tdtypeid);
|
||||
HeapTupleHeaderSetTypMod(dtuple, tupleDesc->tdtypmod);
|
||||
}
|
||||
|
||||
return PointerGetDatum(dtuple);
|
||||
}
|
||||
|
|
@ -1029,13 +1015,13 @@ ExecEvalWholeRowSlow(WholeRowVarExprState *wrvstate, ExprContext *econtext,
|
|||
}
|
||||
|
||||
/*
|
||||
* We have to make a copy of the tuple so we can safely insert the Datum
|
||||
* overhead fields, which are not set in on-disk tuples.
|
||||
* Copy the slot tuple and make sure any toasted fields get detoasted.
|
||||
*/
|
||||
dtuple = (HeapTupleHeader) palloc(tuple->t_len);
|
||||
memcpy((char *) dtuple, (char *) tuple->t_data, tuple->t_len);
|
||||
dtuple = DatumGetHeapTupleHeader(ExecFetchSlotTupleDatum(slot));
|
||||
|
||||
HeapTupleHeaderSetDatumLength(dtuple, tuple->t_len);
|
||||
/*
|
||||
* Reset datum's type ID fields to match the Var.
|
||||
*/
|
||||
HeapTupleHeaderSetTypeId(dtuple, variable->vartype);
|
||||
HeapTupleHeaderSetTypMod(dtuple, variable->vartypmod);
|
||||
|
||||
|
|
|
|||
|
|
@ -89,6 +89,7 @@
|
|||
#include "postgres.h"
|
||||
|
||||
#include "access/htup_details.h"
|
||||
#include "access/tuptoaster.h"
|
||||
#include "funcapi.h"
|
||||
#include "catalog/pg_type.h"
|
||||
#include "nodes/nodeFuncs.h"
|
||||
|
|
@ -700,27 +701,21 @@ ExecFetchSlotMinimalTuple(TupleTableSlot *slot)
|
|||
* ExecFetchSlotTupleDatum
|
||||
* Fetch the slot's tuple as a composite-type Datum.
|
||||
*
|
||||
* We convert the slot's contents to local physical-tuple form,
|
||||
* and fill in the Datum header fields. Note that the result
|
||||
* always points to storage owned by the slot.
|
||||
* The result is always freshly palloc'd in the caller's memory context.
|
||||
* --------------------------------
|
||||
*/
|
||||
Datum
|
||||
ExecFetchSlotTupleDatum(TupleTableSlot *slot)
|
||||
{
|
||||
HeapTuple tup;
|
||||
HeapTupleHeader td;
|
||||
TupleDesc tupdesc;
|
||||
|
||||
/* Make sure we can scribble on the slot contents ... */
|
||||
tup = ExecMaterializeSlot(slot);
|
||||
/* ... and set up the composite-Datum header fields, in case not done */
|
||||
td = tup->t_data;
|
||||
/* Fetch slot's contents in regular-physical-tuple form */
|
||||
tup = ExecFetchSlotTuple(slot);
|
||||
tupdesc = slot->tts_tupleDescriptor;
|
||||
HeapTupleHeaderSetDatumLength(td, tup->t_len);
|
||||
HeapTupleHeaderSetTypeId(td, tupdesc->tdtypeid);
|
||||
HeapTupleHeaderSetTypMod(td, tupdesc->tdtypmod);
|
||||
return PointerGetDatum(td);
|
||||
|
||||
/* Convert to Datum form */
|
||||
return heap_copy_tuple_as_datum(tup, tupdesc);
|
||||
}
|
||||
|
||||
/* --------------------------------
|
||||
|
|
@ -1132,6 +1127,66 @@ BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values)
|
|||
return tuple;
|
||||
}
|
||||
|
||||
/*
|
||||
* HeapTupleHeaderGetDatum - convert a HeapTupleHeader pointer to a Datum.
|
||||
*
|
||||
* This must *not* get applied to an on-disk tuple; the tuple should be
|
||||
* freshly made by heap_form_tuple or some wrapper routine for it (such as
|
||||
* BuildTupleFromCStrings). Be sure also that the tupledesc used to build
|
||||
* the tuple has a properly "blessed" rowtype.
|
||||
*
|
||||
* Formerly this was a macro equivalent to PointerGetDatum, relying on the
|
||||
* fact that heap_form_tuple fills in the appropriate tuple header fields
|
||||
* for a composite Datum. However, we now require that composite Datums not
|
||||
* contain any external TOAST pointers. We do not want heap_form_tuple itself
|
||||
* to enforce that; more specifically, the rule applies only to actual Datums
|
||||
* and not to HeapTuple structures. Therefore, HeapTupleHeaderGetDatum is
|
||||
* now a function that detects whether there are externally-toasted fields
|
||||
* and constructs a new tuple with inlined fields if so. We still need
|
||||
* heap_form_tuple to insert the Datum header fields, because otherwise this
|
||||
* code would have no way to obtain a tupledesc for the tuple.
|
||||
*
|
||||
* Note that if we do build a new tuple, it's palloc'd in the current
|
||||
* memory context. Beware of code that changes context between the initial
|
||||
* heap_form_tuple/etc call and calling HeapTuple(Header)GetDatum.
|
||||
*
|
||||
* For performance-critical callers, it could be worthwhile to take extra
|
||||
* steps to ensure that there aren't TOAST pointers in the output of
|
||||
* heap_form_tuple to begin with. It's likely however that the costs of the
|
||||
* typcache lookup and tuple disassembly/reassembly are swamped by TOAST
|
||||
* dereference costs, so that the benefits of such extra effort would be
|
||||
* minimal.
|
||||
*
|
||||
* XXX it would likely be better to create wrapper functions that produce
|
||||
* a composite Datum from the field values in one step. However, there's
|
||||
* enough code using the existing APIs that we couldn't get rid of this
|
||||
* hack anytime soon.
|
||||
*/
|
||||
Datum
|
||||
HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
|
||||
{
|
||||
Datum result;
|
||||
TupleDesc tupDesc;
|
||||
|
||||
/* No work if there are no external TOAST pointers in the tuple */
|
||||
if (!HeapTupleHeaderHasExternal(tuple))
|
||||
return PointerGetDatum(tuple);
|
||||
|
||||
/* Use the type data saved by heap_form_tuple to look up the rowtype */
|
||||
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
|
||||
HeapTupleHeaderGetTypMod(tuple));
|
||||
|
||||
/* And do the flattening */
|
||||
result = toast_flatten_tuple_to_datum(tuple,
|
||||
HeapTupleHeaderGetDatumLength(tuple),
|
||||
tupDesc);
|
||||
|
||||
ReleaseTupleDesc(tupDesc);
|
||||
|
||||
return result;
|
||||
}
|
||||
|
||||
|
||||
/*
|
||||
* Functions for sending tuples to the frontend (or other specified destination)
|
||||
* as though it is a SELECT result. These are used by utility commands that
|
||||
|
|
|
|||
|
|
@ -954,7 +954,6 @@ postquel_get_single_result(TupleTableSlot *slot,
|
|||
/* We must return the whole tuple as a Datum. */
|
||||
fcinfo->isnull = false;
|
||||
value = ExecFetchSlotTupleDatum(slot);
|
||||
value = datumCopy(value, fcache->typbyval, fcache->typlen);
|
||||
}
|
||||
else
|
||||
{
|
||||
|
|
|
|||
|
|
@ -742,12 +742,7 @@ SPI_returntuple(HeapTuple tuple, TupleDesc tupdesc)
|
|||
oldcxt = MemoryContextSwitchTo(_SPI_current->savedcxt);
|
||||
}
|
||||
|
||||
dtup = (HeapTupleHeader) palloc(tuple->t_len);
|
||||
memcpy((char *) dtup, (char *) tuple->t_data, tuple->t_len);
|
||||
|
||||
HeapTupleHeaderSetDatumLength(dtup, tuple->t_len);
|
||||
HeapTupleHeaderSetTypeId(dtup, tupdesc->tdtypeid);
|
||||
HeapTupleHeaderSetTypMod(dtup, tupdesc->tdtypmod);
|
||||
dtup = DatumGetHeapTupleHeader(heap_copy_tuple_as_datum(tuple, tupdesc));
|
||||
|
||||
if (oldcxt)
|
||||
MemoryContextSwitchTo(oldcxt);
|
||||
|
|
|
|||
|
|
@ -19,6 +19,7 @@
|
|||
#include "access/htup_details.h"
|
||||
#include "access/tuptoaster.h"
|
||||
#include "catalog/pg_type.h"
|
||||
#include "funcapi.h"
|
||||
#include "libpq/pqformat.h"
|
||||
#include "utils/builtins.h"
|
||||
#include "utils/lsyscache.h"
|
||||
|
|
|
|||
|
|
@ -476,6 +476,9 @@ do { \
|
|||
(tup)->t_infomask2 = ((tup)->t_infomask2 & ~HEAP_NATTS_MASK) | (natts) \
|
||||
)
|
||||
|
||||
#define HeapTupleHeaderHasExternal(tup) \
|
||||
(((tup)->t_infomask & HEAP_HASEXTERNAL) != 0)
|
||||
|
||||
|
||||
/*
|
||||
* BITMAPLEN(NATTS) -
|
||||
|
|
@ -730,6 +733,7 @@ extern Datum heap_getsysattr(HeapTuple tup, int attnum, TupleDesc tupleDesc,
|
|||
bool *isnull);
|
||||
extern HeapTuple heap_copytuple(HeapTuple tuple);
|
||||
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
|
||||
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
|
||||
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
|
||||
Datum *values, bool *isnull);
|
||||
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
|
||||
|
|
|
|||
|
|
@ -184,16 +184,14 @@ extern struct varlena *heap_tuple_untoast_attr_slice(struct varlena * attr,
|
|||
extern HeapTuple toast_flatten_tuple(HeapTuple tup, TupleDesc tupleDesc);
|
||||
|
||||
/* ----------
|
||||
* toast_flatten_tuple_attribute -
|
||||
* toast_flatten_tuple_to_datum -
|
||||
*
|
||||
* If a Datum is of composite type, "flatten" it to contain no toasted fields.
|
||||
* This must be invoked on any potentially-composite field that is to be
|
||||
* inserted into a tuple. Doing this preserves the invariant that toasting
|
||||
* goes only one level deep in a tuple.
|
||||
* "Flatten" a tuple containing out-of-line toasted fields into a Datum.
|
||||
* ----------
|
||||
*/
|
||||
extern Datum toast_flatten_tuple_attribute(Datum value,
|
||||
Oid typeId, int32 typeMod);
|
||||
extern Datum toast_flatten_tuple_to_datum(HeapTupleHeader tup,
|
||||
uint32 tup_len,
|
||||
TupleDesc tupleDesc);
|
||||
|
||||
/* ----------
|
||||
* toast_compress_datum -
|
||||
|
|
|
|||
|
|
@ -313,7 +313,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena * datum);
|
|||
#define PG_RETURN_TEXT_P(x) PG_RETURN_POINTER(x)
|
||||
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
|
||||
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
|
||||
#define PG_RETURN_HEAPTUPLEHEADER(x) PG_RETURN_POINTER(x)
|
||||
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
|
||||
|
||||
|
||||
/*-------------------------------------------------------------------------
|
||||
|
|
|
|||
|
|
@ -200,6 +200,8 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
|
|||
* HeapTuple BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values) -
|
||||
* build a HeapTuple given user data in C string form. values is an array
|
||||
* of C strings, one for each attribute of the return tuple.
|
||||
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
|
||||
* HeapTupleHeader to a Datum.
|
||||
*
|
||||
* Macro declarations:
|
||||
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
|
||||
|
|
@ -216,9 +218,9 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
|
|||
*----------
|
||||
*/
|
||||
|
||||
#define HeapTupleGetDatum(_tuple) PointerGetDatum((_tuple)->t_data)
|
||||
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
|
||||
/* obsolete version of above */
|
||||
#define TupleGetDatum(_slot, _tuple) PointerGetDatum((_tuple)->t_data)
|
||||
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
|
||||
|
||||
extern TupleDesc RelationNameGetTupleDesc(const char *relname);
|
||||
extern TupleDesc TypeGetTupleDesc(Oid typeoid, List *colaliases);
|
||||
|
|
@ -227,6 +229,7 @@ extern TupleDesc TypeGetTupleDesc(Oid typeoid, List *colaliases);
|
|||
extern TupleDesc BlessTupleDesc(TupleDesc tupdesc);
|
||||
extern AttInMetadata *TupleDescGetAttInMetadata(TupleDesc tupdesc);
|
||||
extern HeapTuple BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values);
|
||||
extern Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple);
|
||||
extern TupleTableSlot *TupleDescGetSlot(TupleDesc tupdesc);
|
||||
|
||||
|
||||
|
|
|
|||
|
|
@ -4473,18 +4473,17 @@ exec_eval_datum(PLpgSQL_execstate *estate,
|
|||
tup = make_tuple_from_row(estate, row, row->rowtupdesc);
|
||||
if (tup == NULL) /* should not happen */
|
||||
elog(ERROR, "row not compatible with its own tupdesc");
|
||||
MemoryContextSwitchTo(oldcontext);
|
||||
*typeid = row->rowtupdesc->tdtypeid;
|
||||
*typetypmod = row->rowtupdesc->tdtypmod;
|
||||
*value = HeapTupleGetDatum(tup);
|
||||
*isnull = false;
|
||||
MemoryContextSwitchTo(oldcontext);
|
||||
break;
|
||||
}
|
||||
|
||||
case PLPGSQL_DTYPE_REC:
|
||||
{
|
||||
PLpgSQL_rec *rec = (PLpgSQL_rec *) datum;
|
||||
HeapTupleData worktup;
|
||||
|
||||
if (!HeapTupleIsValid(rec->tup))
|
||||
ereport(ERROR,
|
||||
|
|
@ -4496,21 +4495,12 @@ exec_eval_datum(PLpgSQL_execstate *estate,
|
|||
/* Make sure we have a valid type/typmod setting */
|
||||
BlessTupleDesc(rec->tupdesc);
|
||||
|
||||
/*
|
||||
* In a trigger, the NEW and OLD parameters are likely to be
|
||||
* on-disk tuples that don't have the desired Datum fields.
|
||||
* Copy the tuple body and insert the right values.
|
||||
*/
|
||||
oldcontext = MemoryContextSwitchTo(estate->eval_econtext->ecxt_per_tuple_memory);
|
||||
heap_copytuple_with_tuple(rec->tup, &worktup);
|
||||
HeapTupleHeaderSetDatumLength(worktup.t_data, worktup.t_len);
|
||||
HeapTupleHeaderSetTypeId(worktup.t_data, rec->tupdesc->tdtypeid);
|
||||
HeapTupleHeaderSetTypMod(worktup.t_data, rec->tupdesc->tdtypmod);
|
||||
MemoryContextSwitchTo(oldcontext);
|
||||
*typeid = rec->tupdesc->tdtypeid;
|
||||
*typetypmod = rec->tupdesc->tdtypmod;
|
||||
*value = HeapTupleGetDatum(&worktup);
|
||||
*value = heap_copy_tuple_as_datum(rec->tup, rec->tupdesc);
|
||||
*isnull = false;
|
||||
MemoryContextSwitchTo(oldcontext);
|
||||
break;
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1676,3 +1676,33 @@ select * from t1;
|
|||
[5:5]={"(42,43)"}
|
||||
(1 row)
|
||||
|
||||
-- Check that arrays of composites are safely detoasted when needed
|
||||
create temp table src (f1 text);
|
||||
insert into src
|
||||
select string_agg(random()::text,'') from generate_series(1,10000);
|
||||
create type textandtext as (c1 text, c2 text);
|
||||
create temp table dest (f1 textandtext[]);
|
||||
insert into dest select array[row(f1,f1)::textandtext] from src;
|
||||
select length(md5((f1[1]).c2)) from dest;
|
||||
length
|
||||
--------
|
||||
32
|
||||
(1 row)
|
||||
|
||||
delete from src;
|
||||
select length(md5((f1[1]).c2)) from dest;
|
||||
length
|
||||
--------
|
||||
32
|
||||
(1 row)
|
||||
|
||||
truncate table src;
|
||||
drop table src;
|
||||
select length(md5((f1[1]).c2)) from dest;
|
||||
length
|
||||
--------
|
||||
32
|
||||
(1 row)
|
||||
|
||||
drop table dest;
|
||||
drop type textandtext;
|
||||
|
|
|
|||
|
|
@ -810,5 +810,14 @@ make_tuple_indirect(PG_FUNCTION_ARGS)
|
|||
|
||||
MemoryContextSwitchTo(old_context);
|
||||
|
||||
PG_RETURN_HEAPTUPLEHEADER(newtup->t_data);
|
||||
/*
|
||||
* We intentionally don't use PG_RETURN_HEAPTUPLEHEADER here, because that
|
||||
* would cause the indirect toast pointers to be flattened out of the
|
||||
* tuple immediately, rendering subsequent testing irrelevant. So just
|
||||
* return the HeapTupleHeader pointer as-is. This violates the general
|
||||
* rule that composite Datums shouldn't contain toast pointers, but so
|
||||
* long as the regression test scripts don't insert the result of this
|
||||
* function into a container type (record, array, etc) it should be OK.
|
||||
*/
|
||||
PG_RETURN_POINTER(newtup->t_data);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -459,3 +459,20 @@ insert into t1 (f1[5].q1) values(42);
|
|||
select * from t1;
|
||||
update t1 set f1[5].q2 = 43;
|
||||
select * from t1;
|
||||
|
||||
-- Check that arrays of composites are safely detoasted when needed
|
||||
|
||||
create temp table src (f1 text);
|
||||
insert into src
|
||||
select string_agg(random()::text,'') from generate_series(1,10000);
|
||||
create type textandtext as (c1 text, c2 text);
|
||||
create temp table dest (f1 textandtext[]);
|
||||
insert into dest select array[row(f1,f1)::textandtext] from src;
|
||||
select length(md5((f1[1]).c2)) from dest;
|
||||
delete from src;
|
||||
select length(md5((f1[1]).c2)) from dest;
|
||||
truncate table src;
|
||||
drop table src;
|
||||
select length(md5((f1[1]).c2)) from dest;
|
||||
drop table dest;
|
||||
drop type textandtext;
|
||||
|
|
|
|||
Loading…
Reference in New Issue