2009-08-18 12:34:39 +02:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
|
|
|
* unaccent.c
|
|
|
|
* Text search unaccent dictionary
|
|
|
|
*
|
2020-01-01 18:21:45 +01:00
|
|
|
* Copyright (c) 2009-2020, PostgreSQL Global Development Group
|
2009-08-18 12:34:39 +02:00
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
2010-09-20 22:08:53 +02:00
|
|
|
* contrib/unaccent/unaccent.c
|
2009-08-18 12:34:39 +02:00
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include "postgres.h"
|
|
|
|
|
|
|
|
#include "catalog/namespace.h"
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
2018-11-21 00:36:57 +01:00
|
|
|
#include "catalog/pg_ts_dict.h"
|
2009-08-18 12:34:39 +02:00
|
|
|
#include "commands/defrem.h"
|
2014-07-01 17:22:43 +02:00
|
|
|
#include "lib/stringinfo.h"
|
2009-08-18 12:34:39 +02:00
|
|
|
#include "tsearch/ts_cache.h"
|
|
|
|
#include "tsearch/ts_locale.h"
|
|
|
|
#include "tsearch/ts_public.h"
|
|
|
|
#include "utils/builtins.h"
|
2018-09-06 16:49:45 +02:00
|
|
|
#include "utils/lsyscache.h"
|
2017-01-21 02:29:53 +01:00
|
|
|
#include "utils/regproc.h"
|
2018-09-06 16:49:45 +02:00
|
|
|
#include "utils/syscache.h"
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
PG_MODULE_MAGIC;
|
|
|
|
|
|
|
|
/*
|
2014-07-01 03:46:29 +02:00
|
|
|
* An unaccent dictionary uses a trie to find a string to replace. Each node
|
|
|
|
* of the trie is an array of 256 TrieChar structs; the N-th element of the
|
|
|
|
* array corresponds to next byte value N. That element can contain both a
|
|
|
|
* replacement string (to be used if the source string ends with this byte)
|
|
|
|
* and a link to another trie node (to be followed if there are more bytes).
|
|
|
|
*
|
|
|
|
* Note that the trie search logic pays no attention to multibyte character
|
|
|
|
* boundaries. This is OK as long as both the data entered into the trie and
|
|
|
|
* the data we're trying to look up are validly encoded; no partial-character
|
|
|
|
* matches will occur.
|
2009-08-18 12:34:39 +02:00
|
|
|
*/
|
2013-05-08 19:57:42 +02:00
|
|
|
typedef struct TrieChar
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2013-05-08 19:57:42 +02:00
|
|
|
struct TrieChar *nextChar;
|
2009-08-18 12:34:39 +02:00
|
|
|
char *replaceTo;
|
|
|
|
int replacelen;
|
2013-05-08 19:57:42 +02:00
|
|
|
} TrieChar;
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
/*
|
2013-05-08 19:57:42 +02:00
|
|
|
* placeChar - put str into trie's structure, byte by byte.
|
2014-07-01 03:46:29 +02:00
|
|
|
*
|
|
|
|
* If node is NULL, we need to make a new node, which will be returned;
|
|
|
|
* otherwise the return value is the same as node.
|
2009-08-18 12:34:39 +02:00
|
|
|
*/
|
2013-05-08 19:57:42 +02:00
|
|
|
static TrieChar *
|
2014-07-01 03:46:29 +02:00
|
|
|
placeChar(TrieChar *node, const unsigned char *str, int lenstr,
|
|
|
|
const char *replaceTo, int replacelen)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2013-05-08 19:57:42 +02:00
|
|
|
TrieChar *curnode;
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
if (!node)
|
2014-07-01 03:46:29 +02:00
|
|
|
node = (TrieChar *) palloc0(sizeof(TrieChar) * 256);
|
|
|
|
|
|
|
|
Assert(lenstr > 0); /* else str[0] doesn't exist */
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
curnode = node + *str;
|
|
|
|
|
2014-07-01 03:46:29 +02:00
|
|
|
if (lenstr <= 1)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
|
|
|
if (curnode->replaceTo)
|
2014-07-01 04:03:37 +02:00
|
|
|
ereport(WARNING,
|
|
|
|
(errcode(ERRCODE_CONFIG_FILE_ERROR),
|
|
|
|
errmsg("duplicate source strings, first one will be used")));
|
2009-08-18 12:34:39 +02:00
|
|
|
else
|
|
|
|
{
|
|
|
|
curnode->replacelen = replacelen;
|
2014-07-01 03:46:29 +02:00
|
|
|
curnode->replaceTo = (char *) palloc(replacelen);
|
2009-08-18 12:34:39 +02:00
|
|
|
memcpy(curnode->replaceTo, replaceTo, replacelen);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2014-07-01 03:46:29 +02:00
|
|
|
curnode->nextChar = placeChar(curnode->nextChar, str + 1, lenstr - 1,
|
|
|
|
replaceTo, replacelen);
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
return node;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2013-05-08 19:57:42 +02:00
|
|
|
* initTrie - create trie from file.
|
|
|
|
*
|
|
|
|
* Function converts UTF8-encoded file into current encoding.
|
2009-08-18 12:34:39 +02:00
|
|
|
*/
|
2013-05-08 19:57:42 +02:00
|
|
|
static TrieChar *
|
2017-10-31 15:34:31 +01:00
|
|
|
initTrie(const char *filename)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2013-05-08 19:57:42 +02:00
|
|
|
TrieChar *volatile rootTrie = NULL;
|
2009-08-18 12:34:39 +02:00
|
|
|
MemoryContext ccxt = CurrentMemoryContext;
|
|
|
|
tsearch_readline_state trst;
|
2009-08-18 17:57:26 +02:00
|
|
|
volatile bool skip;
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
filename = get_tsearch_config_filename(filename, "rules");
|
|
|
|
if (!tsearch_readline_begin(&trst, filename))
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_CONFIG_FILE_ERROR),
|
|
|
|
errmsg("could not open unaccent file \"%s\": %m",
|
|
|
|
filename)));
|
|
|
|
|
|
|
|
do
|
|
|
|
{
|
2011-11-07 17:48:53 +01:00
|
|
|
/*
|
|
|
|
* pg_do_encoding_conversion() (called by tsearch_readline()) will
|
|
|
|
* emit exception if it finds untranslatable characters in current
|
|
|
|
* locale. We just skip such lines, continuing with the next.
|
|
|
|
*/
|
2009-08-18 12:34:39 +02:00
|
|
|
skip = true;
|
|
|
|
|
|
|
|
PG_TRY();
|
|
|
|
{
|
2011-11-07 17:48:53 +01:00
|
|
|
char *line;
|
|
|
|
|
2009-08-18 12:34:39 +02:00
|
|
|
while ((line = tsearch_readline(&trst)) != NULL)
|
|
|
|
{
|
2014-07-01 02:51:26 +02:00
|
|
|
/*----------
|
|
|
|
* The format of each line must be "src" or "src trg", where
|
|
|
|
* src and trg are sequences of one or more non-whitespace
|
|
|
|
* characters, separated by whitespace. Whitespace at start
|
|
|
|
* or end of line is ignored. If trg is omitted, an empty
|
|
|
|
* string is used as the replacement.
|
|
|
|
*
|
|
|
|
* We use a simple state machine, with states
|
|
|
|
* 0 initial (before src)
|
|
|
|
* 1 in src
|
|
|
|
* 2 in whitespace after src
|
|
|
|
* 3 in trg
|
|
|
|
* 4 in whitespace after trg
|
2014-07-01 04:03:37 +02:00
|
|
|
* -1 syntax error detected
|
2014-07-01 02:51:26 +02:00
|
|
|
*----------
|
2011-11-07 17:48:53 +01:00
|
|
|
*/
|
|
|
|
int state;
|
|
|
|
char *ptr;
|
|
|
|
char *src = NULL;
|
|
|
|
char *trg = NULL;
|
|
|
|
int ptrlen;
|
|
|
|
int srclen = 0;
|
|
|
|
int trglen = 0;
|
|
|
|
|
|
|
|
state = 0;
|
|
|
|
for (ptr = line; *ptr; ptr += ptrlen)
|
|
|
|
{
|
|
|
|
ptrlen = pg_mblen(ptr);
|
|
|
|
/* ignore whitespace, but end src or trg */
|
|
|
|
if (t_isspace(ptr))
|
|
|
|
{
|
|
|
|
if (state == 1)
|
|
|
|
state = 2;
|
|
|
|
else if (state == 3)
|
|
|
|
state = 4;
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
switch (state)
|
|
|
|
{
|
|
|
|
case 0:
|
|
|
|
/* start of src */
|
|
|
|
src = ptr;
|
|
|
|
srclen = ptrlen;
|
|
|
|
state = 1;
|
|
|
|
break;
|
|
|
|
case 1:
|
|
|
|
/* continue src */
|
|
|
|
srclen += ptrlen;
|
|
|
|
break;
|
|
|
|
case 2:
|
|
|
|
/* start of trg */
|
|
|
|
trg = ptr;
|
|
|
|
trglen = ptrlen;
|
|
|
|
state = 3;
|
|
|
|
break;
|
|
|
|
case 3:
|
|
|
|
/* continue trg */
|
|
|
|
trglen += ptrlen;
|
|
|
|
break;
|
|
|
|
default:
|
|
|
|
/* bogus line format */
|
|
|
|
state = -1;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2014-07-01 02:51:26 +02:00
|
|
|
if (state == 1 || state == 2)
|
|
|
|
{
|
|
|
|
/* trg was omitted, so use "" */
|
|
|
|
trg = "";
|
|
|
|
trglen = 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (state > 0)
|
2013-05-08 19:57:42 +02:00
|
|
|
rootTrie = placeChar(rootTrie,
|
2011-11-07 17:48:53 +01:00
|
|
|
(unsigned char *) src, srclen,
|
|
|
|
trg, trglen);
|
2014-07-01 04:03:37 +02:00
|
|
|
else if (state < 0)
|
|
|
|
ereport(WARNING,
|
|
|
|
(errcode(ERRCODE_CONFIG_FILE_ERROR),
|
|
|
|
errmsg("invalid syntax: more than two strings in unaccent rule")));
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
pfree(line);
|
|
|
|
}
|
2011-11-07 17:48:53 +01:00
|
|
|
skip = false;
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
|
|
|
PG_CATCH();
|
|
|
|
{
|
|
|
|
ErrorData *errdata;
|
|
|
|
MemoryContext ecxt;
|
|
|
|
|
|
|
|
ecxt = MemoryContextSwitchTo(ccxt);
|
|
|
|
errdata = CopyErrorData();
|
|
|
|
if (errdata->sqlerrcode == ERRCODE_UNTRANSLATABLE_CHARACTER)
|
|
|
|
{
|
|
|
|
FlushErrorState();
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
MemoryContextSwitchTo(ecxt);
|
|
|
|
PG_RE_THROW();
|
|
|
|
}
|
|
|
|
}
|
|
|
|
PG_END_TRY();
|
|
|
|
}
|
|
|
|
while (skip);
|
|
|
|
|
|
|
|
tsearch_readline_end(&trst);
|
|
|
|
|
2013-05-08 19:57:42 +02:00
|
|
|
return rootTrie;
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2014-07-01 03:46:29 +02:00
|
|
|
* findReplaceTo - find longest possible match in trie
|
|
|
|
*
|
|
|
|
* On success, returns pointer to ending subnode, plus length of matched
|
|
|
|
* source string in *p_matchlen. On failure, returns NULL.
|
2009-08-18 12:34:39 +02:00
|
|
|
*/
|
2013-05-08 19:57:42 +02:00
|
|
|
static TrieChar *
|
2014-07-01 03:46:29 +02:00
|
|
|
findReplaceTo(TrieChar *node, const unsigned char *src, int srclen,
|
|
|
|
int *p_matchlen)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2014-07-01 03:46:29 +02:00
|
|
|
TrieChar *result = NULL;
|
|
|
|
int matchlen = 0;
|
|
|
|
|
|
|
|
*p_matchlen = 0; /* prevent uninitialized-variable warnings */
|
|
|
|
|
|
|
|
while (node && matchlen < srclen)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2014-07-01 03:46:29 +02:00
|
|
|
node = node + src[matchlen];
|
|
|
|
matchlen++;
|
|
|
|
|
|
|
|
if (node->replaceTo)
|
|
|
|
{
|
|
|
|
result = node;
|
|
|
|
*p_matchlen = matchlen;
|
|
|
|
}
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
node = node->nextChar;
|
|
|
|
}
|
|
|
|
|
2014-07-01 03:46:29 +02:00
|
|
|
return result;
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
PG_FUNCTION_INFO_V1(unaccent_init);
|
|
|
|
Datum
|
|
|
|
unaccent_init(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
List *dictoptions = (List *) PG_GETARG_POINTER(0);
|
2013-05-08 19:57:42 +02:00
|
|
|
TrieChar *rootTrie = NULL;
|
2009-08-18 12:34:39 +02:00
|
|
|
bool fileloaded = false;
|
|
|
|
ListCell *l;
|
|
|
|
|
|
|
|
foreach(l, dictoptions)
|
|
|
|
{
|
|
|
|
DefElem *defel = (DefElem *) lfirst(l);
|
|
|
|
|
Avoid unnecessary use of pg_strcasecmp for already-downcased identifiers.
We have a lot of code in which option names, which from the user's
viewpoint are logically keywords, are passed through the grammar as plain
identifiers, and then matched to string literals during command execution.
This approach avoids making words into lexer keywords unnecessarily. Some
places matched these strings using plain strcmp, some using pg_strcasecmp.
But the latter should be unnecessary since identifiers would have been
downcased on their way through the parser. Aside from any efficiency
concerns (probably not a big factor), the lack of consistency in this area
creates a hazard of subtle bugs due to different places coming to different
conclusions about whether two option names are the same or different.
Hence, standardize on using strcmp() to match any option names that are
expected to have been fed through the parser.
This does create a user-visible behavioral change, which is that while
formerly all of these would work:
alter table foo set (fillfactor = 50);
alter table foo set (FillFactor = 50);
alter table foo set ("fillfactor" = 50);
alter table foo set ("FillFactor" = 50);
now the last case will fail because that double-quoted identifier is
different from the others. However, none of our documentation says that
you can use a quoted identifier in such contexts at all, and we should
discourage doing so since it would break if we ever decide to parse such
constructs as true lexer keywords rather than poor man's substitutes.
So this shouldn't create a significant compatibility issue for users.
Daniel Gustafsson, reviewed by Michael Paquier, small changes by me
Discussion: https://postgr.es/m/29405B24-564E-476B-98C0-677A29805B84@yesql.se
2018-01-27 00:25:02 +01:00
|
|
|
if (strcmp(defel->defname, "rules") == 0)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
|
|
|
if (fileloaded)
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
|
|
|
|
errmsg("multiple Rules parameters")));
|
2013-05-08 19:57:42 +02:00
|
|
|
rootTrie = initTrie(defGetString(defel));
|
2009-08-18 12:34:39 +02:00
|
|
|
fileloaded = true;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
|
|
|
|
errmsg("unrecognized Unaccent parameter: \"%s\"",
|
|
|
|
defel->defname)));
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
if (!fileloaded)
|
|
|
|
{
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
|
|
|
|
errmsg("missing Rules parameter")));
|
|
|
|
}
|
|
|
|
|
2013-05-08 19:57:42 +02:00
|
|
|
PG_RETURN_POINTER(rootTrie);
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
PG_FUNCTION_INFO_V1(unaccent_lexize);
|
|
|
|
Datum
|
|
|
|
unaccent_lexize(PG_FUNCTION_ARGS)
|
|
|
|
{
|
2013-05-08 19:57:42 +02:00
|
|
|
TrieChar *rootTrie = (TrieChar *) PG_GETARG_POINTER(0);
|
2009-08-18 12:34:39 +02:00
|
|
|
char *srcchar = (char *) PG_GETARG_POINTER(1);
|
|
|
|
int32 len = PG_GETARG_INT32(2);
|
2014-07-01 17:22:43 +02:00
|
|
|
char *srcstart = srcchar;
|
|
|
|
TSLexeme *res;
|
|
|
|
StringInfoData buf;
|
|
|
|
|
|
|
|
/* we allocate storage for the buffer only if needed */
|
|
|
|
buf.data = NULL;
|
2009-08-18 12:34:39 +02:00
|
|
|
|
2014-07-01 03:46:29 +02:00
|
|
|
while (len > 0)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2014-07-01 03:46:29 +02:00
|
|
|
TrieChar *node;
|
|
|
|
int matchlen;
|
2009-08-18 12:34:39 +02:00
|
|
|
|
2014-07-01 03:46:29 +02:00
|
|
|
node = findReplaceTo(rootTrie, (unsigned char *) srcchar, len,
|
|
|
|
&matchlen);
|
2009-08-18 12:34:39 +02:00
|
|
|
if (node && node->replaceTo)
|
|
|
|
{
|
2014-07-01 17:22:43 +02:00
|
|
|
if (buf.data == NULL)
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2014-07-01 17:22:43 +02:00
|
|
|
/* initialize buffer */
|
|
|
|
initStringInfo(&buf);
|
|
|
|
/* insert any data we already skipped over */
|
2009-08-18 12:34:39 +02:00
|
|
|
if (srcchar != srcstart)
|
2014-07-01 17:22:43 +02:00
|
|
|
appendBinaryStringInfo(&buf, srcstart, srcchar - srcstart);
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
2014-07-01 17:22:43 +02:00
|
|
|
appendBinaryStringInfo(&buf, node->replaceTo, node->replacelen);
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
2014-07-01 03:46:29 +02:00
|
|
|
else
|
2009-08-18 12:34:39 +02:00
|
|
|
{
|
2014-07-01 03:46:29 +02:00
|
|
|
matchlen = pg_mblen(srcchar);
|
2014-07-01 17:22:43 +02:00
|
|
|
if (buf.data != NULL)
|
|
|
|
appendBinaryStringInfo(&buf, srcchar, matchlen);
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
|
|
|
|
2014-07-01 03:46:29 +02:00
|
|
|
srcchar += matchlen;
|
|
|
|
len -= matchlen;
|
2009-08-18 12:34:39 +02:00
|
|
|
}
|
|
|
|
|
2014-07-01 17:22:43 +02:00
|
|
|
/* return a result only if we made at least one substitution */
|
|
|
|
if (buf.data != NULL)
|
|
|
|
{
|
|
|
|
res = (TSLexeme *) palloc0(sizeof(TSLexeme) * 2);
|
|
|
|
res->lexeme = buf.data;
|
|
|
|
res->flags = TSL_FILTER;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
res = NULL;
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
PG_RETURN_POINTER(res);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Function-like wrapper for dictionary
|
|
|
|
*/
|
|
|
|
PG_FUNCTION_INFO_V1(unaccent_dict);
|
|
|
|
Datum
|
|
|
|
unaccent_dict(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
text *str;
|
|
|
|
int strArg;
|
|
|
|
Oid dictOid;
|
|
|
|
TSDictionaryCacheEntry *dict;
|
|
|
|
TSLexeme *res;
|
|
|
|
|
|
|
|
if (PG_NARGS() == 1)
|
|
|
|
{
|
2018-09-06 16:49:45 +02:00
|
|
|
/*
|
|
|
|
* Use the "unaccent" dictionary that is in the same schema that this
|
|
|
|
* function is in.
|
|
|
|
*/
|
|
|
|
Oid procnspid = get_func_namespace(fcinfo->flinfo->fn_oid);
|
|
|
|
const char *dictname = "unaccent";
|
|
|
|
|
Remove WITH OIDS support, change oid catalog column visibility.
Previously tables declared WITH OIDS, including a significant fraction
of the catalog tables, stored the oid column not as a normal column,
but as part of the tuple header.
This special column was not shown by default, which was somewhat odd,
as it's often (consider e.g. pg_class.oid) one of the more important
parts of a row. Neither pg_dump nor COPY included the contents of the
oid column by default.
The fact that the oid column was not an ordinary column necessitated a
significant amount of special case code to support oid columns. That
already was painful for the existing, but upcoming work aiming to make
table storage pluggable, would have required expanding and duplicating
that "specialness" significantly.
WITH OIDS has been deprecated since 2005 (commit ff02d0a05280e0).
Remove it.
Removing includes:
- CREATE TABLE and ALTER TABLE syntax for declaring the table to be
WITH OIDS has been removed (WITH (oids[ = true]) will error out)
- pg_dump does not support dumping tables declared WITH OIDS and will
issue a warning when dumping one (and ignore the oid column).
- restoring an pg_dump archive with pg_restore will warn when
restoring a table with oid contents (and ignore the oid column)
- COPY will refuse to load binary dump that includes oids.
- pg_upgrade will error out when encountering tables declared WITH
OIDS, they have to be altered to remove the oid column first.
- Functionality to access the oid of the last inserted row (like
plpgsql's RESULT_OID, spi's SPI_lastoid, ...) has been removed.
The syntax for declaring a table WITHOUT OIDS (or WITH (oids = false)
for CREATE TABLE) is still supported. While that requires a bit of
support code, it seems unnecessary to break applications / dumps that
do not use oids, and are explicit about not using them.
The biggest user of WITH OID columns was postgres' catalog. This
commit changes all 'magic' oid columns to be columns that are normally
declared and stored. To reduce unnecessary query breakage all the
newly added columns are still named 'oid', even if a table's column
naming scheme would indicate 'reloid' or such. This obviously
requires adapting a lot code, mostly replacing oid access via
HeapTupleGetOid() with access to the underlying Form_pg_*->oid column.
The bootstrap process now assigns oids for all oid columns in
genbki.pl that do not have an explicit value (starting at the largest
oid previously used), only oids assigned later by oids will be above
FirstBootstrapObjectId. As the oid column now is a normal column the
special bootstrap syntax for oids has been removed.
Oids are not automatically assigned during insertion anymore, all
backend code explicitly assigns oids with GetNewOidWithIndex(). For
the rare case that insertions into the catalog via SQL are called for
the new pg_nextoid() function can be used (which only works on catalog
tables).
The fact that oid columns on system tables are now normal columns
means that they will be included in the set of columns expanded
by * (i.e. SELECT * FROM pg_class will now include the table's oid,
previously it did not). It'd not technically be hard to hide oid
column by default, but that'd mean confusing behavior would either
have to be carried forward forever, or it'd cause breakage down the
line.
While it's not unlikely that further adjustments are needed, the
scope/invasiveness of the patch makes it worthwhile to get merge this
now. It's painful to maintain externally, too complicated to commit
after the code code freeze, and a dependency of a number of other
patches.
Catversion bump, for obvious reasons.
Author: Andres Freund, with contributions by John Naylor
Discussion: https://postgr.es/m/20180930034810.ywp2c7awz7opzcfr@alap3.anarazel.de
2018-11-21 00:36:57 +01:00
|
|
|
dictOid = GetSysCacheOid2(TSDICTNAMENSP, Anum_pg_ts_dict_oid,
|
2018-09-06 16:49:45 +02:00
|
|
|
PointerGetDatum(dictname),
|
|
|
|
ObjectIdGetDatum(procnspid));
|
|
|
|
if (!OidIsValid(dictOid))
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_UNDEFINED_OBJECT),
|
|
|
|
errmsg("text search dictionary \"%s.%s\" does not exist",
|
|
|
|
get_namespace_name(procnspid), dictname)));
|
2009-08-18 12:34:39 +02:00
|
|
|
strArg = 0;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
dictOid = PG_GETARG_OID(0);
|
|
|
|
strArg = 1;
|
|
|
|
}
|
2017-03-13 00:35:34 +01:00
|
|
|
str = PG_GETARG_TEXT_PP(strArg);
|
2009-08-18 12:34:39 +02:00
|
|
|
|
|
|
|
dict = lookup_ts_dictionary_cache(dictOid);
|
|
|
|
|
|
|
|
res = (TSLexeme *) DatumGetPointer(FunctionCall4(&(dict->lexize),
|
|
|
|
PointerGetDatum(dict->dictData),
|
2017-03-13 00:35:34 +01:00
|
|
|
PointerGetDatum(VARDATA_ANY(str)),
|
|
|
|
Int32GetDatum(VARSIZE_ANY_EXHDR(str)),
|
2009-08-18 12:34:39 +02:00
|
|
|
PointerGetDatum(NULL)));
|
|
|
|
|
|
|
|
PG_FREE_IF_COPY(str, strArg);
|
|
|
|
|
|
|
|
if (res == NULL)
|
|
|
|
{
|
|
|
|
PG_RETURN_TEXT_P(PG_GETARG_TEXT_P_COPY(strArg));
|
|
|
|
}
|
|
|
|
else if (res->lexeme == NULL)
|
|
|
|
{
|
|
|
|
pfree(res);
|
|
|
|
PG_RETURN_TEXT_P(PG_GETARG_TEXT_P_COPY(strArg));
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
text *txt = cstring_to_text(res->lexeme);
|
|
|
|
|
|
|
|
pfree(res->lexeme);
|
|
|
|
pfree(res);
|
|
|
|
|
|
|
|
PG_RETURN_TEXT_P(txt);
|
|
|
|
}
|
|
|
|
}
|