rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/bin/psql/psqlscan.l

2061 lines
52 KiB
Plaintext
Raw Normal View History

Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
%top{
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*-------------------------------------------------------------------------
*
* psqlscan.l
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
* lexical scanner for psql (and other frontend programs)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*
* This code is mainly needed to determine where the end of a SQL statement
* is: we are looking for semicolons that are not within quotes, comments,
* or parentheses. The most reliable way to handle this is to borrow the
* backend's flex lexer rules, lock, stock, and barrel. The rules below
* are (except for a few) the same as the backend's, but their actions are
* just ECHO whereas the backend's actions generally do other things.
*
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
* XXX The rules in this file must be kept in sync with the backend lexer!!!
*
* XXX Avoid creating backtracking cases --- see the backend lexer for info.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*
* The most difficult aspect of this code is that we need to work in multibyte
* encodings that are not ASCII-safe. A "safe" encoding is one in which each
* byte of a multibyte character has the high bit set (it's >= 0x80). Since
* all our lexing rules treat all high-bit-set characters alike, we don't
* really need to care whether such a byte is part of a sequence or not.
* In an "unsafe" encoding, we still expect the first byte of a multibyte
* sequence to be >= 0x80, but later bytes might not be. If we scan such
* a sequence as-is, the lexing rules could easily be fooled into matching
* such bytes to ordinary ASCII characters. Our solution for this is to
* substitute 0xFF for each non-first byte within the data presented to flex.
* The flex rules will then pass the FF's through unmolested. The emit()
* subroutine is responsible for looking back to the original string and
* replacing FF's with the corresponding original bytes.
*
Update copyright for 2016 Backpatch certain files through 9.1
2016-01-02 19:33:40 +01:00
* Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/bin/psql/psqlscan.l
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*
*-------------------------------------------------------------------------
*/
#include "postgres_fe.h"
#include "psqlscan.h"
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
#include "libpq-fe.h"
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
%{
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* We use a stack of flex buffers to handle substitution of psql variables.
* Each stacked buffer contains the as-yet-unread text from one psql variable.
* When we pop the stack all the way, we resume reading from the outer buffer
* identified by scanbufhandle.
*/
typedef struct StackElem
{
YY_BUFFER_STATE buf; /* flex input control structure */
char *bufstring; /* data actually being scanned by flex */
char *origstring; /* copy of original data, if needed */
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
char *varname; /* name of variable providing data, or NULL */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
struct StackElem *next;
} StackElem;
/*
* All working state of the lexer must be stored in PsqlScanStateData
* between calls. This allows us to have multiple open lexer operations,
* which is needed for nested include files. The lexer itself is not
* recursive, but it must be re-entrant.
*/
typedef struct PsqlScanStateData
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yyscan_t scanner; /* Flex's state for this PsqlScanState */
PQExpBuffer output_buf; /* current output buffer */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
StackElem *buffer_stack; /* stack of variable expansion buffers */
/*
* These variables always refer to the outer buffer, never to any
* stacked variable-expansion buffer.
*/
YY_BUFFER_STATE scanbufhandle;
char *scanbuf; /* start of outer-level input buffer */
const char *scanline; /* current input line at outer level */
/* safe_encoding, curline, refline are used by emit() to replace FFs */
int encoding; /* encoding being used now */
bool safe_encoding; /* is current encoding "safe"? */
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
bool std_strings; /* are string literals standard? */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
const char *curline; /* actual flex input string for cur buf */
const char *refline; /* original data for cur buffer */
/*
* All this state lives across successive input lines, until explicitly
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
* reset by psql_scan_reset. start_state is adopted by yylex() on
* entry, and updated with its finishing state on exit.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
int start_state; /* yylex's starting/finishing state */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
int paren_depth; /* depth of nesting in parentheses */
int xcdepth; /* depth of nesting in slash-star comments */
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
char *dolqstart; /* current $foo$ quote start string */
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
/*
* Callback functions provided by the program making use of the lexer.
*/
const PsqlScanCallbacks *callbacks;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
} PsqlScanStateData;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/*
* Set the type of yyextra; we use it as a pointer back to the containing
* PsqlScanState.
*/
#define YY_EXTRA_TYPE PsqlScanState
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/*
* These variables do not need to be saved across calls. Yeah, it's a bit
* of a hack, but putting them into PsqlScanStateData would be klugy too.
*/
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
static enum slash_option_type option_type;
static char *option_quote;
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
static int unquoted_option_chars;
static int backtick_start_offset;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Return values from yylex() */
#define LEXRES_EOL 0 /* end of input */
#define LEXRES_SEMI 1 /* command-terminating semicolon found */
#define LEXRES_BACKSLASH 2 /* backslash command start */
#define LEXRES_OK 3 /* OK completion of backslash argument */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
static void evaluate_backtick(PsqlScanState state);
static void push_new_buffer(PsqlScanState state,
const char *newstr, const char *varname);
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
static void pop_buffer_stack(PsqlScanState state);
static bool var_is_current_source(PsqlScanState state, const char *varname);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
static YY_BUFFER_STATE prepare_buffer(PsqlScanState state,
const char *txt, int len,
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
char **txtcopy);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
static void emit(PsqlScanState state, const char *txt, int len);
static char *extract_substring(PsqlScanState state, const char *txt, int len);
static void escape_variable(PsqlScanState state, const char *txt, int len,
bool as_ident);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
#define ECHO emit(cur_state, yytext, yyleng)
/*
* Work around a bug in flex 2.5.35: it emits a couple of functions that
* it forgets to emit declarations for. Since we use -Wmissing-prototypes,
* this would cause warnings. Providing our own declarations should be
* harmless even when the bug gets fixed.
*/
extern int psql_yyget_column(yyscan_t yyscanner);
extern void psql_yyset_column(int column_no, yyscan_t yyscanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
%}
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
%option reentrant
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
%option 8bit
%option never-interactive
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
%option nodefault
Add "%option noinput" to the scanners to avoid compiler warnings. GCC 4.3 began to realize that the input() function isn't used and printed warnings.
2008-05-09 17:36:31 +02:00
%option noinput
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
%option nounput
%option noyywrap
Add "%option warn" to all flex input files that lacked it. This is recommended in the flex manual, and there seems no good reason not to use it everywhere.
2011-08-25 19:55:57 +02:00
%option warn
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
%option prefix="psql_yy"
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* All of the following definitions and rules should exactly match
* src/backend/parser/scan.l so far as the flex patterns are concerned.
* The rule bodies are just ECHO as opposed to what the backend does,
* however. (But be sure to duplicate code that affects the lexing process,
* such as BEGIN().) Also, psqlscan uses a single <<EOF>> rule whereas
* scan.l has a separate one for each exclusive state.
*/
/*
* OK, here is a short description of lex/flex rules behavior.
* The longest pattern which matches an input string is always chosen.
* For equal-length patterns, the first occurring in the rules list is chosen.
* INITIAL is the starting state, to which all non-conditional rules apply.
* Exclusive states change parsing rules while the state is active. When in
* an exclusive state, only those rules defined for that state apply.
*
* We use exclusive states for quoted strings, extended comments,
* and to eliminate parsing troubles for numeric strings.
* Exclusive states:
* <xb> bit string literal
* <xc> extended C-style comments
* <xd> delimited identifiers (double-quoted identifiers)
* <xh> hexadecimal numeric string
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
* <xq> standard quoted strings
* <xe> extended quoted strings (support backslash escape sequences)
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
* <xdolq> $foo$ quoted strings
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
* <xui> quoted identifier with Unicode escapes
Fix lexing of U& sequences just before EOF. Commit a5ff502fceadc7c203b0d7a11b45c73f1b421f69 was a brick shy of a load in the backend lexer too, not just psql. Per further testing of bug #9068. In passing, improve related comments.
2014-02-04 01:47:57 +01:00
* <xuiend> end of a quoted identifier with Unicode escapes, UESCAPE can follow
* <xus> quoted string with Unicode escapes
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
* <xusend> end of a quoted string with Unicode escapes, UESCAPE can follow
Fix up some oversights in psql's Unicode-escape support. Original patch failed to include new exclusive states in a switch that needed to include them; and also was guilty of very fuzzy thinking about how to handle error cases. Per bug #5729 from Alan Choi.
2010-10-27 04:23:04 +02:00
*
* Note: we intentionally don't mimic the backend's <xeu> state; we have
* no need to distinguish it from <xe> state, and no good way to get out
* of it in error cases. The backend just throws yyerror() in those
* cases, but that's not an option here.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
%x xb
%x xc
%x xd
%x xh
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
%x xe
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
%x xq
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
%x xdolq
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
%x xui
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
%x xuiend
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
%x xus
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
%x xusend
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Additional exclusive states for psql only: lex backslash commands */
%x xslashcmd
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
%x xslashargstart
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
%x xslasharg
%x xslashquote
%x xslashbackquote
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
%x xslashdquote
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
%x xslashwholeline
%x xslashend
/*
* In order to make the world safe for Windows and Mac clients as well as
* Unix ones, we accept either \n or \r as a newline. A DOS-style \r\n
* sequence will be seen as two successive newlines, but that doesn't cause
* any problems. Comments that start with -- and extend to the next
* newline are treated as equivalent to a single whitespace character.
*
* NOTE a fine point: if there is no newline following --, we will absorb
* everything to the end of the input as a comment. This is correct. Older
* versions of Postgres failed to recognize -- as a comment if the input
* did not end with a newline.
*
* XXX perhaps \f (formfeed) should be treated as a newline as well?
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
*
* XXX if you change the set of whitespace characters, fix scanner_isspace()
* to agree, and see also the plpgsql lexer.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
space [ \t\n\r\f]
horiz_space [ \t\f]
newline [\n\r]
non_newline [^\n\r]
comment ("--"{non_newline}*)
whitespace ({space}+|{comment})
/*
* SQL requires at least one newline in the whitespace separating
* string literals that are to be concatenated. Silly, but who are we
* to argue? Note that {whitespace_with_newline} should not have * after
* it, whereas {whitespace} should generally have a * after it...
*/
special_whitespace ({space}+|{comment}{newline})
horiz_whitespace ({horiz_space}|{comment})
whitespace_with_newline ({horiz_whitespace}*{newline}{special_whitespace}*)
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
/*
* To ensure that {quotecontinue} can be scanned without having to back up
* if the full pattern isn't matched, we include trailing whitespace in
* {quotestop}. This matches all cases where {quotecontinue} fails to match,
* except for {quote} followed by whitespace and just one "-" (not two,
* which would start a {comment}). To cover that we have {quotefail}.
* The actions for {quotestop} and {quotefail} must throw back characters
* beyond the quote proper.
*/
quote '
quotestop {quote}{whitespace}*
quotecontinue {quote}{whitespace_with_newline}{quote}
quotefail {quote}{whitespace}*"-"
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Bit string
* It is tempting to scan the string for only those characters
* which are allowed. However, this leads to silently swallowed
* characters if illegal characters are included in the string.
* For example, if xbinside is [01] then B'ABCD' is interpreted
* as a zero-length string, and the ABCD' is lost!
* Better to pass the string forward and let the input routines
* validate the contents.
*/
xbstart [bB]{quote}
xbinside [^']*
Code review for escape-strings patch. Sync psql and plpgsql lexers with main, avoid using a SQL-defined SQLSTATE for what is most definitely not a SQL-compatible error condition, fix documentation omissions, adhere to message style guidelines, don't use two GUC_REPORT variables when one is sufficient. Nothing done about pg_dump issues.
2005-06-26 21:16:07 +02:00
/* Hexadecimal number */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
xhstart [xX]{quote}
xhinside [^']*
Code review for escape-strings patch. Sync psql and plpgsql lexers with main, avoid using a SQL-defined SQLSTATE for what is most definitely not a SQL-compatible error condition, fix documentation omissions, adhere to message style guidelines, don't use two GUC_REPORT variables when one is sufficient. Nothing done about pg_dump issues.
2005-06-26 21:16:07 +02:00
/* National character */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
xnstart [nN]{quote}
Code review for escape-strings patch. Sync psql and plpgsql lexers with main, avoid using a SQL-defined SQLSTATE for what is most definitely not a SQL-compatible error condition, fix documentation omissions, adhere to message style guidelines, don't use two GUC_REPORT variables when one is sufficient. Nothing done about pg_dump issues.
2005-06-26 21:16:07 +02:00
/* Quoted string that allows backslash escapes */
xestart [eE]{quote}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
xeinside [^\\']+
xeescape [\\][^0-7]
xeoctesc [\\][0-7]{1,3}
xehexesc [\\]x[0-9A-Fa-f]{1,2}
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
xeunicode [\\](u[0-9A-Fa-f]{4}|U[0-9A-Fa-f]{8})
xeunicodefail [\\](u[0-9A-Fa-f]{0,3}|U[0-9A-Fa-f]{0,7})
Code review for escape-strings patch. Sync psql and plpgsql lexers with main, avoid using a SQL-defined SQLSTATE for what is most definitely not a SQL-compatible error condition, fix documentation omissions, adhere to message style guidelines, don't use two GUC_REPORT variables when one is sufficient. Nothing done about pg_dump issues.
2005-06-26 21:16:07 +02:00
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Extended quote
Code review for escape-strings patch. Sync psql and plpgsql lexers with main, avoid using a SQL-defined SQLSTATE for what is most definitely not a SQL-compatible error condition, fix documentation omissions, adhere to message style guidelines, don't use two GUC_REPORT variables when one is sufficient. Nothing done about pg_dump issues.
2005-06-26 21:16:07 +02:00
* xqdouble implements embedded quote, ''''
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
xqstart {quote}
xqdouble {quote}{quote}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
xqinside [^']+
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
/* $foo$ style quotes ("dollar quoting")
* The quoted string starts with $foo$ where "foo" is an optional string
Remove useless whitespace at end of lines
2010-11-23 21:27:50 +01:00
* in the form of an identifier, except that it may not contain "$",
* and extends to the first occurrence of an identical string.
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
* There is *no* processing of the quoted text.
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
*
* {dolqfailed} is an error rule to avoid scanner backup when {dolqdelim}
* fails to match its trailing "$".
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
*/
dolq_start [A-Za-z\200-\377_]
dolq_cont [A-Za-z\200-\377_0-9]
dolqdelim \$({dolq_start}{dolq_cont}*)?\$
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
dolqfailed \${dolq_start}{dolq_cont}*
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
dolqinside [^$]+
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Double quote
* Allows embedded spaces and other special characters into identifiers.
*/
dquote \"
xdstart {dquote}
xdstop {dquote}
xddouble {dquote}{dquote}
xdinside [^"]+
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
/* Unicode escapes */
uescape [uU][eE][sS][cC][aA][pP][eE]{whitespace}*{quote}[^']{quote}
/* error rule to avoid backup */
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
uescapefail [uU][eE][sS][cC][aA][pP][eE]{whitespace}*"-"|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*{quote}[^']|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*{quote}|[uU][eE][sS][cC][aA][pP][eE]{whitespace}*|[uU][eE][sS][cC][aA][pP]|[uU][eE][sS][cC][aA]|[uU][eE][sS][cC]|[uU][eE][sS]|[uU][eE]|[uU]
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
/* Quoted identifier with Unicode escapes */
xuistart [uU]&{dquote}
/* Quoted string with Unicode escapes */
xusstart [uU]&{quote}
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
/* Optional UESCAPE after a quoted string or identifier with Unicode escapes. */
xustop1 {uescapefail}?
xustop2 {uescape}
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
/* error rule to avoid backup */
xufailed [uU]&
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* C-style comments
*
* The "extended comment" syntax closely resembles allowable operator syntax.
* The tricky part here is to get lex to recognize a string starting with
* slash-star as a comment, when interpreting it as an operator would produce
* a longer match --- remember lex will prefer a longer match! Also, if we
* have something like plus-slash-star, lex will think this is a 3-character
* operator whereas we want to see it as a + operator and a comment start.
* The solution is two-fold:
* 1. append {op_chars}* to xcstart so that it matches as much text as
* {operator} would. Then the tie-breaker (first matching rule of same
* length) ensures xcstart wins. We put back the extra stuff with yyless()
* in case it contains a star-slash that should terminate the comment.
* 2. In the operator rule, check for slash-star within the operator, and
* if found throw it back with yyless(). This handles the plus-slash-star
* problem.
* Dash-dash comments have similar interactions with the operator rule.
*/
xcstart \/\*{op_chars}*
xcstop \*+\/
xcinside [^*/]+
digit [0-9]
ident_start [A-Za-z\200-\377_]
ident_cont [A-Za-z\200-\377_0-9\$]
identifier {ident_start}{ident_cont}*
Make operator precedence follow the SQL standard more closely. While the SQL standard is pretty vague on the overall topic of operator precedence (because it never presents a unified BNF for all expressions), it does seem reasonable to conclude from the spec for <boolean value expression> that OR has the lowest precedence, then AND, then NOT, then IS tests, then the six standard comparison operators, then everything else (since any non-boolean operator in a WHERE clause would need to be an argument of one of these). We were only sort of on board with that: most notably, while "<" ">" and "=" had properly low precedence, "<=" ">=" and "<>" were treated as generic operators and so had significantly higher precedence. And "IS" tests were even higher precedence than those, which is very clearly wrong per spec. Another problem was that "foo NOT SOMETHING bar" constructs, such as "x NOT LIKE y", were treated inconsistently because of a bison implementation artifact: they had the documented precedence with respect to operators to their right, but behaved like NOT (i.e., very low priority) with respect to operators to their left. Fixing the precedence issues is just a small matter of rearranging the precedence declarations in gram.y, except for the NOT problem, which requires adding an additional lookahead case in base_yylex() so that we can attach a different token precedence to NOT LIKE and allied two-word operators. The bulk of this patch is not the bug fix per se, but adding logic to parse_expr.c to allow giving warnings if an expression has changed meaning because of these precedence changes. These warnings are off by default and are enabled by the new GUC operator_precedence_warning. It's believed that very few applications will be affected by these changes, but it was agreed that a warning mechanism is essential to help debug any that are.
2015-03-11 18:22:52 +01:00
/* Assorted special-case operators and operator-like tokens */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
typecast "::"
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
dot_dot \.\.
colon_equals ":="
Clean up the mess from => patch. Commit 865f14a2d31af23a05bbf2df04c274629c5d5c4d was quite a few bricks shy of a load: psql, ecpg, and plpgsql were all left out-of-step with the core lexer. Of these only the last was likely to be a fatal problem; but still, a minimal amount of grepping, or even just reading the comments adjacent to the places that were changed, would have found the other places that needed to be changed.
2015-03-10 16:48:34 +01:00
equals_greater "=>"
Make operator precedence follow the SQL standard more closely. While the SQL standard is pretty vague on the overall topic of operator precedence (because it never presents a unified BNF for all expressions), it does seem reasonable to conclude from the spec for <boolean value expression> that OR has the lowest precedence, then AND, then NOT, then IS tests, then the six standard comparison operators, then everything else (since any non-boolean operator in a WHERE clause would need to be an argument of one of these). We were only sort of on board with that: most notably, while "<" ">" and "=" had properly low precedence, "<=" ">=" and "<>" were treated as generic operators and so had significantly higher precedence. And "IS" tests were even higher precedence than those, which is very clearly wrong per spec. Another problem was that "foo NOT SOMETHING bar" constructs, such as "x NOT LIKE y", were treated inconsistently because of a bison implementation artifact: they had the documented precedence with respect to operators to their right, but behaved like NOT (i.e., very low priority) with respect to operators to their left. Fixing the precedence issues is just a small matter of rearranging the precedence declarations in gram.y, except for the NOT problem, which requires adding an additional lookahead case in base_yylex() so that we can attach a different token precedence to NOT LIKE and allied two-word operators. The bulk of this patch is not the bug fix per se, but adding logic to parse_expr.c to allow giving warnings if an expression has changed meaning because of these precedence changes. These warnings are off by default and are enabled by the new GUC operator_precedence_warning. It's believed that very few applications will be affected by these changes, but it was agreed that a warning mechanism is essential to help debug any that are.
2015-03-11 18:22:52 +01:00
less_equals "<="
greater_equals ">="
less_greater "<>"
not_equals "!="
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* "self" is the set of chars that should be returned as single-character
* tokens. "op_chars" is the set of chars that can make up "Op" tokens,
* which can be one or more characters long (but if a single-char token
* appears in the "self" set, it is not to be returned as an Op). Note
* that the sets overlap, but each has some chars that are not in the other.
*
* If you change either set, adjust the character lists appearing in the
* rule for "operator"!
*/
self [,()\[\].;\:\+\-\*\/\%\^\<\>\=]
op_chars [\~\!\@\#\^\&\|\`\?\+\-\*\/\%\<\>\=]
operator {op_chars}+
Remove useless whitespace at end of lines
2010-11-23 21:27:50 +01:00
/* we no longer allow unary minus in numbers.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
* instead we pass it separately to parser. there it gets
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
* coerced via doNegate() -- Leon aug 20 1999
*
Remove plpgsql's separate lexer (finally!), in favor of using the core lexer directly. This was a lot of trouble, but should be worth it in terms of not having to keep the plpgsql lexer in step with core anymore. In addition the handling of keywords is significantly better-structured, allowing us to de-reserve a number of words that plpgsql formerly treated as reserved.
2009-11-12 01:13:00 +01:00
* {decimalfail} is used because we would like "1..10" to lex as 1, dot_dot, 10.
*
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
* {realfail1} and {realfail2} are added to prevent the need for scanner
* backup when the {real} rule fails to match completely.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
integer {digit}+
decimal (({digit}*\.{digit}+)|({digit}+\.{digit}*))
Remove plpgsql's separate lexer (finally!), in favor of using the core lexer directly. This was a lot of trouble, but should be worth it in terms of not having to keep the plpgsql lexer in step with core anymore. In addition the handling of keywords is significantly better-structured, allowing us to de-reserve a number of words that plpgsql formerly treated as reserved.
2009-11-12 01:13:00 +01:00
decimalfail {digit}+\.\.
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
real ({integer}|{decimal})[Ee][-+]?{digit}+
realfail1 ({integer}|{decimal})[Ee]
realfail2 ({integer}|{decimal})[Ee][-+]
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
param \${integer}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
/* psql-specific: characters allowed in variable names */
variable_char [A-Za-z\200-\377_0-9]
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
other .
/*
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
* Dollar quoted strings are totally opaque, and no escaping is done on them.
* Other quoted strings must allow some special characters such as single-quote
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
* and newline.
* Embedded single-quotes are implemented both in the SQL standard
* style of two adjacent single quotes "''" and in the Postgres/Java style
* of escaped-quote "\'".
* Other embedded escaped characters are matched explicitly and the leading
* backslash is dropped from the string.
* Note that xcstart must appear before operator, as explained above!
* Also whitespace (comment) must appear before operator.
*/
%%
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
%{
/* Declare some local variables inside yylex(), for convenience */
PsqlScanState cur_state = yyextra;
PQExpBuffer output_buf = cur_state->output_buf;
/*
* Force flex into the state indicated by start_state. This has a
* couple of purposes: it lets some of the functions below set a
* new starting state without ugly direct access to flex variables,
* and it allows us to transition from one flex lexer to another
* so that we can lex different parts of the source string using
* separate lexers.
*/
BEGIN(cur_state->start_state);
%}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{whitespace} {
/*
* Note that the whitespace rule includes both true
* whitespace and single-line ("--" style) comments.
* We suppress whitespace at the start of the query
* buffer. We also suppress all single-line comments,
* which is pretty dubious but is the historical
* behavior.
*/
if (!(output_buf->len == 0 || yytext[0] == '-'))
ECHO;
}
{xcstart} {
cur_state->xcdepth = 0;
BEGIN(xc);
/* Put back any characters past slash-star; see above */
yyless(2);
ECHO;
}
<xc>{xcstart} {
cur_state->xcdepth++;
/* Put back any characters past slash-star; see above */
yyless(2);
ECHO;
}
<xc>{xcstop} {
if (cur_state->xcdepth <= 0)
{
BEGIN(INITIAL);
}
else
cur_state->xcdepth--;
ECHO;
}
<xc>{xcinside} {
ECHO;
}
<xc>{op_chars} {
ECHO;
}
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
<xc>\*+ {
ECHO;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{xbstart} {
BEGIN(xb);
ECHO;
}
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
<xb>{quotestop} |
<xb>{quotefail} {
yyless(1);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
BEGIN(INITIAL);
ECHO;
}
<xh>{xhinside} |
<xb>{xbinside} {
ECHO;
}
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
<xh>{quotecontinue} |
<xb>{quotecontinue} {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
{xhstart} {
/* Hexadecimal bit type.
* At some point we should simply pass the string
* forward to the parser and label it there.
* In the meantime, place a leading "x" on the string
* to mark it for the input routine as a hex string.
*/
BEGIN(xh);
ECHO;
}
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
<xh>{quotestop} |
<xh>{quotefail} {
yyless(1);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
BEGIN(INITIAL);
ECHO;
}
{xnstart} {
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
yyless(1); /* eat only 'n' this time */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
{xqstart} {
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
if (cur_state->std_strings)
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
BEGIN(xq);
else
BEGIN(xe);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
Code review for escape-strings patch. Sync psql and plpgsql lexers with main, avoid using a SQL-defined SQLSTATE for what is most definitely not a SQL-compatible error condition, fix documentation omissions, adhere to message style guidelines, don't use two GUC_REPORT variables when one is sufficient. Nothing done about pg_dump issues.
2005-06-26 21:16:07 +02:00
{xestart} {
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
BEGIN(xe);
Code review for escape-strings patch. Sync psql and plpgsql lexers with main, avoid using a SQL-defined SQLSTATE for what is most definitely not a SQL-compatible error condition, fix documentation omissions, adhere to message style guidelines, don't use two GUC_REPORT variables when one is sufficient. Nothing done about pg_dump issues.
2005-06-26 21:16:07 +02:00
ECHO;
}
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
{xusstart} {
BEGIN(xus);
ECHO;
}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
<xq,xe>{quotestop} |
<xq,xe>{quotefail} {
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
yyless(1);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
BEGIN(INITIAL);
ECHO;
}
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
<xus>{quotestop} |
<xus>{quotefail} {
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
yyless(1);
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
BEGIN(xusend);
ECHO;
}
<xusend>{whitespace} {
ECHO;
}
<xusend>{other} |
<xusend>{xustop1} {
yyless(0);
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
BEGIN(INITIAL);
ECHO;
}
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
<xusend>{xustop2} {
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
BEGIN(INITIAL);
ECHO;
}
<xq,xe,xus>{xqdouble} {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
<xq,xus>{xqinside} {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
<xe>{xeinside} {
ECHO;
}
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
<xe>{xeunicode} {
ECHO;
}
Fix up some oversights in psql's Unicode-escape support. Original patch failed to include new exclusive states in a switch that needed to include them; and also was guilty of very fuzzy thinking about how to handle error cases. Per bug #5729 from Alan Choi.
2010-10-27 04:23:04 +02:00
<xe>{xeunicodefail} {
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
ECHO;
}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
<xe>{xeescape} {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
<xe>{xeoctesc} {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
<xe>{xehexesc} {
Add support for \x hex strings in psql variables.
2005-06-02 03:23:48 +02:00
ECHO;
}
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
<xq,xe,xus>{quotecontinue} {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
<xe>. {
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
/* This is only needed for \ just before EOF */
ECHO;
}
{dolqdelim} {
cur_state->dolqstart = pg_strdup(yytext);
BEGIN(xdolq);
ECHO;
}
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
{dolqfailed} {
/* throw back all but the initial "$" */
yyless(1);
ECHO;
}
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
<xdolq>{dolqdelim} {
if (strcmp(yytext, cur_state->dolqstart) == 0)
{
free(cur_state->dolqstart);
cur_state->dolqstart = NULL;
BEGIN(INITIAL);
}
else
{
/*
* When we fail to match $...$ to dolqstart, transfer
* the $... part to the output, but put back the final
* $ for rescanning. Consider $delim$...$junk$delim$
*/
yyless(yyleng-1);
}
ECHO;
}
<xdolq>{dolqinside} {
ECHO;
}
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
<xdolq>{dolqfailed} {
ECHO;
}
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
<xdolq>. {
/* This is only needed for $ inside the quoted text */
ECHO;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{xdstart} {
BEGIN(xd);
ECHO;
}
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
{xuistart} {
BEGIN(xui);
ECHO;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
<xd>{xdstop} {
BEGIN(INITIAL);
ECHO;
}
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
<xui>{dquote} {
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
yyless(1);
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
BEGIN(xuiend);
ECHO;
}
<xuiend>{whitespace} {
ECHO;
}
<xuiend>{other} |
<xuiend>{xustop1} {
yyless(0);
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
BEGIN(INITIAL);
ECHO;
}
Also update psqlscan.l with the UESCAPE error rule changes. Even though this patch had no user-visible difference, better keep the code in psqlscan.l sync with the backend lexer. And of course it's nice to shrink the psql binary, too. Ecpg's version of the lexer doesn't have the error rule, it doesn't try to avoid backing up, so it doesn't need to be modified. As reminded by Tom Lane
2013-03-14 19:31:27 +01:00
<xuiend>{xustop2} {
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
BEGIN(INITIAL);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
Unicode escapes in strings and identifiers
2008-10-29 09:04:54 +01:00
<xd,xui>{xddouble} {
ECHO;
}
<xd,xui>{xdinside} {
ECHO;
}
{xufailed} {
/* throw back all but the initial u/U */
yyless(1);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
{typecast} {
ECHO;
}
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
{dot_dot} {
ECHO;
}
{colon_equals} {
ECHO;
}
Clean up the mess from => patch. Commit 865f14a2d31af23a05bbf2df04c274629c5d5c4d was quite a few bricks shy of a load: psql, ecpg, and plpgsql were all left out-of-step with the core lexer. Of these only the last was likely to be a fatal problem; but still, a minimal amount of grepping, or even just reading the comments adjacent to the places that were changed, would have found the other places that needed to be changed.
2015-03-10 16:48:34 +01:00
{equals_greater} {
ECHO;
}
Make operator precedence follow the SQL standard more closely. While the SQL standard is pretty vague on the overall topic of operator precedence (because it never presents a unified BNF for all expressions), it does seem reasonable to conclude from the spec for <boolean value expression> that OR has the lowest precedence, then AND, then NOT, then IS tests, then the six standard comparison operators, then everything else (since any non-boolean operator in a WHERE clause would need to be an argument of one of these). We were only sort of on board with that: most notably, while "<" ">" and "=" had properly low precedence, "<=" ">=" and "<>" were treated as generic operators and so had significantly higher precedence. And "IS" tests were even higher precedence than those, which is very clearly wrong per spec. Another problem was that "foo NOT SOMETHING bar" constructs, such as "x NOT LIKE y", were treated inconsistently because of a bison implementation artifact: they had the documented precedence with respect to operators to their right, but behaved like NOT (i.e., very low priority) with respect to operators to their left. Fixing the precedence issues is just a small matter of rearranging the precedence declarations in gram.y, except for the NOT problem, which requires adding an additional lookahead case in base_yylex() so that we can attach a different token precedence to NOT LIKE and allied two-word operators. The bulk of this patch is not the bug fix per se, but adding logic to parse_expr.c to allow giving warnings if an expression has changed meaning because of these precedence changes. These warnings are off by default and are enabled by the new GUC operator_precedence_warning. It's believed that very few applications will be affected by these changes, but it was agreed that a warning mechanism is essential to help debug any that are.
2015-03-11 18:22:52 +01:00
{less_equals} {
ECHO;
}
{greater_equals} {
ECHO;
}
{less_greater} {
ECHO;
}
{not_equals} {
ECHO;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* These rules are specific to psql --- they implement parenthesis
* counting and detection of command-ending semicolon. These must
* appear before the {self} rule so that they take precedence over it.
*/
"(" {
cur_state->paren_depth++;
ECHO;
}
")" {
if (cur_state->paren_depth > 0)
cur_state->paren_depth--;
ECHO;
}
";" {
ECHO;
if (cur_state->paren_depth == 0)
{
/* Terminate lexing temporarily */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
cur_state->start_state = YY_START;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
return LEXRES_SEMI;
}
}
/*
* psql-specific rules to handle backslash commands and variable
* substitution. We want these before {self}, also.
*/
"\\"[;:] {
/* Force a semicolon or colon into the query buffer */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
emit(cur_state, yytext + 1, 1);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
"\\" {
/* Terminate lexing temporarily */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
cur_state->start_state = YY_START;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
return LEXRES_BACKSLASH;
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:{variable_char}+ {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Possible psql variable substitution */
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
char *varname;
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
char *value;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
varname = extract_substring(cur_state,
yytext + 1,
yyleng - 1);
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
if (cur_state->callbacks->get_variable)
value = cur_state->callbacks->get_variable(varname,
false,
false);
else
value = NULL;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (value)
{
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
/* It is a variable, check for recursion */
if (var_is_current_source(cur_state, varname))
{
/* Recursive expansion --- don't go there */
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
cur_state->callbacks->write_error("skipping recursive expansion of variable \"%s\"\n",
varname);
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
/* Instead copy the string as is */
ECHO;
}
else
{
/* OK, perform substitution */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
push_new_buffer(cur_state, value, varname);
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
/* yy_scan_string already made buffer active */
}
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
free(value);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
else
{
/*
* if the variable doesn't exist we'll copy the
* string as is
*/
ECHO;
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
free(varname);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:'{variable_char}+' {
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
escape_variable(cur_state, yytext, yyleng, false);
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:\"{variable_char}+\" {
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
escape_variable(cur_state, yytext, yyleng, true);
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
}
Fix psql lexer to avoid use of backtracking. Per previous experimentation, backtracking slows down lexing performance significantly (by about a third). It's usually pretty easy to avoid, just need to have rules that accept an incomplete construct and do whatever the lexer would have done otherwise. The backtracking was introduced by the patch that added quoted variable substitution. Back-patch to 9.0 where that was added.
2011-08-25 20:33:08 +02:00
/*
* These rules just avoid the need for scanner backup if one of the
* two rules above fails to match completely.
*/
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:'{variable_char}* {
Fix psql lexer to avoid use of backtracking. Per previous experimentation, backtracking slows down lexing performance significantly (by about a third). It's usually pretty easy to avoid, just need to have rules that accept an incomplete construct and do whatever the lexer would have done otherwise. The backtracking was introduced by the patch that added quoted variable substitution. Back-patch to 9.0 where that was added.
2011-08-25 20:33:08 +02:00
/* Throw back everything but the colon */
yyless(1);
ECHO;
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:\"{variable_char}* {
Fix psql lexer to avoid use of backtracking. Per previous experimentation, backtracking slows down lexing performance significantly (by about a third). It's usually pretty easy to avoid, just need to have rules that accept an incomplete construct and do whatever the lexer would have done otherwise. The backtracking was introduced by the patch that added quoted variable substitution. Back-patch to 9.0 where that was added.
2011-08-25 20:33:08 +02:00
/* Throw back everything but the colon */
yyless(1);
ECHO;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* Back to backend-compatible rules.
*/
{self} {
ECHO;
}
{operator} {
/*
* Check for embedded slash-star or dash-dash; those
* are comment starts, so operator must stop there.
* Note that slash-star or dash-dash at the first
* character will match a prior rule, not this one.
*/
int nchars = yyleng;
char *slashstar = strstr(yytext, "/*");
char *dashdash = strstr(yytext, "--");
if (slashstar && dashdash)
{
/* if both appear, take the first one */
if (slashstar > dashdash)
slashstar = dashdash;
}
else if (!slashstar)
slashstar = dashdash;
if (slashstar)
nchars = slashstar - yytext;
/*
* For SQL compatibility, '+' and '-' cannot be the
* last char of a multi-char operator unless the operator
* contains chars that are not in SQL operators.
* The idea is to lex '=-' as two operators, but not
* to forbid operator names like '?-' that could not be
* sequences of SQL operators.
*/
while (nchars > 1 &&
(yytext[nchars-1] == '+' ||
yytext[nchars-1] == '-'))
{
int ic;
for (ic = nchars-2; ic >= 0; ic--)
{
if (strchr("~!@#^&|`?%", yytext[ic]))
break;
}
if (ic >= 0)
break; /* found a char that makes it OK */
nchars--; /* else remove the +/-, and check again */
}
if (nchars < yyleng)
{
/* Strip the unwanted chars from the token */
yyless(nchars);
}
ECHO;
}
{param} {
ECHO;
}
{integer} {
ECHO;
}
{decimal} {
ECHO;
}
Remove plpgsql's separate lexer (finally!), in favor of using the core lexer directly. This was a lot of trouble, but should be worth it in terms of not having to keep the plpgsql lexer in step with core anymore. In addition the handling of keywords is significantly better-structured, allowing us to de-reserve a number of words that plpgsql formerly treated as reserved.
2009-11-12 01:13:00 +01:00
{decimalfail} {
/* throw back the .., and treat as integer */
yyless(yyleng-2);
ECHO;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{real} {
ECHO;
}
Tweak the backend scanner (and psqlscan.l, which must track the backend scanner anyway) to avoid having any backup states. According to the flex manual, this should speed things up, and indeed the backend scanner is about a third faster according to some quick profiling checks. I haven't tried to measure the speed change in psql, but it probably is similar.
2005-05-26 03:24:29 +02:00
{realfail1} {
/*
* throw back the [Ee], and treat as {decimal}. Note
* that it is possible the input is actually {integer},
* but since this case will almost certainly lead to a
* syntax error anyway, we don't bother to distinguish.
*/
yyless(yyleng-1);
ECHO;
}
{realfail2} {
/* throw back the [Ee][+-], and proceed as above */
yyless(yyleng-2);
ECHO;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{identifier} {
ECHO;
}
{other} {
ECHO;
}
/*
* Everything from here down is psql-specific.
*/
<<EOF>> {
StackElem *stackelem = cur_state->buffer_stack;
if (stackelem == NULL)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
{
cur_state->start_state = YY_START;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
return LEXRES_EOL; /* end of input reached */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* We were expanding a variable, so pop the inclusion
* stack and keep lexing
*/
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
pop_buffer_stack(cur_state);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
stackelem = cur_state->buffer_stack;
if (stackelem != NULL)
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(stackelem->buf, cur_state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
cur_state->curline = stackelem->bufstring;
cur_state->refline = stackelem->origstring ? stackelem->origstring : stackelem->bufstring;
}
else
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(cur_state->scanbufhandle, cur_state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
cur_state->curline = cur_state->scanbuf;
cur_state->refline = cur_state->scanline;
}
}
/*
* Exclusive lexer states to handle backslash command lexing
*/
<xslashcmd>{
/* command name ends at whitespace or backslash; eat all else */
{space}|"\\" {
yyless(0);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
cur_state->start_state = YY_START;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
return LEXRES_OK;
}
{other} { ECHO; }
}
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
<xslashargstart>{
/*
* Discard any whitespace before argument, then go to xslasharg state.
* An exception is that "|" is only special at start of argument, so we
* check for it here.
*/
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{space}+ { }
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
"|" {
if (option_type == OT_FILEPIPE)
{
/* treat like whole-string case */
ECHO;
BEGIN(xslashwholeline);
}
else
{
/* vertical bar is not special otherwise */
yyless(0);
BEGIN(xslasharg);
}
}
{other} {
yyless(0);
BEGIN(xslasharg);
}
}
<xslasharg>{
/*
* Default processing of text in a slash command's argument.
*
* Note: unquoted_option_chars counts the number of characters at the
* end of the argument that were not subject to any form of quoting.
* psql_scan_slash_option needs this to strip trailing semicolons safely.
*/
{space}|"\\" {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
* Unquoted space is end of arg; do not eat. Likewise
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
* backslash is end of command or next command, do not eat
*
* XXX this means we can't conveniently accept options
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
* that include unquoted backslashes; therefore, option
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
* processing that encourages use of backslashes is rather
* broken.
*/
yyless(0);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
cur_state->start_state = YY_START;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
return LEXRES_OK;
}
{quote} {
*option_quote = '\'';
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
unquoted_option_chars = 0;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
BEGIN(xslashquote);
}
"`" {
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
backtick_start_offset = output_buf->len;
*option_quote = '`';
unquoted_option_chars = 0;
BEGIN(xslashbackquote);
}
{dquote} {
ECHO;
*option_quote = '"';
unquoted_option_chars = 0;
BEGIN(xslashdquote);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:{variable_char}+ {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Possible psql variable substitution */
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
if (option_type == OT_NO_EVAL ||
cur_state->callbacks->get_variable == NULL)
Prevent evaluation of backticks while discarding unwanted arguments after an unknown or failed psql backslash command, and also while discarding "extra" arguments of a putatively valid backslash command. In the case of an unknown/failed command, make sure we discard the whole rest of the line, rather than trying to resume at the next backslash. Per discussion with Thomer Gil.
2004-12-19 20:39:47 +01:00
ECHO;
else
{
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
char *varname;
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
char *value;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
varname = extract_substring(cur_state,
yytext + 1,
yyleng - 1);
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
value = cur_state->callbacks->get_variable(varname,
false,
false);
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
free(varname);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Prevent evaluation of backticks while discarding unwanted arguments after an unknown or failed psql backslash command, and also while discarding "extra" arguments of a putatively valid backslash command. In the case of an unknown/failed command, make sure we discard the whole rest of the line, rather than trying to resume at the next backslash. Per discussion with Thomer Gil.
2004-12-19 20:39:47 +01:00
/*
* The variable value is just emitted without any
* further examination. This is consistent with the
* pre-8.0 code behavior, if not with the way that
* variables are handled outside backslash commands.
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
* Note that we needn't guard against recursion here.
Prevent evaluation of backticks while discarding unwanted arguments after an unknown or failed psql backslash command, and also while discarding "extra" arguments of a putatively valid backslash command. In the case of an unknown/failed command, make sure we discard the whole rest of the line, rather than trying to resume at the next backslash. Per discussion with Thomer Gil.
2004-12-19 20:39:47 +01:00
*/
if (value)
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
{
Prevent evaluation of backticks while discarding unwanted arguments after an unknown or failed psql backslash command, and also while discarding "extra" arguments of a putatively valid backslash command. In the case of an unknown/failed command, make sure we discard the whole rest of the line, rather than trying to resume at the next backslash. Per discussion with Thomer Gil.
2004-12-19 20:39:47 +01:00
appendPQExpBufferStr(output_buf, value);
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
free(value);
}
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
else
ECHO;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
*option_quote = ':';
}
unquoted_option_chars = 0;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:'{variable_char}+' {
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
if (option_type == OT_NO_EVAL)
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
ECHO;
else
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
escape_variable(cur_state, yytext, yyleng, false);
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
*option_quote = ':';
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
}
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
unquoted_option_chars = 0;
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:\"{variable_char}+\" {
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
if (option_type == OT_NO_EVAL)
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
ECHO;
else
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
escape_variable(cur_state, yytext, yyleng, true);
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
*option_quote = ':';
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
}
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
unquoted_option_chars = 0;
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:'{variable_char}* {
Fix psql lexer to avoid use of backtracking. Per previous experimentation, backtracking slows down lexing performance significantly (by about a third). It's usually pretty easy to avoid, just need to have rules that accept an incomplete construct and do whatever the lexer would have done otherwise. The backtracking was introduced by the patch that added quoted variable substitution. Back-patch to 9.0 where that was added.
2011-08-25 20:33:08 +02:00
/* Throw back everything but the colon */
yyless(1);
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
unquoted_option_chars++;
Fix psql lexer to avoid use of backtracking. Per previous experimentation, backtracking slows down lexing performance significantly (by about a third). It's usually pretty easy to avoid, just need to have rules that accept an incomplete construct and do whatever the lexer would have done otherwise. The backtracking was introduced by the patch that added quoted variable substitution. Back-patch to 9.0 where that was added.
2011-08-25 20:33:08 +02:00
ECHO;
}
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
:\"{variable_char}* {
Fix psql lexer to avoid use of backtracking. Per previous experimentation, backtracking slows down lexing performance significantly (by about a third). It's usually pretty easy to avoid, just need to have rules that accept an incomplete construct and do whatever the lexer would have done otherwise. The backtracking was introduced by the patch that added quoted variable substitution. Back-patch to 9.0 where that was added.
2011-08-25 20:33:08 +02:00
/* Throw back everything but the colon */
yyless(1);
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
unquoted_option_chars++;
Fix psql lexer to avoid use of backtracking. Per previous experimentation, backtracking slows down lexing performance significantly (by about a third). It's usually pretty easy to avoid, just need to have rules that accept an incomplete construct and do whatever the lexer would have done otherwise. The backtracking was introduced by the patch that added quoted variable substitution. Back-patch to 9.0 where that was added.
2011-08-25 20:33:08 +02:00
ECHO;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
{other} {
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
unquoted_option_chars++;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
ECHO;
}
}
<xslashquote>{
Fix seriously broken patch for psql '' ... per its comment, emit() is NOT the thing to use here.
2006-06-01 00:11:44 +02:00
/*
* single-quoted text: copy literally except for '' and backslash
* sequences
*/
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
{quote} { BEGIN(xslasharg); }
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Fix seriously broken patch for psql '' ... per its comment, emit() is NOT the thing to use here.
2006-06-01 00:11:44 +02:00
{xqdouble} { appendPQExpBufferChar(output_buf, '\''); }
Support '' for literal ' in psql single-quote strings, documentation update.
2006-05-31 13:35:17 +02:00
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
"\\n" { appendPQExpBufferChar(output_buf, '\n'); }
"\\t" { appendPQExpBufferChar(output_buf, '\t'); }
"\\b" { appendPQExpBufferChar(output_buf, '\b'); }
"\\r" { appendPQExpBufferChar(output_buf, '\r'); }
"\\f" { appendPQExpBufferChar(output_buf, '\f'); }
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
{xeoctesc} {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* octal case */
appendPQExpBufferChar(output_buf,
Have psql escape bytes in strings for variables follow the backend conventions of only allowing octal, like \045. Remove support for \decimal, \0octal, and \0xhex which matches the strtol() function but didn't make sense with backslashes. These now return the same character: test=> \set x '\54' test=> \echo :x , test=> \set x '\054' test=> \echo :x , THIS IS A BACKWARD COMPATIBILITY CHANGE.
2005-05-30 16:50:35 +02:00
(char) strtol(yytext + 1, NULL, 8));
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
{xehexesc} {
Add support for \x hex strings in psql variables.
2005-06-02 03:23:48 +02:00
/* hex case */
appendPQExpBufferChar(output_buf,
(char) strtol(yytext + 2, NULL, 16));
}
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
"\\". { emit(cur_state, yytext + 1, 1); }
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
{other}|\n { ECHO; }
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
<xslashbackquote>{
/*
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
* backticked text: copy everything until next backquote, then evaluate.
*
* XXX Possible future behavioral change: substitute for :VARIABLE?
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
"`" {
/* In NO_EVAL mode, don't evaluate the command */
if (option_type != OT_NO_EVAL)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
evaluate_backtick(cur_state);
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
BEGIN(xslasharg);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
{other}|\n { ECHO; }
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
<xslashdquote>{
/* double-quoted text: copy verbatim, including the double quotes */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{dquote} {
ECHO;
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
BEGIN(xslasharg);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
{other}|\n { ECHO; }
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
<xslashwholeline>{
/* copy everything until end of input line */
/* but suppress leading whitespace */
{space}+ {
if (output_buf->len > 0)
ECHO;
}
{other} { ECHO; }
}
<xslashend>{
/* at end of command, eat a double backslash, but not anything else */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
"\\\\" {
cur_state->start_state = YY_START;
return LEXRES_OK;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
{other}|\n {
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
yyless(0);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
cur_state->start_state = YY_START;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
return LEXRES_OK;
}
}
%%
/*
* Create a lexer working state struct.
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
*
* callbacks is a struct of function pointers that encapsulate some
* behavior we need from the surrounding program. This struct must
* remain valid for the lifespan of the PsqlScanState.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
PsqlScanState
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
psql_scan_create(const PsqlScanCallbacks *callbacks)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{
PsqlScanState state;
Standardize naming of malloc/realloc/strdup wrapper functions. We had a number of variants on the theme of "malloc or die", with the majority named like "pg_malloc", but by no means all. Standardize on the names pg_malloc, pg_malloc0, pg_realloc, pg_strdup. Get rid of pg_calloc entirely in favor of using pg_malloc0. This is an essentially cosmetic change, so no back-patch. (I did find a couple of places where psql and pg_dump were using plain malloc or strdup instead of the pg_ versions, but they don't look significant enough to bother back-patching.)
2012-10-02 21:35:10 +02:00
state = (PsqlScanStateData *) pg_malloc0(sizeof(PsqlScanStateData));
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
state->callbacks = callbacks;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yylex_init_extra(state, &state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
psql_scan_reset(state);
return state;
}
/*
* Destroy a lexer working state struct, releasing all resources.
*/
void
psql_scan_destroy(PsqlScanState state)
{
psql_scan_finish(state);
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
psql_scan_reset(state);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yylex_destroy(state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
free(state);
}
/*
* Set up to perform lexing of the given input line.
*
* The text at *line, extending for line_len bytes, will be scanned by
* subsequent calls to the psql_scan routines. psql_scan_finish should
* be called when scanning is complete. Note that the lexer retains
* a pointer to the storage at *line --- this string must not be altered
* or freed until after psql_scan_finish is called.
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
*
* encoding is the libpq identifier for the character encoding in use,
* and std_strings says whether standard_conforming_strings is on.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
void
psql_scan_setup(PsqlScanState state,
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
const char *line, int line_len,
int encoding, bool std_strings)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{
/* Mustn't be scanning already */
Provide Assert() for frontend code. Per discussion on-hackers. psql is converted to use the new code. Follows a suggestion from Heikki Linnakangas.
2012-12-15 00:03:07 +01:00
Assert(state->scanbufhandle == NULL);
Assert(state->buffer_stack == NULL);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Do we need to hack the character set encoding? */
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
state->encoding = encoding;
state->safe_encoding = pg_valid_server_encoding_id(encoding);
/* Save standard-strings flag as well */
state->std_strings = std_strings;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Set up flex input buffer with appropriate translation and padding */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->scanbufhandle = prepare_buffer(state, line, line_len,
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
&state->scanbuf);
state->scanline = line;
/* Set lookaside data in case we have to map unsafe encoding */
state->curline = state->scanbuf;
state->refline = state->scanline;
}
/*
* Do lexical analysis of SQL command text.
*
* The text previously passed to psql_scan_setup is scanned, and appended
* (possibly with transformation) to query_buf.
*
* The return value indicates the condition that stopped scanning:
*
* PSCAN_SEMICOLON: found a command-ending semicolon. (The semicolon is
* transferred to query_buf.) The command accumulated in query_buf should
* be executed, then clear query_buf and call again to scan the remainder
* of the line.
*
* PSCAN_BACKSLASH: found a backslash that starts a psql special command.
* Any previous data on the line has been transferred to query_buf.
* The caller will typically next call psql_scan_slash_command(),
* perhaps psql_scan_slash_option(), and psql_scan_slash_command_end().
*
* PSCAN_INCOMPLETE: the end of the line was reached, but we have an
* incomplete SQL command. *prompt is set to the appropriate prompt type.
*
* PSCAN_EOL: the end of the line was reached, and there is no lexical
* reason to consider the command incomplete. The caller may or may not
* choose to send it. *prompt is set to the appropriate prompt type if
* the caller chooses to collect more input.
*
* In the PSCAN_INCOMPLETE and PSCAN_EOL cases, psql_scan_finish() should
* be called next, then the cycle may be repeated with a fresh input line.
*
* In all cases, *prompt is set to an appropriate prompt type code for the
* next line-input operation.
*/
PsqlScanResult
psql_scan(PsqlScanState state,
PQExpBuffer query_buf,
promptStatus_t *prompt)
{
PsqlScanResult result;
int lexresult;
/* Must be scanning already */
Provide Assert() for frontend code. Per discussion on-hackers. psql is converted to use the new code. Follows a suggestion from Heikki Linnakangas.
2012-12-15 00:03:07 +01:00
Assert(state->scanbufhandle != NULL);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set current output target */
state->output_buf = query_buf;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set input source */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (state->buffer_stack != NULL)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->buffer_stack->buf, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
else
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->scanbufhandle, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* And lex. */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
lexresult = yylex(state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* Check termination state and return appropriate result info.
*/
switch (lexresult)
{
case LEXRES_EOL: /* end of input */
switch (state->start_state)
{
Fix up some oversights in psql's Unicode-escape support. Original patch failed to include new exclusive states in a switch that needed to include them; and also was guilty of very fuzzy thinking about how to handle error cases. Per bug #5729 from Alan Choi.
2010-10-27 04:23:04 +02:00
/* This switch must cover all non-slash-command states. */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
case INITIAL:
Switch in psql_scan() must cover all lexer states (except backslash cases). Oversight in commit f7559c0101afa33bfb4e104036ca46adac900111, which changed UESCAPE lexing in psql. Per bug #9068 from Manuel Gómez.
2014-02-03 00:59:34 +01:00
case xuiend: /* we treat these like INITIAL */
case xusend:
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (state->paren_depth > 0)
{
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_PAREN;
}
else if (query_buf->len > 0)
{
result = PSCAN_EOL;
*prompt = PROMPT_CONTINUE;
}
else
{
/* never bother to send an empty buffer */
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_READY;
}
break;
case xb:
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_SINGLEQUOTE;
break;
case xc:
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_COMMENT;
break;
case xd:
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_DOUBLEQUOTE;
break;
case xh:
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_SINGLEQUOTE;
break;
Fix up some oversights in psql's Unicode-escape support. Original patch failed to include new exclusive states in a switch that needed to include them; and also was guilty of very fuzzy thinking about how to handle error cases. Per bug #5729 from Alan Choi.
2010-10-27 04:23:04 +02:00
case xe:
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_SINGLEQUOTE;
break;
Fix up some oversights in psql's Unicode-escape support. Original patch failed to include new exclusive states in a switch that needed to include them; and also was guilty of very fuzzy thinking about how to handle error cases. Per bug #5729 from Alan Choi.
2010-10-27 04:23:04 +02:00
case xq:
Enable standard_conforming_strings to be turned on. Kevin Grittner
2006-03-06 20:49:20 +01:00
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_SINGLEQUOTE;
break;
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
case xdolq:
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_DOLLARQUOTE;
break;
Fix up some oversights in psql's Unicode-escape support. Original patch failed to include new exclusive states in a switch that needed to include them; and also was guilty of very fuzzy thinking about how to handle error cases. Per bug #5729 from Alan Choi.
2010-10-27 04:23:04 +02:00
case xui:
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_DOUBLEQUOTE;
break;
case xus:
result = PSCAN_INCOMPLETE;
*prompt = PROMPT_SINGLEQUOTE;
break;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
default:
/* can't get here */
fprintf(stderr, "invalid YY_START\n");
exit(1);
}
break;
case LEXRES_SEMI: /* semicolon */
result = PSCAN_SEMICOLON;
*prompt = PROMPT_READY;
break;
case LEXRES_BACKSLASH: /* backslash */
result = PSCAN_BACKSLASH;
*prompt = PROMPT_READY;
break;
default:
/* can't get here */
fprintf(stderr, "invalid yylex result\n");
exit(1);
}
return result;
}
/*
* Clean up after scanning a string. This flushes any unread input and
* releases resources (but not the PsqlScanState itself). Note however
* that this does not reset the lexer scan state; that can be done by
* psql_scan_reset(), which is an orthogonal operation.
*
* It is legal to call this when not scanning anything (makes it easier
* to deal with error recovery).
*/
void
psql_scan_finish(PsqlScanState state)
{
/* Drop any incomplete variable expansions. */
while (state->buffer_stack != NULL)
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
pop_buffer_stack(state);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Done with the outer scan buffer, too */
if (state->scanbufhandle)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_delete_buffer(state->scanbufhandle, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
state->scanbufhandle = NULL;
if (state->scanbuf)
free(state->scanbuf);
state->scanbuf = NULL;
}
/*
* Reset lexer scanning state to start conditions. This is appropriate
* for executing \r psql commands (or any other time that we discard the
* prior contents of query_buf). It is not, however, necessary to do this
* when we execute and clear the buffer after getting a PSCAN_SEMICOLON or
* PSCAN_EOL scan result, because the scan state must be INITIAL when those
* conditions are returned.
*
* Note that this is unrelated to flushing unread input; that task is
* done by psql_scan_finish().
*/
void
psql_scan_reset(PsqlScanState state)
{
state->start_state = INITIAL;
state->paren_depth = 0;
state->xcdepth = 0; /* not really necessary */
Implement dollar-quoting in the backend lexer and psql. Documentation is still lacking, as is support in plpgsql and other places, but this is the basic feature. Patch by Andrew Dunstan, some tweaking by Tom Lane. Also, enable %option nodefault in these two lexers, and patch some gaps revealed thereby.
2004-02-24 22:45:18 +01:00
if (state->dolqstart)
free(state->dolqstart);
state->dolqstart = NULL;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
/*
* Return true if lexer is currently in an "inside quotes" state.
*
* This is pretty grotty but is needed to preserve the old behavior
* that mainloop.c drops blank lines not inside quotes without even
* echoing them.
*/
bool
psql_scan_in_quote(PsqlScanState state)
{
return state->start_state != INITIAL;
}
/*
* Scan the command name of a psql backslash command. This should be called
* after psql_scan() returns PSCAN_BACKSLASH. It is assumed that the input
* has been consumed through the leading backslash.
*
* The return value is a malloc'd copy of the command name, as parsed off
* from the input.
*/
char *
psql_scan_slash_command(PsqlScanState state)
{
PQExpBufferData mybuf;
/* Must be scanning already */
Provide Assert() for frontend code. Per discussion on-hackers. psql is converted to use the new code. Follows a suggestion from Heikki Linnakangas.
2012-12-15 00:03:07 +01:00
Assert(state->scanbufhandle != NULL);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* Build a local buffer that we'll return the data of */
initPQExpBuffer(&mybuf);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set current output target */
state->output_buf = &mybuf;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set input source */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (state->buffer_stack != NULL)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->buffer_stack->buf, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
else
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->scanbufhandle, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set lexer start state */
state->start_state = xslashcmd;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* And lex. */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yylex(state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* There are no possible errors in this lex state... */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Reset lexer state in case it's time to return to regular parsing */
state->start_state = INITIAL;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
return mybuf.data;
}
/*
* Parse off the next argument for a backslash command, and return it as a
* malloc'd string. If there are no more arguments, returns NULL.
*
* type tells what processing, if any, to perform on the option string;
* for example, if it's a SQL identifier, we want to downcase any unquoted
* letters.
*
* if quote is not NULL, *quote is set to 0 if no quoting was found, else
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
* the last quote symbol used in the argument.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*
* if semicolon is true, unquoted trailing semicolon(s) that would otherwise
* be taken as part of the option string will be stripped.
*
* NOTE: the only possible syntax errors for backslash options are unmatched
* quotes, which are detected when we run out of input. Therefore, on a
* syntax error we just throw away the string and return NULL; there is no
* need to worry about flushing remaining input.
*/
char *
psql_scan_slash_option(PsqlScanState state,
enum slash_option_type type,
char *quote,
bool semicolon)
{
PQExpBufferData mybuf;
Clean up compiler warnings from unused variables with asserts disabled For those variables only used when asserts are enabled, use a new macro PG_USED_FOR_ASSERTS_ONLY, which expands to __attribute__((unused)) when asserts are not enabled.
2012-03-21 22:30:14 +01:00
int lexresult PG_USED_FOR_ASSERTS_ONLY;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
int final_state;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
char local_quote;
/* Must be scanning already */
Provide Assert() for frontend code. Per discussion on-hackers. psql is converted to use the new code. Follows a suggestion from Heikki Linnakangas.
2012-12-15 00:03:07 +01:00
Assert(state->scanbufhandle != NULL);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (quote == NULL)
quote = &local_quote;
*quote = 0;
/* Build a local buffer that we'll return the data of */
initPQExpBuffer(&mybuf);
/* Set up static variables that will be used by yylex */
option_type = type;
option_quote = quote;
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
unquoted_option_chars = 0;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set current output target */
state->output_buf = &mybuf;
/* Set input source */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (state->buffer_stack != NULL)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->buffer_stack->buf, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
else
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->scanbufhandle, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set lexer start state */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (type == OT_WHOLE_LINE)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->start_state = xslashwholeline;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
else
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->start_state = xslashargstart;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* And lex. */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
lexresult = yylex(state->scanner);
/* Reset lexer state in case it's time to return to regular parsing */
final_state = state->start_state;
state->start_state = INITIAL;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* Check the lex result: we should have gotten back either LEXRES_OK
* or LEXRES_EOL (the latter indicating end of string). If we were inside
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
* a quoted string, as indicated by final_state, EOL is an error.
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
*/
Provide Assert() for frontend code. Per discussion on-hackers. psql is converted to use the new code. Follows a suggestion from Heikki Linnakangas.
2012-12-15 00:03:07 +01:00
Assert(lexresult == LEXRES_EOL || lexresult == LEXRES_OK);
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
switch (final_state)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
case xslashargstart:
/* empty arg */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
break;
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
case xslasharg:
/* Strip any unquoted trailing semi-colons if requested */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (semicolon)
{
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
while (unquoted_option_chars-- > 0 &&
mybuf.len > 0 &&
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
mybuf.data[mybuf.len - 1] == ';')
{
mybuf.data[--mybuf.len] = '\0';
}
}
/*
* If SQL identifier processing was requested, then we strip out
* excess double quotes and downcase unquoted letters.
* Doubled double-quotes become output double-quotes, per spec.
*
* Note that a string like FOO"BAR"BAZ will be converted to
* fooBARbaz; this is somewhat inconsistent with the SQL spec,
* which would have us parse it as several identifiers. But
* for psql's purposes, we want a string like "foo"."bar" to
* be treated as one option, so there's little choice.
*/
if (type == OT_SQLID || type == OT_SQLIDHACK)
{
bool inquotes = false;
char *cp = mybuf.data;
while (*cp)
{
if (*cp == '"')
{
if (inquotes && cp[1] == '"')
{
/* Keep the first quote, remove the second */
cp++;
}
inquotes = !inquotes;
/* Collapse out quote at *cp */
memmove(cp, cp + 1, strlen(cp));
mybuf.len--;
/* do not advance cp */
}
else
{
if (!inquotes && type == OT_SQLID)
Solve the 'Turkish problem' with undesirable locale behavior for case conversion of basic ASCII letters. Remove all uses of strcasecmp and strncasecmp in favor of new functions pg_strcasecmp and pg_strncasecmp; remove most but not all direct uses of toupper and tolower in favor of pg_toupper and pg_tolower. These functions use the same notions of case folding already developed for identifier case conversion. I left the straight locale-based folding in place for situations where we are just manipulating user data and not trying to match it to built-in strings --- for example, the SQL upper() function is still locale dependent. Perhaps this will prove not to be what's wanted, but at the moment we can initdb and pass regression tests in Turkish locale.
2004-05-07 02:24:59 +02:00
*cp = pg_tolower((unsigned char) *cp);
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
cp += PQmblen(cp, state->encoding);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
}
}
break;
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
case xslashquote:
case xslashbackquote:
case xslashdquote:
/* must have hit EOL inside quotes */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->callbacks->write_error("unterminated quoted string\n");
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
termPQExpBuffer(&mybuf);
return NULL;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
case xslashwholeline:
/* always okay */
break;
default:
/* can't get here */
fprintf(stderr, "invalid YY_START\n");
exit(1);
}
/*
* An unquoted empty argument isn't possible unless we are at end of
* command. Return NULL instead.
*/
if (mybuf.len == 0 && *quote == 0)
{
termPQExpBuffer(&mybuf);
return NULL;
}
/* Else return the completed string. */
return mybuf.data;
}
/*
* Eat up any unused \\ to complete a backslash command.
*/
void
psql_scan_slash_command_end(PsqlScanState state)
{
/* Must be scanning already */
Provide Assert() for frontend code. Per discussion on-hackers. psql is converted to use the new code. Follows a suggestion from Heikki Linnakangas.
2012-12-15 00:03:07 +01:00
Assert(state->scanbufhandle != NULL);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set current output target */
state->output_buf = NULL; /* we won't output anything */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set input source */
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
if (state->buffer_stack != NULL)
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->buffer_stack->buf, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
else
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_switch_to_buffer(state->scanbufhandle, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Set lexer start state */
state->start_state = xslashend;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* And lex. */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yylex(state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* There are no possible errors in this lex state... */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
/* Reset lexer state in case it's time to return to regular parsing */
state->start_state = INITIAL;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
/*
* Evaluate a backticked substring of a slash command's argument.
*
* The portion of output_buf starting at backtick_start_offset is evaluated
* as a shell command and then replaced by the command's output.
*/
static void
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
evaluate_backtick(PsqlScanState state)
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
PQExpBuffer output_buf = state->output_buf;
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
char *cmd = output_buf->data + backtick_start_offset;
PQExpBufferData cmd_output;
FILE *fd;
bool error = false;
char buf[512];
size_t result;
initPQExpBuffer(&cmd_output);
fd = popen(cmd, PG_BINARY_R);
if (!fd)
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->callbacks->write_error("%s: %s\n", cmd, strerror(errno));
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
error = true;
}
if (!error)
{
do
{
result = fread(buf, 1, sizeof(buf), fd);
if (ferror(fd))
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->callbacks->write_error("%s: %s\n", cmd, strerror(errno));
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
error = true;
break;
}
appendBinaryPQExpBuffer(&cmd_output, buf, result);
} while (!feof(fd));
}
if (fd && pclose(fd) == -1)
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->callbacks->write_error("%s: %s\n", cmd, strerror(errno));
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
error = true;
}
Suppress remaining -Waddress warnings from recent gcc versions. Still an exercise in satisfying pedants.
2011-10-19 03:44:23 +02:00
if (PQExpBufferDataBroken(cmd_output))
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->callbacks->write_error("%s: out of memory\n", cmd);
Clean up weird corner cases in lexing of psql meta-command arguments. These changes allow backtick command evaluation and psql variable interpolation to happen on substrings of a single meta-command argument. Formerly, no such evaluations happened at all if the backtick or colon wasn't the first character of the argument, and we considered an argument completed as soon as we'd processed one backtick, variable reference, or quoted substring. A string like 'FOO'BAR was thus taken as two arguments not one, not exactly what one would expect. In the new coding, an argument is considered terminated only by unquoted whitespace or backslash. Also, clean up a bunch of omissions, infelicities and outright errors in the psql documentation of variables and metacommand argument syntax.
2011-08-26 19:52:23 +02:00
error = true;
}
/* Now done with cmd, delete it from output_buf */
output_buf->len = backtick_start_offset;
output_buf->data[output_buf->len] = '\0';
/* If no error, transfer result to output_buf */
if (!error)
{
/* strip any trailing newline */
if (cmd_output.len > 0 &&
cmd_output.data[cmd_output.len - 1] == '\n')
cmd_output.len--;
appendBinaryPQExpBuffer(output_buf, cmd_output.data, cmd_output.len);
}
termPQExpBuffer(&cmd_output);
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* Push the given string onto the stack of stuff to scan.
*
* NOTE SIDE EFFECT: the new buffer is made the active flex input buffer.
*/
static void
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
push_new_buffer(PsqlScanState state, const char *newstr, const char *varname)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{
StackElem *stackelem;
stackelem = (StackElem *) pg_malloc(sizeof(StackElem));
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
/*
* In current usage, the passed varname points at the current flex
* input buffer; we must copy it before calling prepare_buffer()
* because that will change the buffer state.
*/
stackelem->varname = varname ? pg_strdup(varname) : NULL;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
stackelem->buf = prepare_buffer(state, newstr, strlen(newstr),
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
&stackelem->bufstring);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->curline = stackelem->bufstring;
if (state->safe_encoding)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{
stackelem->origstring = NULL;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->refline = stackelem->bufstring;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
else
{
stackelem->origstring = pg_strdup(newstr);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
state->refline = stackelem->origstring;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
stackelem->next = state->buffer_stack;
state->buffer_stack = stackelem;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
/*
* Pop the topmost buffer stack item (there must be one!)
*
* NB: after this, the flex input state is unspecified; caller must
* switch to an appropriate buffer to continue lexing.
*/
static void
pop_buffer_stack(PsqlScanState state)
{
StackElem *stackelem = state->buffer_stack;
state->buffer_stack = stackelem->next;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
yy_delete_buffer(stackelem->buf, state->scanner);
Fix psql to not go into infinite recursion when expanding a variable that refers to itself (directly or indirectly). Instead, print a message when recursion is detected, and don't expand the repeated reference. Per bug #5448 from Francis Markham. Back-patch to 8.0. Although the issue exists in 7.4 as well, it seems impractical to fix there because of the lack of any state stack that could be used to track active expansions.
2010-05-06 00:18:56 +02:00
free(stackelem->bufstring);
if (stackelem->origstring)
free(stackelem->origstring);
if (stackelem->varname)
free(stackelem->varname);
free(stackelem);
}
/*
* Check if specified variable name is the source for any string
* currently being scanned
*/
static bool
var_is_current_source(PsqlScanState state, const char *varname)
{
StackElem *stackelem;
for (stackelem = state->buffer_stack;
stackelem != NULL;
stackelem = stackelem->next)
{
if (stackelem->varname && strcmp(stackelem->varname, varname) == 0)
return true;
}
return false;
}
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/*
* Set up a flex input buffer to scan the given data. We always make a
* copy of the data. If working in an unsafe encoding, the copy has
* multibyte sequences replaced by FFs to avoid fooling the lexer rules.
*
* NOTE SIDE EFFECT: the new buffer is made the active flex input buffer.
*/
static YY_BUFFER_STATE
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
prepare_buffer(PsqlScanState state, const char *txt, int len, char **txtcopy)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{
char *newtxt;
/* Flex wants two \0 characters after the actual data */
newtxt = pg_malloc(len + 2);
*txtcopy = newtxt;
newtxt[len] = newtxt[len + 1] = YY_END_OF_BUFFER_CHAR;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
if (state->safe_encoding)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
memcpy(newtxt, txt, len);
else
{
/* Gotta do it the hard way */
int i = 0;
while (i < len)
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
int thislen = PQmblen(txt + i, state->encoding);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
/* first byte should always be okay... */
newtxt[i] = txt[i];
i++;
Fix psql crash while parsing SQL file whose encoding is different from client encoding and the client encoding is not *safe* one. Such an example is, file encoding is UTF-8 and client encoding SJIS. Patch contributed by Jiang Guiqing.
2012-12-02 13:11:15 +01:00
while (--thislen > 0 && i < len)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
newtxt[i++] = (char) 0xFF;
}
}
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
return yy_scan_buffer(newtxt, len + 2, state->scanner);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
}
/*
* emit() --- body for ECHO macro
*
* NB: this must be used for ALL and ONLY the text copied from the flex
* input data. If you pass it something that is not part of the yytext
* string, you are making a mistake. Internally generated text can be
* appended directly to output_buf.
*/
static void
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
emit(PsqlScanState state, const char *txt, int len)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
{
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
PQExpBuffer output_buf = state->output_buf;
if (state->safe_encoding)
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
appendBinaryPQExpBuffer(output_buf, txt, len);
else
{
/* Gotta do it the hard way */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
const char *reference = state->refline;
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
int i;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
reference += (txt - state->curline);
Re-implement psql's input scanning to use a flex-generated lexer, as per recent discussion. The lexer is used for both SQL command text and backslash commands. The purpose of this change is to make it easier to track the behavior of the backend's SQL lexer --- essentially identical flex rules are now used by psql. Also, this cleans up a lot of very squirrelly code in mainloop.c and command.c. The flex code is somewhat bulkier than the removed code, but should be lots easier to maintain.
2004-02-19 20:40:09 +01:00
for (i = 0; i < len; i++)
{
char ch = txt[i];
if (ch == (char) 0xFF)
ch = reference[i];
appendPQExpBufferChar(output_buf, ch);
}
}
}
Sync psql's scanner with recent changes in backend scanner's flex rules. Marko Kreen, Tom Lane
2009-09-27 05:27:24 +02:00
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
/*
* extract_substring --- fetch the true value of (part of) the current token
*
* This is like emit(), except that the data is returned as a malloc'd string
* rather than being pushed directly to output_buf.
*/
static char *
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
extract_substring(PsqlScanState state, const char *txt, int len)
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
{
char *result = (char *) pg_malloc(len + 1);
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
if (state->safe_encoding)
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
memcpy(result, txt, len);
else
{
/* Gotta do it the hard way */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
const char *reference = state->refline;
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
int i;
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
reference += (txt - state->curline);
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
for (i = 0; i < len; i++)
{
char ch = txt[i];
if (ch == (char) 0xFF)
ch = reference[i];
result[i] = ch;
}
}
result[len] = '\0';
return result;
}
/*
* escape_variable --- process :'VARIABLE' or :"VARIABLE"
*
* If the variable name is found, escape its value using the appropriate
* quoting method and emit the value to output_buf. (Since the result is
* surely quoted, there is never any reason to rescan it.) If we don't
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
* find the variable or escaping fails, emit the token as-is.
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
*/
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
static void
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
escape_variable(PsqlScanState state, const char *txt, int len, bool as_ident)
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
{
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
char *varname;
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
char *value;
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
/* Variable lookup. */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
varname = extract_substring(state, txt + 2, len - 3);
if (state->callbacks->get_variable)
value = state->callbacks->get_variable(varname, true, as_ident);
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
else
value = NULL;
Support non-ASCII letters in psql variable names. As in the backend, the implementation actually accepts any non-ASCII character, but we only document that you can use letters.
2011-08-26 16:41:31 +02:00
free(varname);
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
if (value)
{
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
/* Emit the suitably-escaped value */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
appendPQExpBufferStr(state->output_buf, value);
Decouple psqlscan.l from surrounding program. Remove assorted external references from psqlscan.l in preparation for making it usable by other frontend programs. This mostly involves getting rid of direct calls to psql_error() and GetVariable() in favor of introducing a callback-functions struct to encapsulate variable fetching and error printing. In addition, pass the current encoding and standard-strings status as additional parameters to psql_scan_setup instead of looking directly at "pset" or calling additional functions. I did not bother to change some references to psql_error that are in functions that will soon migrate to a psql-specific backslash-command lexer. Other than that, this version of psqlscan.l is capable of compiling standalone. It still depends on assorted src/common functions as well as some encoding-related libpq functions, but we expect that all programs using it will be happy with those dependencies. Kyotaro Horiguchi, somewhat editorialized on by me
2016-03-18 20:05:49 +01:00
free(value);
}
else
{
/* Emit original token as-is */
Convert psql's flex lexer to be re-entrant, and make it compile standalone. Change psqlscan.l to specify '%option reentrant', adjust internal APIs to match, and get rid of its internal static variables. While this is good cleanup in an abstract sense, the reason to do it right now is that it seems the only practical way to support use of separate flex lexers with common PsqlScanState infrastructure. If we build two non-reentrant lexers then we are going to have problems with dangling buffer pointers in whichever lexer isn't active when we transition from one buffer to another, as well as curious side-effects if we try to share any code between the files. (Horiguchi-san had a different solution to that in his pending patch, but I find it ugly and probably broken for corner cases.) Depending on which version of flex you're using, this may result in getting a "warning: unused variable 'yyg'" warning from psqlscan, similar to the one you'd have seen for a long time in backend/parser/scan.l. I put a local -Wno-error into CFLAGS for the file, for the convenience of those who compile with -Werror. Also, stop compiling psqlscan as part of mainloop.c, and make it a standalone build target instead. This is a lot cleaner than before, though it doesn't really change much in practice as of this commit. (I'm not sure whether the MSVC build scripts will need some help with this part, but the buildfarm will soon tell us.)
2016-03-19 02:21:52 +01:00
emit(state, txt, len);
Allow psql variables to be interpolated with literal or identifier escaping. Loosely based on a patch by Pavel Stehule.
2010-01-29 18:44:12 +01:00
}
}