911588a3f8
Our previous validator used a traditional algorithm that performed comparison and branching one byte at a time. It's useful in that we always know exactly how many bytes we have validated, but that precision comes at a cost. Input validation can show up prominently in profiles of COPY FROM, and future improvements to COPY FROM such as parallelism or faster line parsing will put more pressure on input validation. Hence, add fast paths for both ASCII and multibyte UTF-8: Use bitwise operations to check 16 bytes at a time for ASCII. If that fails, use a "shift-based" DFA on those bytes to handle the general case, including multibyte. These paths are relatively free of branches and thus robust against all kinds of byte patterns. With these algorithms, UTF-8 validation is several times faster, depending on platform and the input byte distribution. The previous coding in pg_utf8_verifystr() is retained for short strings and for when the fast path returns an error. Review, performance testing, and additional hacking by: Heikki Linakangas, Vladimir Sitnikov, Amit Khandekar, Thomas Munro, and Greg Stark Discussion: https://www.postgresql.org/message-id/CAFBsxsEV_SzH%2BOLyCiyon%3DiwggSyMh_eF6A3LU2tiWf3Cy2ZQg%40mail.gmail.com |
||
|---|---|---|
| .. | ||
| data | ||
| expected | ||
| input | ||
| output | ||
| sql | ||
| .gitignore | ||
| GNUmakefile | ||
| Makefile | ||
| README | ||
| parallel_schedule | ||
| pg_regress.c | ||
| pg_regress.h | ||
| pg_regress_main.c | ||
| regress.c | ||
| regressplans.sh | ||
| resultmap | ||
| standby_schedule | ||
README
Documentation concerning how to run these regression tests and interpret the results can be found in the PostgreSQL manual, in the chapter "Regression Tests".