postgresql

History

Andrew Dunstan 3311ea86ed Introduce a non-recursive JSON parser This parser uses an explicit prediction stack, unlike the present recursive descent parser where the parser state is represented on the call stack. This difference makes the new parser suitable for use in incremental parsing of huge JSON documents that cannot be conveniently handled piece-wise by the recursive descent parser. One potential use for this will be in parsing large backup manifests associated with incremental backups. Because this parser is somewhat slower than the recursive descent parser, it is not replacing that parser, but is an additional parser available to callers. For testing purposes, if the build is done with -DFORCE_JSON_PSTACK, all JSON parsing is done with the non-recursive parser, in which case only trivial regression differences in error messages should be observed. Author: Andrew Dunstan Reviewed-By: Jacob Champion Discussion: https://postgr.es/m/7b0a51d6-0d9d-7366-3a1a-f74397a02f55@dunslane.net		2024-04-04 06:46:40 -04:00
..
t	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00
Makefile	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00
README	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00
meson.build	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00
test_json_parser_incremental.c	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00
test_json_parser_perf.c	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00
tiny.json	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00
tiny.out	Introduce a non-recursive JSON parser	2024-04-04 06:46:40 -04:00

README

Module `test_json_parser`
=========================

This module contains two programs for testing the json parsers.

- `test_json_parser_incremental` is for testing the incremental parser, It
  reads in a file and pases it in very small chunks (60 bytes at a time) to
  the incremental parser. It's not meant to be a speed test but to test the
  accuracy of the incremental parser. It takes one argument: the name of the
  input file.
- `test_json_parser_perf` is for speed testing both the standard
  recursive descent parser and the non-recursive incremental
  parser. If given the `-i` flag it uses the non-recursive parser,
  otherwise the stardard parser. The remaining flags are the number of
  parsing iterations and the file containing the input. Even when
  using the non-recursive parser, the input is passed to the parser in a
  single chunk. The results are thus comparable to those of the
  standard parser.

The easiest way to use these is to run `make check` and `make speed-check`

The sample input file is a small extract from a list of `delicious`
bookmarks taken some years ago, all wrapped in a single json
array. 10,000 iterations of parsing this file gives a reasonable
benchmark, and that is what the `speed-check` target does.