postgresql/src/common/unicode
Jeff Davis cf64d4e99f Cleanup for unicode-update build target and test.
In preparation for adding more Unicode tables.

Discussion: https://postgr.es/m/63cd8625-68fa-4760-844a-6b7f643336f2@ardentperf.com
Reviewed-by: Jeremy Schneider
2024-01-11 12:35:29 -08:00
..
.gitignore Update display widths as part of updating Unicode 2021-08-26 10:53:56 -04:00
Makefile Cleanup for unicode-update build target and test. 2024-01-11 12:35:29 -08:00
README Add support for automatically updating Unicode derived files 2020-01-09 10:08:14 +01:00
category_test.c Cleanup for unicode-update build target and test. 2024-01-11 12:35:29 -08:00
generate-norm_test_table.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
generate-unicode_category_table.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
generate-unicode_east_asian_fw_table.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
generate-unicode_nonspacing_table.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
generate-unicode_norm_table.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
generate-unicode_normprops_table.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
generate-unicode_version.pl Update copyright for 2024 2024-01-03 20:49:05 -05:00
meson.build Cleanup for unicode-update build target and test. 2024-01-11 12:35:29 -08:00
norm_test.c Update copyright for 2024 2024-01-03 20:49:05 -05:00

README

This directory contains tools to generate the tables in
src/include/common/unicode_norm.h, used for Unicode normalization. The
generated .h file is included in the source tree, so these are normally not
needed to build PostgreSQL, only if you need to re-generate the .h file
from the Unicode data files for some reason, e.g. to update to a new version
of Unicode.

Generating unicode_norm_table.h
-------------------------------

Run

    make update-unicode

from the top level of the source tree and commit the result.

Tests
-----

The Unicode consortium publishes a comprehensive test suite for the
normalization algorithm, in a file called NormalizationTest.txt. This
directory also contains a perl script and some C code, to run our
normalization code with all the test strings in NormalizationTest.txt.
To download NormalizationTest.txt and run the tests:

    make normalization-check

This is also run as part of the update-unicode target.