rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/include/mb/pg_wchar.h

124 lines
4.5 KiB
C
Raw Normal View History

o note that now pg_database has a new attribuite "encoding" even if MULTIBYTE is not enabled. So be sure to run initdb. o these patches are made against the latest source tree (after Bruce's massive patch, I think) BTW, I noticed that after running regression, the oid field of pg_type seems disappeared. regression=> select oid from pg_type; ERROR: attribute 'oid' not found this happens after the constraints test. This occures with/without my patches. strange... o pg_database_mb.h, pg_class_mb.h, pg_attribute_mb.h are no longer used, and shoud be removed. o GetDatabaseInfo() in utils/misc/database.c removed (actually in #ifdef 0). seems nobody uses. t-ishii@sra.co.jp
1998-08-24 03:14:24 +02:00
/* $Id: pg_wchar.h,v 1.3 1998/08/24 01:14:22 momjian Exp $ */
From: t-ishii@sra.co.jp Included are patches intended for allowing PostgreSQL to handle multi-byte charachter sets such as EUC(Extende Unix Code), Unicode and Mule internal code. With the MB patch you can use multi-byte character sets in regexp and LIKE. The encoding system chosen is determined at the compile time. To enable the MB extension, you need to define a variable "MB" in Makefile.global or in Makefile.custom. For further information please take a look at README.mb under doc directory. (Note that unlike "jp patch" I do not use modified GNU regexp any more. I changed Henry Spencer's regexp coming with PostgreSQL.)
1998-03-15 08:39:04 +01:00
#ifndef PG_WCHAR_H
#define PG_WCHAR_H
#include <sys/types.h>
I really hope that I haven't missed anything in this one... From: t-ishii@sra.co.jp Attached are patches to enhance the multi-byte support. (patches are against 7/18 snapshot) * determine encoding at initdb/createdb rather than compile time Now initdb/createdb has an option to specify the encoding. Also, I modified the syntax of CREATE DATABASE to accept encoding option. See README.mb for more details. For this purpose I have added new column "encoding" to pg_database. Also pg_attribute and pg_class are changed to catch up the modification to pg_database. Actually I haved added pg_database_mb.h, pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is enabled. The reason having separate files is I couldn't find a way to use ifdef or whatever in those files. I have to admit it looks ugly. No way. * support for PGCLIENTENCODING when issuing COPY command commands/copy.c modified. * support for SQL92 syntax "SET NAMES" See gram.y. * support for LATIN2-5 * add UNICODE regression test case * new test suite for MB New directory test/mb added. * clean up source files Basic idea is to have MB's own subdirectory for easier maintenance. These are include/mb and backend/utils/mb.
1998-07-24 05:32:46 +02:00
#include "postgres.h"
#include "miscadmin.h" /* for getdatabaseencoding() */
From: t-ishii@sra.co.jp Included are patches intended for allowing PostgreSQL to handle multi-byte charachter sets such as EUC(Extende Unix Code), Unicode and Mule internal code. With the MB patch you can use multi-byte character sets in regexp and LIKE. The encoding system chosen is determined at the compile time. To enable the MB extension, you need to define a variable "MB" in Makefile.global or in Makefile.custom. For further information please take a look at README.mb under doc directory. (Note that unlike "jp patch" I do not use modified GNU regexp any more. I changed Henry Spencer's regexp coming with PostgreSQL.)
1998-03-15 08:39:04 +01:00
o note that now pg_database has a new attribuite "encoding" even if MULTIBYTE is not enabled. So be sure to run initdb. o these patches are made against the latest source tree (after Bruce's massive patch, I think) BTW, I noticed that after running regression, the oid field of pg_type seems disappeared. regression=> select oid from pg_type; ERROR: attribute 'oid' not found this happens after the constraints test. This occures with/without my patches. strange... o pg_database_mb.h, pg_class_mb.h, pg_attribute_mb.h are no longer used, and shoud be removed. o GetDatabaseInfo() in utils/misc/database.c removed (actually in #ifdef 0). seems nobody uses. t-ishii@sra.co.jp
1998-08-24 03:14:24 +02:00
#define SQL_ASCII 0 /* SQL/ASCII */
#define EUC_JP 1 /* EUC for Japanese */
#define EUC_CN 2 /* EUC for Chinese */
#define EUC_KR 3 /* EUC for Korean */
From: t-ishii@sra.co.jp Included are patches intended for allowing PostgreSQL to handle multi-byte charachter sets such as EUC(Extende Unix Code), Unicode and Mule internal code. With the MB patch you can use multi-byte character sets in regexp and LIKE. The encoding system chosen is determined at the compile time. To enable the MB extension, you need to define a variable "MB" in Makefile.global or in Makefile.custom. For further information please take a look at README.mb under doc directory. (Note that unlike "jp patch" I do not use modified GNU regexp any more. I changed Henry Spencer's regexp coming with PostgreSQL.)
1998-03-15 08:39:04 +01:00
#define EUC_TW 3 /* EUC for Taiwan */
o note that now pg_database has a new attribuite "encoding" even if MULTIBYTE is not enabled. So be sure to run initdb. o these patches are made against the latest source tree (after Bruce's massive patch, I think) BTW, I noticed that after running regression, the oid field of pg_type seems disappeared. regression=> select oid from pg_type; ERROR: attribute 'oid' not found this happens after the constraints test. This occures with/without my patches. strange... o pg_database_mb.h, pg_class_mb.h, pg_attribute_mb.h are no longer used, and shoud be removed. o GetDatabaseInfo() in utils/misc/database.c removed (actually in #ifdef 0). seems nobody uses. t-ishii@sra.co.jp
1998-08-24 03:14:24 +02:00
#define UNICODE 5 /* Unicode UTF-8 */
#define MULE_INTERNAL 6 /* Mule internal code */
#define LATIN1 7 /* ISO-8859 Latin 1 */
#define LATIN2 8 /* ISO-8859 Latin 2 */
#define LATIN3 9 /* ISO-8859 Latin 3 */
#define LATIN4 10 /* ISO-8859 Latin 4 */
#define LATIN5 11 /* ISO-8859 Latin 5 */
#define LATIN6 12 /* ISO-8859 Latin 6 */
#define LATIN7 13 /* ISO-8859 Latin 7 */
#define LATIN8 14 /* ISO-8859 Latin 8 */
#define LATIN9 15 /* ISO-8859 Latin 9 */
Hi, here are the patches to enhance existing MB handling. This time I have implemented a framework of encoding translation between the backend and the frontend. Also I have added a new variable setting command: SET CLIENT_ENCODING TO 'encoding'; Other features include: Latin1 support more 8 bit cleaness See doc/README.mb for more details. Note that the pacthes are against May 30 snapshot. Tatsuo Ishii
1998-06-16 09:29:54 +02:00
/* followings are for client encoding only */
I really hope that I haven't missed anything in this one... From: t-ishii@sra.co.jp Attached are patches to enhance the multi-byte support. (patches are against 7/18 snapshot) * determine encoding at initdb/createdb rather than compile time Now initdb/createdb has an option to specify the encoding. Also, I modified the syntax of CREATE DATABASE to accept encoding option. See README.mb for more details. For this purpose I have added new column "encoding" to pg_database. Also pg_attribute and pg_class are changed to catch up the modification to pg_database. Actually I haved added pg_database_mb.h, pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is enabled. The reason having separate files is I couldn't find a way to use ifdef or whatever in those files. I have to admit it looks ugly. No way. * support for PGCLIENTENCODING when issuing COPY command commands/copy.c modified. * support for SQL92 syntax "SET NAMES" See gram.y. * support for LATIN2-5 * add UNICODE regression test case * new test suite for MB New directory test/mb added. * clean up source files Basic idea is to have MB's own subdirectory for easier maintenance. These are include/mb and backend/utils/mb.
1998-07-24 05:32:46 +02:00
#define SJIS 32 /* Shift JIS */
From: t-ishii@sra.co.jp Included are patches intended for allowing PostgreSQL to handle multi-byte charachter sets such as EUC(Extende Unix Code), Unicode and Mule internal code. With the MB patch you can use multi-byte character sets in regexp and LIKE. The encoding system chosen is determined at the compile time. To enable the MB extension, you need to define a variable "MB" in Makefile.global or in Makefile.custom. For further information please take a look at README.mb under doc directory. (Note that unlike "jp patch" I do not use modified GNU regexp any more. I changed Henry Spencer's regexp coming with PostgreSQL.)
1998-03-15 08:39:04 +01:00
From: t-ishii@sra.co.jp As Bruce mentioned, this is due to the conflict among changes we made. Included patches should fix the problem(I changed all MB to MULTIBYTE). Please let me know if you have further problem. P.S. I did not include pathces to configure and gram.c to save the file size(configure.in and gram.y modified).
1998-07-26 06:31:41 +02:00
#ifdef MULTIBYTE
From: t-ishii@sra.co.jp Included are patches intended for allowing PostgreSQL to handle multi-byte charachter sets such as EUC(Extende Unix Code), Unicode and Mule internal code. With the MB patch you can use multi-byte character sets in regexp and LIKE. The encoding system chosen is determined at the compile time. To enable the MB extension, you need to define a variable "MB" in Makefile.global or in Makefile.custom. For further information please take a look at README.mb under doc directory. (Note that unlike "jp patch" I do not use modified GNU regexp any more. I changed Henry Spencer's regexp coming with PostgreSQL.)
1998-03-15 08:39:04 +01:00
typedef unsigned int pg_wchar;
#else
#define pg_wchar char
#endif
/*
* various definitions for EUC
*/
#define SS2 0x8e /* single shift 2 */
#define SS3 0x8f /* single shift 3 */
/*
* various definitions for mule internal code
*/
#define IS_LC1(c) ((unsigned char)(c) >= 0x81 && (unsigned char)(c) <= 0x8f)
#define IS_LCPRV1(c) ((unsigned char)(c) == 0x9a || (unsigned char)(c) == 0x9b)
#define IS_LC2(c) ((unsigned char)(c) >= 0x90 && (unsigned char)(c) <= 0x99)
#define IS_LCPRV2(c) ((unsigned char)(c) == 0x9c || (unsigned char)(c) == 0x9d)
Hi, here are the patches to enhance existing MB handling. This time I have implemented a framework of encoding translation between the backend and the frontend. Also I have added a new variable setting command: SET CLIENT_ENCODING TO 'encoding'; Other features include: Latin1 support more 8 bit cleaness See doc/README.mb for more details. Note that the pacthes are against May 30 snapshot. Tatsuo Ishii
1998-06-16 09:29:54 +02:00
/*
* leading characters
*/
#define LC_ISO8859_1 0x81 /* ISO8859 Latin 1 */
#define LC_ISO8859_2 0x82 /* ISO8859 Latin 2 */
#define LC_ISO8859_3 0x83 /* ISO8859 Latin 3 */
#define LC_ISO8859_4 0x84 /* ISO8859 Latin 4 */
#define LC_ISO8859_5 0x8d /* ISO8859 Latin 5 */
#define LC_JISX0201K 0x89 /* Japanese 1 byte kana */
#define LC_JISX0201R 0x90 /* Japanese 1 byte Roman */
#define LC_GB2312_80 0x91 /* Chinese */
#define LC_JISX0208 0x92 /* Japanese Kanji */
#define LC_KS5601 0x93 /* Korean */
#define LC_JISX0212 0x94 /* Japanese Kanji (JISX0212) */
#define LC_CNS11643_1 0x95 /* CNS 11643-1992 Plane 1 */
#define LC_CNS11643_2 0x96 /* CNS 11643-1992 Plane 2 */
#define LC_CNS11643_3 0xf6 /* CNS 11643-1992 Plane 3 */
#define LC_CNS11643_4 0xf7 /* CNS 11643-1992 Plane 4 */
#define LC_CNS11643_5 0xf8 /* CNS 11643-1992 Plane 5 */
#define LC_CNS11643_6 0xf9 /* CNS 11643-1992 Plane 6 */
#define LC_CNS11643_7 0xfa /* CNS 11643-1992 Plane 7 */
From: t-ishii@sra.co.jp As Bruce mentioned, this is due to the conflict among changes we made. Included patches should fix the problem(I changed all MB to MULTIBYTE). Please let me know if you have further problem. P.S. I did not include pathces to configure and gram.c to save the file size(configure.in and gram.y modified).
1998-07-26 06:31:41 +02:00
#ifdef MULTIBYTE
I really hope that I haven't missed anything in this one... From: t-ishii@sra.co.jp Attached are patches to enhance the multi-byte support. (patches are against 7/18 snapshot) * determine encoding at initdb/createdb rather than compile time Now initdb/createdb has an option to specify the encoding. Also, I modified the syntax of CREATE DATABASE to accept encoding option. See README.mb for more details. For this purpose I have added new column "encoding" to pg_database. Also pg_attribute and pg_class are changed to catch up the modification to pg_database. Actually I haved added pg_database_mb.h, pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is enabled. The reason having separate files is I couldn't find a way to use ifdef or whatever in those files. I have to admit it looks ugly. No way. * support for PGCLIENTENCODING when issuing COPY command commands/copy.c modified. * support for SQL92 syntax "SET NAMES" See gram.y. * support for LATIN2-5 * add UNICODE regression test case * new test suite for MB New directory test/mb added. * clean up source files Basic idea is to have MB's own subdirectory for easier maintenance. These are include/mb and backend/utils/mb.
1998-07-24 05:32:46 +02:00
typedef struct {
int encoding; /* encoding symbol value */
char *name; /* encoding name */
int is_client_only; /* 0: server/client bothg supported
1: client only */
void (*to_mic)(); /* client encoding to MIC */
void (*from_mic)(); /* MIC to client encoding */
} pg_encoding_conv_tbl;
extern pg_encoding_conv_tbl pg_conv_tbl[];
typedef struct {
void (*mb2wchar_with_len)(); /* convert a multi-byte string to a wchar */
int (*mblen)(); /* returns the length of a multi-byte word */
} pg_wchar_tbl;
extern pg_wchar_tbl pg_wchar_table[];
From: t-ishii@sra.co.jp Included are patches intended for allowing PostgreSQL to handle multi-byte charachter sets such as EUC(Extende Unix Code), Unicode and Mule internal code. With the MB patch you can use multi-byte character sets in regexp and LIKE. The encoding system chosen is determined at the compile time. To enable the MB extension, you need to define a variable "MB" in Makefile.global or in Makefile.custom. For further information please take a look at README.mb under doc directory. (Note that unlike "jp patch" I do not use modified GNU regexp any more. I changed Henry Spencer's regexp coming with PostgreSQL.)
1998-03-15 08:39:04 +01:00
extern void pg_mb2wchar(const unsigned char *, pg_wchar *);
extern void pg_mb2wchar_with_len(const unsigned char *, pg_wchar *, int);
extern int pg_char_and_wchar_strcmp(const char *, const pg_wchar *);
extern int pg_wchar_strncmp(const pg_wchar *, const pg_wchar *, size_t);
extern int pg_char_and_wchar_strncmp(const char *, const pg_wchar *, size_t);
extern size_t pg_wchar_strlen(const pg_wchar *);
From: t-ishii@sra.co.jp Hi, here are patches I promised (against 6.3.2): * character_length(), position(), substring() are now aware of multi-byte characters * add octet_length() * add --with-mb option to configure * new regression tests for EUC_KR (contributed by "Soonmyung. Hong" <hong@lunaris.hanmesoft.co.kr>) * add some test cases to the EUC_JP regression test * fix problem in regress/regress.sh in case of System V * fix toupper(), tolower() to handle 8bit chars note that: o patches for both configure.in and configure are included. maybe the one for configure is not necessary. o pg_proc.h was modified to add octet_length(). I used OIDs (1374-1379) for that. Please let me know if these numbers are not appropriate.
1998-04-27 19:10:50 +02:00
extern int pg_mblen(const unsigned char *);
Hi, here are the patches to enhance existing MB handling. This time I have implemented a framework of encoding translation between the backend and the frontend. Also I have added a new variable setting command: SET CLIENT_ENCODING TO 'encoding'; Other features include: Latin1 support more 8 bit cleaness See doc/README.mb for more details. Note that the pacthes are against May 30 snapshot. Tatsuo Ishii
1998-06-16 09:29:54 +02:00
extern int pg_encoding_mblen(int, const unsigned char *);
I really hope that I haven't missed anything in this one... From: t-ishii@sra.co.jp Attached are patches to enhance the multi-byte support. (patches are against 7/18 snapshot) * determine encoding at initdb/createdb rather than compile time Now initdb/createdb has an option to specify the encoding. Also, I modified the syntax of CREATE DATABASE to accept encoding option. See README.mb for more details. For this purpose I have added new column "encoding" to pg_database. Also pg_attribute and pg_class are changed to catch up the modification to pg_database. Actually I haved added pg_database_mb.h, pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is enabled. The reason having separate files is I couldn't find a way to use ifdef or whatever in those files. I have to admit it looks ugly. No way. * support for PGCLIENTENCODING when issuing COPY command commands/copy.c modified. * support for SQL92 syntax "SET NAMES" See gram.y. * support for LATIN2-5 * add UNICODE regression test case * new test suite for MB New directory test/mb added. * clean up source files Basic idea is to have MB's own subdirectory for easier maintenance. These are include/mb and backend/utils/mb.
1998-07-24 05:32:46 +02:00
extern int pg_mule_mblen(const unsigned char *);
Hi, here are the patches to enhance existing MB handling. This time I have implemented a framework of encoding translation between the backend and the frontend. Also I have added a new variable setting command: SET CLIENT_ENCODING TO 'encoding'; Other features include: Latin1 support more 8 bit cleaness See doc/README.mb for more details. Note that the pacthes are against May 30 snapshot. Tatsuo Ishii
1998-06-16 09:29:54 +02:00
extern int pg_mic_mblen(const unsigned char *);
From: t-ishii@sra.co.jp Hi, here are patches I promised (against 6.3.2): * character_length(), position(), substring() are now aware of multi-byte characters * add octet_length() * add --with-mb option to configure * new regression tests for EUC_KR (contributed by "Soonmyung. Hong" <hong@lunaris.hanmesoft.co.kr>) * add some test cases to the EUC_JP regression test * fix problem in regress/regress.sh in case of System V * fix toupper(), tolower() to handle 8bit chars note that: o patches for both configure.in and configure are included. maybe the one for configure is not necessary. o pg_proc.h was modified to add octet_length(). I used OIDs (1374-1379) for that. Please let me know if these numbers are not appropriate.
1998-04-27 19:10:50 +02:00
extern int pg_mbstrlen(const unsigned char *);
extern int pg_mbstrlen_with_len(const unsigned char *, int);
I really hope that I haven't missed anything in this one... From: t-ishii@sra.co.jp Attached are patches to enhance the multi-byte support. (patches are against 7/18 snapshot) * determine encoding at initdb/createdb rather than compile time Now initdb/createdb has an option to specify the encoding. Also, I modified the syntax of CREATE DATABASE to accept encoding option. See README.mb for more details. For this purpose I have added new column "encoding" to pg_database. Also pg_attribute and pg_class are changed to catch up the modification to pg_database. Actually I haved added pg_database_mb.h, pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is enabled. The reason having separate files is I couldn't find a way to use ifdef or whatever in those files. I have to admit it looks ugly. No way. * support for PGCLIENTENCODING when issuing COPY command commands/copy.c modified. * support for SQL92 syntax "SET NAMES" See gram.y. * support for LATIN2-5 * add UNICODE regression test case * new test suite for MB New directory test/mb added. * clean up source files Basic idea is to have MB's own subdirectory for easier maintenance. These are include/mb and backend/utils/mb.
1998-07-24 05:32:46 +02:00
extern pg_encoding_conv_tbl *pg_get_encent_by_encoding(int);
extern bool show_client_encoding(void);
extern bool reset_client_encoding(void);
extern bool parse_client_encoding(const char *);
extern bool show_server_encoding(void);
extern bool reset_server_encoding(void);
extern bool parse_server_encoding(const char *);
extern int pg_set_client_encoding(int);
extern int pg_get_client_encoding(void);
extern unsigned char *pg_client_to_server(unsigned char *, int);
extern unsigned char *pg_server_to_client(unsigned char *, int);
extern int pg_valid_client_encoding(const char *);
extern const char *pg_encoding_to_char(int);
extern int pg_char_to_encoding(const char *);
extern int GetDatabaseEncoding(void);
extern void SetDatabaseEncoding(int);
extern void SetTemplateEncoding(int);
extern int GetTemplateEncoding(void);
From: t-ishii@sra.co.jp Included are patches intended for allowing PostgreSQL to handle multi-byte charachter sets such as EUC(Extende Unix Code), Unicode and Mule internal code. With the MB patch you can use multi-byte character sets in regexp and LIKE. The encoding system chosen is determined at the compile time. To enable the MB extension, you need to define a variable "MB" in Makefile.global or in Makefile.custom. For further information please take a look at README.mb under doc directory. (Note that unlike "jp patch" I do not use modified GNU regexp any more. I changed Henry Spencer's regexp coming with PostgreSQL.)
1998-03-15 08:39:04 +01:00
From: t-ishii@sra.co.jp As Bruce mentioned, this is due to the conflict among changes we made. Included patches should fix the problem(I changed all MB to MULTIBYTE). Please let me know if you have further problem. P.S. I did not include pathces to configure and gram.c to save the file size(configure.in and gram.y modified).
1998-07-26 06:31:41 +02:00
#endif /* MULTIBYTE */
I really hope that I haven't missed anything in this one... From: t-ishii@sra.co.jp Attached are patches to enhance the multi-byte support. (patches are against 7/18 snapshot) * determine encoding at initdb/createdb rather than compile time Now initdb/createdb has an option to specify the encoding. Also, I modified the syntax of CREATE DATABASE to accept encoding option. See README.mb for more details. For this purpose I have added new column "encoding" to pg_database. Also pg_attribute and pg_class are changed to catch up the modification to pg_database. Actually I haved added pg_database_mb.h, pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is enabled. The reason having separate files is I couldn't find a way to use ifdef or whatever in those files. I have to admit it looks ugly. No way. * support for PGCLIENTENCODING when issuing COPY command commands/copy.c modified. * support for SQL92 syntax "SET NAMES" See gram.y. * support for LATIN2-5 * add UNICODE regression test case * new test suite for MB New directory test/mb added. * clean up source files Basic idea is to have MB's own subdirectory for easier maintenance. These are include/mb and backend/utils/mb.
1998-07-24 05:32:46 +02:00
#endif /* PG_WCHAR_H */