Commit Graph

112 Commits

Author SHA1 Message Date
Tatsuo Ishii ed7baeaf4d Remove #ifdef MULTIBYTE per hackers list discussion. 2002-08-29 07:22:30 +00:00
Bruce Momjian ff1793f036 Remove erroneous character from Makefile due to editor error. 2002-08-22 02:18:45 +00:00
Tom Lane b663f3443b Add a bunch of pseudo-types to replace the behavior formerly associated
with OPAQUE, as per recent pghackers discussion.  I still want to do some
more work on the 'cstring' pseudo-type, but I'm going to commit the bulk
of the changes now before the tree starts shifting under me ...
2002-08-22 00:01:51 +00:00
Bruce Momjian d46e3dc00f Changes made so new conversion Makefiles will build out of the source tree. 2002-08-21 21:33:55 +00:00
Tatsuo Ishii 10b374aecf Fix bug in pg_convert() per report from MaC.Yui.
It pfree() wrong pointer.
2002-08-19 04:08:08 +00:00
Tatsuo Ishii 538b101595 Fix memory leak in SetClientEncoding(). 2002-08-14 05:33:34 +00:00
Tatsuo Ishii 969e0246ed Add Cyrillic and other encodings for encoding conversion.
Patches submitted by Kaori Inaba (i-kaori@sra.co.jp).
2002-08-14 02:45:10 +00:00
Tatsuo Ishii 697b472099 Address build problems on cygwin and (hopefully) AIX. 2002-08-08 07:47:43 +00:00
Tatsuo Ishii 3c63578a7e Load and keep conversion function info when SET CLIENT_ENCODING TO is
executed to prevent database access while performing encoding
conversion.
2002-08-08 06:35:26 +00:00
Tatsuo Ishii 6206a880cf Add SQL99 CONVERT() function. 2002-08-06 05:40:47 +00:00
Tatsuo Ishii 0345f58496 Implement DROP CONVERSION
Add regression test
2002-07-25 10:07:13 +00:00
Tatsuo Ishii 19a20e04bd Add Japanese README explaining how to add new conversion.
English README will come soon...
2002-07-24 07:05:41 +00:00
Tatsuo Ishii 86270024ff Oops. Too much ifdef out. 2002-07-19 11:09:25 +00:00
Tatsuo Ishii 248cbb5796 Temporary ifdef out migrating functions to avoid compiler warnings. 2002-07-19 00:22:24 +00:00
Peter Eisentraut 85d2a629c6 Create directory before installing files. 2002-07-18 22:58:08 +00:00
Tatsuo Ishii eb335a034b I have committed many support files for CREATE CONVERSION. Default
conversion procs and conversions are added in initdb. Currently
supported conversions are:

UTF-8(UNICODE) <--> SQL_ASCII, ISO-8859-1 to 16, EUC_JP, EUC_KR,
		    EUC_CN, EUC_TW, SJIS, BIG5, GBK, GB18030, UHC,
		    JOHAB, TCVN

EUC_JP <--> SJIS
EUC_TW <--> BIG5
MULE_INTERNAL <--> EUC_JP, SJIS, EUC_TW, BIG5

Note that initial contents of pg_conversion system catalog are created
in the initdb process. So doing initdb required is ideal, it's
possible to add them to your databases by hand, however. To accomplish
this:

psql -f your_postgresql_install_path/share/conversion_create.sql your_database

So I did not bump up the version in cataversion.h.

TODO:
Add more conversion procs
Add [CASCADE|RESTRICT] to DROP CONVERSION
Add tuples to pg_depend
Add regression tests
Write docs
Add SQL99 CONVERT command?
--
Tatsuo Ishii
2002-07-18 02:02:30 +00:00
Tatsuo Ishii 3c7798f068 Add conversion procs for CREATE CONVERSION 2002-07-16 09:25:06 +00:00
Tatsuo Ishii 15378a53f8 Add support for GB18030 2002-06-14 03:30:56 +00:00
Tatsuo Ishii 14f72b9a4d Add GB18030 support. Contributed by Bill Huang <bill_huanghb@ybb.ne.jp>
(ODBC support has not been committed yet. left for Hiroshi...)
2002-06-13 08:30:22 +00:00
Bruce Momjian e6227fd0ec Add missing Unicode multibyte files. 2002-03-06 06:12:59 +00:00
Bruce Momjian 92288a1cf9 Change made to elog:
o  Change all current CVS messages of NOTICE to WARNING.  We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.

o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.

o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.

o Remove INFO from the client_min_messages options and add NOTICE.

Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.

Regression passed.
2002-03-06 06:10:59 +00:00
Bruce Momjian a8bd7e1c6e > Tatsuo Ishii wrote:
> > > > It was made to cope with encoding such as an Asian bloc in 7.2Beta2.
> > > >
> > > > Added ServerEncoding
> > > >         Korean (JOHAB), Thai (WIN874),
> > > >         Vietnamese (TCVN), Arabic (WIN1256)
> > > >
> > > > Added ClientEncoding
> > > >         Simplified Chinese (GBK), Korean (UHC)
> > > >
> > > >
> > > >
> http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2b2.newencoding.diff.tar.gz
> > > > (608K)
> > >
> > > Looks good.  I need some people to review this for me.
> >
> > For me they look good too. The only missing part is a
> > documentation. I will ask him to write it up. If he couldn't, I will
> > do it for him.
> > > The diff is 3mb
> > > but appears to address only additions to multibyte.  I have attached a
> > > list of files it modifies.  Also, look at the sizes of the mb/
> > > directory.  It is getting large:
> > >
> > >   4       ./CVS
> > >   6       ./Unicode/CVS
> > >   3433    ./Unicode
> > >   6197    .
> >
> > Yes. We definitely need the on-the-fly encoding addition capability:
> > i.e. CREATE CHRACTER SET in the future...
> > --
> > Tatsuo Ishii
> >
> >

Address chainge.

http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2.newencoding.diff.gz

Add PsqlODBC and document ...etc patch.

Eiji Tokuya
2002-03-05 05:52:50 +00:00
Tatsuo Ishii 933761e7b1 Simplify pg_convert() in that it calls pg_convert2 using new fmgr interface. 2001-11-20 01:32:29 +00:00
Tatsuo Ishii 5590d5fe99 Fix nasty bugs in pg_convert() and pg_convert2().
o they sometimes returns a result garbage string appended.
    o they do not work if client encoding is different from server
      encoding
2001-11-19 06:48:39 +00:00
Bruce Momjian ea08e6cd55 New pgindent run with fixes suggested by Tom. Patch manually reviewed,
initdb/regression tests pass.
2001-11-05 17:46:40 +00:00
Bruce Momjian 6783b2372e Another pgindent run. Fixes enum indenting, and improves #endif
spacing.  Also adds space for one-line comments.
2001-10-28 06:26:15 +00:00
Bruce Momjian b81844b173 pgindent run on all C files. Java run to follow. initdb/regression
tests pass.
2001-10-25 05:50:21 +00:00
Tatsuo Ishii cfe01796e6 Ok, here is the modified encoding table (column1 is the standard name,
2 is our "official" name, and 3 is alias). If there's no objection, I
will change them.

ASCII		SQL_ASCII
UTF-8		UNICODE		UTF_8
MULE-INTERNAL	MULE_INTERNAL
ISO-8859-1	LATIN1		ISO_8859_1
ISO-8859-2	LATIN2		ISO_8859_2
ISO-8859-3	LATIN3		ISO_8859_3
ISO-8859-4	LATIN4		ISO_8859_4
ISO-8859-5	ISO_8859_5
ISO-8859-6	ISO_8859_6
ISO-8859-7	ISO_8859_7
ISO-8859-8	ISO_8859_8
ISO-8859-9	LATIN5		ISO_8859_9
ISO-8859-10	LATIN6		ISO_8859_10
ISO-8859-13	LATIN7		ISO_8859_13
ISO-8859-14	LATIN8		ISO_8859_14
ISO-8859-15	LATIN9		ISO_8859_15
ISO-8859-16	LATIN10		ISO_8859_16
2001-10-16 10:09:17 +00:00
Tatsuo Ishii d07bacd54a Add UTF-8 char >= 0x10000 check 2001-10-15 01:19:15 +00:00
Tatsuo Ishii f426465ba9 Add a new function "pg_client_encoding" which returns the current client
side encoding name. This is necessary for client API's such as JDBC
to perform correct encoding conversions. See my email "[HACKERS]
pg_client_encoding" 10 Sep 2001.
2001-10-12 02:08:34 +00:00
Tatsuo Ishii 51053d3216 Add support for ISO-8859-6 to 16 2001-10-11 14:20:35 +00:00
Tatsuo Ishii 1b20315008 Fix bug in mic2ascii(). It does not handle correctly if none ASCII
chars are in the input.
2001-09-25 01:27:03 +00:00
Tatsuo Ishii be629abfc8 Add pg_database_encoding_max_length() function. 2001-09-23 10:59:45 +00:00
Tatsuo Ishii 8ebdac0ed5 Remove test drivers
Also fix comment in conv.c.
2001-09-22 08:44:49 +00:00
Tom Lane e3f5bc3492 Fix type_maximum_size() to give the right answer in MULTIBYTE cases.
Avoid use of prototype-less function pointers in MB code.
2001-09-21 15:27:38 +00:00
Peter Eisentraut fd5e95971e Remove old file. 2001-09-19 21:28:55 +00:00
Tatsuo Ishii e1de3e0833 Implement following item in TODO:
* Reject character sequences those are not valid in their charset
2001-09-11 04:50:36 +00:00
Tatsuo Ishii d330f09a56 Backout Karel's patch 2001-09-09 01:15:11 +00:00
Bruce Momjian fdbf796f36 > > A simple and robus solution is in the begin of mbutils.c set default
> > ClientEncoding to SQL_ASCII (like default DatabaseEncoding). Bruce, can
> > you change it? It's one line change. Again thanks.

 Forget it! A default client encoding must be set by actual database encoding...
Please apply the small attached patch that solve it better.

Karel Zak
2001-09-08 14:30:15 +00:00
Bruce Momjian d9044b5637 Remove file, per Karel. 2001-09-07 15:14:16 +00:00
Bruce Momjian 4ea26bf354 Remove variable length macros used in debugging, per Karel. 2001-09-07 15:01:45 +00:00
Bruce Momjian 7bfc83f673 Remove unused files for Karel's patch. 2001-09-07 14:17:17 +00:00
Bruce Momjian 9f5185cf63 Remove common.c, removed in Karal's patch. 2001-09-07 14:00:25 +00:00
Tatsuo Ishii 3bdd67a203 Add missing files. 2001-09-07 03:32:11 +00:00
Tatsuo Ishii 227767112c Commit Karel's patch.
-------------------------------------------------------------------
Subject: Re: [PATCHES] encoding names
From: Karel Zak <zakkr@zf.jcu.cz>
To: Peter Eisentraut <peter_e@gmx.net>
Cc: pgsql-patches <pgsql-patches@postgresql.org>
Date: Fri, 31 Aug 2001 17:24:38 +0200

On Thu, Aug 30, 2001 at 01:30:40AM +0200, Peter Eisentraut wrote:
> > 		- convert encoding 'name' to 'id'
>
> I thought we decided not to add functions returning "new" names until we
> know exactly what the new names should be, and pending schema

 Ok, the patch not to add functions.

> better
>
>     ...(): encoding name too long

 Fixed.

 I found new bug in command/variable.c in parse_client_encoding(), nobody
probably never see this error:

if (pg_set_client_encoding(encoding))
{
	elog(ERROR, "Conversion between %s and %s is not supported",
                     value, GetDatabaseEncodingName());
}

because pg_set_client_encoding() returns -1 for error and 0 as true.
It's fixed too.

 IMHO it can be apply.

		Karel
PS:

    * following files are renamed:

src/utils/mb/Unicode/KOI8_to_utf8.map  -->
        src/utils/mb/Unicode/koi8r_to_utf8.map

src/utils/mb/Unicode/WIN_to_utf8.map  -->
        src/utils/mb/Unicode/win1251_to_utf8.map

src/utils/mb/Unicode/utf8_to_KOI8.map -->
        src/utils/mb/Unicode/utf8_to_koi8r.map

src/utils/mb/Unicode/utf8_to_WIN.map -->
        src/utils/mb/Unicode/utf8_to_win1251.map

   * new file:

src/utils/mb/encname.c

   * removed file:

src/utils/mb/common.c

--
 Karel Zak  <zakkr@zf.jcu.cz>
 http://home.zf.jcu.cz/~zakkr/

 C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz
2001-09-06 04:57:30 +00:00
Tatsuo Ishii ab9b6c45cf Add conver/convert2 functions. They are similar to the SQL99's convert. 2001-08-15 07:07:40 +00:00
Tatsuo Ishii 1032445e5d TODO item:
* Make n of CHAR(n)/VARCHAR(n) the number of letters, not bytes
2001-07-15 11:07:37 +00:00
Tatsuo Ishii e23f8c4557 Fix a message error in utf_to_local 2001-05-28 01:00:25 +00:00
Bruce Momjian 0cec2bb0cd BTW it does not add encodign it just patches existing one (KOI8) to
support two - KOI8-R and KOI8-U (latter is superset of the former if
not to take to the account pseudographics)

Andy Rysin
2001-05-03 21:38:45 +00:00
Tatsuo Ishii c527366b60 Add missing Unicode support for Cyrillic encodings.
Patches contributed by Victor Wagner.
2001-04-29 07:27:38 +00:00