Commit Graph

135 Commits

Author SHA1 Message Date
Bruce Momjian e50f52a074 pgindent run. 2002-09-04 20:31:48 +00:00
Peter Eisentraut 77f7763b55 Remove all traces of multibyte and locale options. Clean up comments
referring to "multibyte" where it really means character encoding.
2002-09-03 21:45:44 +00:00
Tatsuo Ishii ed7baeaf4d Remove #ifdef MULTIBYTE per hackers list discussion. 2002-08-29 07:22:30 +00:00
Tatsuo Ishii 10b374aecf Fix bug in pg_convert() per report from MaC.Yui.
It pfree() wrong pointer.
2002-08-19 04:08:08 +00:00
Tatsuo Ishii 538b101595 Fix memory leak in SetClientEncoding(). 2002-08-14 05:33:34 +00:00
Tatsuo Ishii 3c63578a7e Load and keep conversion function info when SET CLIENT_ENCODING TO is
executed to prevent database access while performing encoding
conversion.
2002-08-08 06:35:26 +00:00
Tatsuo Ishii 0345f58496 Implement DROP CONVERSION
Add regression test
2002-07-25 10:07:13 +00:00
Tatsuo Ishii eb335a034b I have committed many support files for CREATE CONVERSION. Default
conversion procs and conversions are added in initdb. Currently
supported conversions are:

UTF-8(UNICODE) <--> SQL_ASCII, ISO-8859-1 to 16, EUC_JP, EUC_KR,
		    EUC_CN, EUC_TW, SJIS, BIG5, GBK, GB18030, UHC,
		    JOHAB, TCVN

EUC_JP <--> SJIS
EUC_TW <--> BIG5
MULE_INTERNAL <--> EUC_JP, SJIS, EUC_TW, BIG5

Note that initial contents of pg_conversion system catalog are created
in the initdb process. So doing initdb required is ideal, it's
possible to add them to your databases by hand, however. To accomplish
this:

psql -f your_postgresql_install_path/share/conversion_create.sql your_database

So I did not bump up the version in cataversion.h.

TODO:
Add more conversion procs
Add [CASCADE|RESTRICT] to DROP CONVERSION
Add tuples to pg_depend
Add regression tests
Write docs
Add SQL99 CONVERT command?
--
Tatsuo Ishii
2002-07-18 02:02:30 +00:00
Tatsuo Ishii 933761e7b1 Simplify pg_convert() in that it calls pg_convert2 using new fmgr interface. 2001-11-20 01:32:29 +00:00
Tatsuo Ishii 5590d5fe99 Fix nasty bugs in pg_convert() and pg_convert2().
o they sometimes returns a result garbage string appended.
    o they do not work if client encoding is different from server
      encoding
2001-11-19 06:48:39 +00:00
Bruce Momjian b81844b173 pgindent run on all C files. Java run to follow. initdb/regression
tests pass.
2001-10-25 05:50:21 +00:00
Tatsuo Ishii f426465ba9 Add a new function "pg_client_encoding" which returns the current client
side encoding name. This is necessary for client API's such as JDBC
to perform correct encoding conversions. See my email "[HACKERS]
pg_client_encoding" 10 Sep 2001.
2001-10-12 02:08:34 +00:00
Tom Lane e3f5bc3492 Fix type_maximum_size() to give the right answer in MULTIBYTE cases.
Avoid use of prototype-less function pointers in MB code.
2001-09-21 15:27:38 +00:00
Tatsuo Ishii d330f09a56 Backout Karel's patch 2001-09-09 01:15:11 +00:00
Bruce Momjian fdbf796f36 > > A simple and robus solution is in the begin of mbutils.c set default
> > ClientEncoding to SQL_ASCII (like default DatabaseEncoding). Bruce, can
> > you change it? It's one line change. Again thanks.

 Forget it! A default client encoding must be set by actual database encoding...
Please apply the small attached patch that solve it better.

Karel Zak
2001-09-08 14:30:15 +00:00
Tatsuo Ishii 227767112c Commit Karel's patch.
-------------------------------------------------------------------
Subject: Re: [PATCHES] encoding names
From: Karel Zak <zakkr@zf.jcu.cz>
To: Peter Eisentraut <peter_e@gmx.net>
Cc: pgsql-patches <pgsql-patches@postgresql.org>
Date: Fri, 31 Aug 2001 17:24:38 +0200

On Thu, Aug 30, 2001 at 01:30:40AM +0200, Peter Eisentraut wrote:
> > 		- convert encoding 'name' to 'id'
>
> I thought we decided not to add functions returning "new" names until we
> know exactly what the new names should be, and pending schema

 Ok, the patch not to add functions.

> better
>
>     ...(): encoding name too long

 Fixed.

 I found new bug in command/variable.c in parse_client_encoding(), nobody
probably never see this error:

if (pg_set_client_encoding(encoding))
{
	elog(ERROR, "Conversion between %s and %s is not supported",
                     value, GetDatabaseEncodingName());
}

because pg_set_client_encoding() returns -1 for error and 0 as true.
It's fixed too.

 IMHO it can be apply.

		Karel
PS:

    * following files are renamed:

src/utils/mb/Unicode/KOI8_to_utf8.map  -->
        src/utils/mb/Unicode/koi8r_to_utf8.map

src/utils/mb/Unicode/WIN_to_utf8.map  -->
        src/utils/mb/Unicode/win1251_to_utf8.map

src/utils/mb/Unicode/utf8_to_KOI8.map -->
        src/utils/mb/Unicode/utf8_to_koi8r.map

src/utils/mb/Unicode/utf8_to_WIN.map -->
        src/utils/mb/Unicode/utf8_to_win1251.map

   * new file:

src/utils/mb/encname.c

   * removed file:

src/utils/mb/common.c

--
 Karel Zak  <zakkr@zf.jcu.cz>
 http://home.zf.jcu.cz/~zakkr/

 C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz
2001-09-06 04:57:30 +00:00
Tatsuo Ishii ab9b6c45cf Add conver/convert2 functions. They are similar to the SQL99's convert. 2001-08-15 07:07:40 +00:00
Tatsuo Ishii 1032445e5d TODO item:
* Make n of CHAR(n)/VARCHAR(n) the number of letters, not bytes
2001-07-15 11:07:37 +00:00
Tom Lane fbee97664e getdatabaseencoding() and PG_encoding_to_char() were being sloppy about
converting char* strings to type 'name'.  Imagine my surprise when 7.1
release coredumped upon start when compiled --enable-multibyte ...
2001-04-16 02:42:01 +00:00
Tom Lane 572fda2711 Modify wchar conversion routines to not fetch the next byte past the end
of a counted input string.  Marinos Yannikos' recent crash report turns
out to be due to applying pg_ascii2wchar_with_len to a TEXT object that
is smack up against the end of memory.  This is the second just-barely-
reproducible bug report I have seen that traces to some bit of code
fetching one more byte than it is allowed to.  Let's be more careful
out there, boys and girls.
While at it, I changed the code to not risk a similar crash when there
is a truncated multibyte character at the end of an input string.  The
output in this case might not be the most reasonable output possible;
if anyone wants to improve it further, step right up...
2001-03-08 00:24:34 +00:00
Tom Lane d08741eab5 Restructure the key include files per recent pghackers discussion: there
are now separate files "postgres.h" and "postgres_fe.h", which are meant
to be the primary include files for backend .c files and frontend .c files
respectively.  By default, only include files meant for frontend use are
installed into the installation include directory.  There is a new make
target 'make install-all-headers' that adds the whole content of the
src/include tree to the installed fileset, for use by people who want to
develop server-side code without keeping the complete source tree on hand.
Cleaned up a whole lot of crufty and inconsistent header inclusions.
2001-02-10 02:31:31 +00:00
Tom Lane 2cf48ca04b Extend CREATE DATABASE to allow selection of a template database to be
cloned, rather than always cloning template1.  Modify initdb to generate
two identical databases rather than one, template0 and template1.
Connections to template0 are disallowed, so that it will always remain
in its virgin as-initdb'd state.  pg_dumpall now dumps databases with
restore commands that say CREATE DATABASE foo WITH TEMPLATE = template0.
This allows proper behavior when there is user-added data in template1.
initdb forced!
2000-11-14 18:37:49 +00:00
Tatsuo Ishii 1acf6f9c8e Add support for code conversion between Unicode and other encodings.
Supported encodings are: EUC_JP, EUC_CN, EUC_KR, EUC_TW, Shift JIS,
Big5, ISO8859-[1-5].
TODO: testings! and documentations...
2000-10-30 10:41:05 +00:00
Tatsuo Ishii de53ce8131 Support for conversion between UNICODE and other encodings
currently ISO8859-[1-5] and EUC_JP are supported.
support for other encodings will be coming soon.
2000-10-12 06:06:50 +00:00
Tatsuo Ishii bfdd6a716d Change pg_mblen and pg_encoding_mblen return types from void
to int so that they return the number of whcars.
2000-08-27 10:40:48 +00:00
Tom Lane f2d1205322 Another batch of fmgr updates. I think I have gotten all old-style
functions that take pass-by-value datatypes.  Should be ready for
port testing ...
2000-06-13 07:35:40 +00:00
Tom Lane 4644fc8071 Eliminate query length limitation imposed by pg_client_to_server
and pg_server_to_client.  Eliminate copy.c's restriction on the length
of a single attribute.
1999-09-11 22:28:11 +00:00
Bruce Momjian 3406901a29 Move some system includes into c.h, and remove duplicates. 1999-07-17 20:18:55 +00:00
Bruce Momjian 33e826d167 Fix for multi-byte includes. 1999-07-17 16:25:28 +00:00
Bruce Momjian 07842084fe pgindent run over code. 1999-05-25 16:15:34 +00:00
Bruce Momjian a7ad43cd18 Included patches make some enhancements to the multi-byte support.
o allow to use Big5 (a Chinese encoding used in Taiwan) as a client
  encoding. In this case the server side encoding should be EUC_TW

o add EUC_TW and Big5 test cases to the regression and the mb test
  (contributed by Jonah Kuo)

o fix mistake in include/mb/pg_wchar.h. An encoding id for EUC_TW was
  not correct (was 3 and now is 4)

o update documents (doc/README.mb and README.mb.jp)

o update psql helpfile (bin/psql/psqlHelp.h)

--
Tatsuo Ishii
t-ishii@sra.co.jp
1999-02-02 18:51:40 +00:00
Bruce Momjian f52e7346ea MB patches from Tatsuo Ishii 1998-09-25 01:46:25 +00:00
Bruce Momjian fa1a8d6a97 OK, folks, here is the pgindent output. 1998-09-01 04:40:42 +00:00
Marc G. Fournier 5979d73841 From: t-ishii@sra.co.jp
As Bruce mentioned, this is due to the conflict among changes we made.
Included patches should fix the problem(I changed all MB to
MULTIBYTE). Please let me know if you have further problem.

P.S. I did not include pathces to configure and gram.c to save the
file size(configure.in and gram.y modified).
1998-07-26 04:31:41 +00:00
Marc G. Fournier bf00bbb0c4 I really hope that I haven't missed anything in this one...
From: t-ishii@sra.co.jp

Attached are patches to enhance the multi-byte support.  (patches are
against 7/18 snapshot)

* determine encoding at initdb/createdb rather than compile time

Now initdb/createdb has an option to specify the encoding. Also, I
modified the syntax of CREATE DATABASE to accept encoding option. See
README.mb for more details.

For this purpose I have added new column "encoding" to pg_database.
Also pg_attribute and pg_class are changed to catch up the
modification to pg_database.  Actually I haved added pg_database_mb.h,
pg_attribute_mb.h and pg_class_mb.h. These are used only when MB is
enabled. The reason having separate files is I couldn't find a way to
use ifdef or whatever in those files. I have to admit it looks
ugly. No way.

* support for PGCLIENTENCODING when issuing COPY command

commands/copy.c modified.

* support for SQL92 syntax "SET NAMES"

See gram.y.

* support for LATIN2-5
* add UNICODE regression test case
* new test suite for MB

New directory test/mb added.

* clean up source files

Basic idea is to have MB's own subdirectory for easier maintenance.
These are include/mb and backend/utils/mb.
1998-07-24 03:32:46 +00:00