Commit Graph

228 Commits

Author SHA1 Message Date
Tom Lane d78397d301 Change typreceive function API so that receive functions get the same
optional arguments as text input functions, ie, typioparam OID and
atttypmod.  Make all the datatypes that use typmod enforce it the same
way in typreceive as they do in typinput.  This fixes a problem with
failure to enforce length restrictions during COPY FROM BINARY.
2005-07-10 21:14:00 +00:00
Tatsuo Ishii e2d088de03 Allow direct conversion between EUC_JP and SJIS to improve
performance. patches submitted by Atsushi Ogawa.
2005-06-24 13:56:39 +00:00
Bruce Momjian 5955945828 Support 3 and 4-byte unicode characters.
John Hansen
2005-06-15 00:15:08 +00:00
Tatsuo Ishii b4cbd60fcf Fix bug in MIC -> EUC_JP conversion. Per Atsushi Ogawa. 2005-06-10 16:43:56 +00:00
Tom Lane 893b57c871 Alter the signature for encoding conversion functions to declare the
output area as INTERNAL not CSTRING.  This is to prevent people from
calling the functions by hand.  This is a permanent solution for the
back branches but I hope it is just a stopgap for HEAD.
2005-05-03 19:17:59 +00:00
Bruce Momjian e7fb9f18bf Add support for Win1252 encoding.
Roland Volkmann
2005-03-14 18:31:25 +00:00
Bruce Momjian 41e2a80f57 Update comments for new encoding names. 2005-03-14 00:19:13 +00:00
Bruce Momjian ee1bd33dd0 Document aliases for our supported encodings.
Add a few encodings that were not documented.
2005-03-13 01:26:30 +00:00
Neil Conway 4cd2fd66f8 Unbreak out-of-tree builds, by fixing a typo. 2005-03-07 23:18:06 +00:00
Bruce Momjian e3d7de6b99 Rename canonical encodings, per Peter:
UNICODE => UTF8
	ALT => WIN866
	WIN => WIN1251
	TCVN => WIN1258

The old codes continue to work.
2005-03-07 04:30:55 +00:00
Tom Lane 7e1c8ef4fc Some more missed copyright notices. Many of these look like they
should have been caught by the src/tools/copyright script ... why
weren't they?
2005-01-01 20:44:34 +00:00
PostgreSQL Daemon 2ff501590b Tag appropriate files for rc3
Also performed an initial run through of upgrading our Copyright date to
extend to 2005 ... first run here was very simple ... change everything
where: grep 1996-2004 && the word 'Copyright' ... scanned through the
generated list with 'less' first, and after, to make sure that I only
picked up the right entries ...
2004-12-31 22:04:05 +00:00
Bruce Momjian e09567d850 Back out addition of Win1252 encoding. 2004-12-04 18:19:33 +00:00
Bruce Momjian 08e0b34bad Back out fix for Unicode characters above 0x10000 2004-12-03 01:20:33 +00:00
Bruce Momjian 4ea4f8bd06 Fix for Unicode characters above 0x10000.
John Hansen
2004-12-02 22:37:14 +00:00
Bruce Momjian 7af770d005 Add Charset WIN1252 support.
Roland Volkmann
2004-12-02 22:14:38 +00:00
Neil Conway 7069dbcc31 More minor cosmetic improvements:
- remove another senseless "extern" keyword that was applied to a
function definition
- change a foo more function signatures from "some_type foo()" to
"some_type foo(void)"
- rewrite another K&R style function definition
- make the type of the "action" function pointer in the KeyWord struct
in src/backend/utils/adt/formatting.c more precise
2004-10-13 01:25:13 +00:00
Neil Conway 0e72b9d440 Cosmetic improvements/code cleanup:
- replace some function signatures of the form "some_type foo()" with
"some_type foo(void)"
- replace a few instances of a literal 0 being used as a NULL pointer;
there are more instances of this in the code, but I just fixed a few
- in src/backend/utils/mb/wstrncmp.c, replace K&R style function
declarations with ANSI style, remove use of 'register' keyword
- remove an "extern" modifier that was applied to a function definition
(rather than a declaration)
2004-10-10 23:37:45 +00:00
Bruce Momjian e1c8b37afb Add new macro as shorthand for MS VC and Borland C++:
+ #if   defined(_MSC_VER) || defined(__BORLANDC__)
+ #define       WIN32_CLIENT_ONLY
+ #endif
2004-09-27 23:24:45 +00:00
Peter Eisentraut 152a101f2b Allow WIN1250 as server encoding. 2004-09-17 21:59:57 +00:00
Bruce Momjian 15d3f9f6b7 Another pgindent run with lib typedefs added. 2004-08-30 02:54:42 +00:00
Bruce Momjian b6b71b85bc Pgindent run for 8.0. 2004-08-29 05:07:03 +00:00
Bruce Momjian da9a8649d8 Update copyright to 2004. 2004-08-29 04:13:13 +00:00
Tatsuo Ishii e8c3205037 Add PQmbdsplen() which returns the "display length" of a character.
Still some works needed:
- UTF-8, MULE_INTERNAL always returns 1
2004-03-15 10:41:26 +00:00
Tom Lane ecb156d484 If we don't have shared libraries, we don't have conversions. Make
conversion_create.sql be empty (except for a helpful comment) in this
case.  Allows initdb to succeed with --disable-shared.
2004-01-21 19:22:19 +00:00
Tom Lane a4f8f124b7 Fix bit-rot in support for building with --disable-shared. This patch
gets us past 'make install', but initdb still fails for lack of conversion
libraries ...
2004-01-21 19:04:11 +00:00
PostgreSQL Daemon 55b113257c make sure the $Id tags are converted to $PostgreSQL as well ... 2003-11-29 22:41:33 +00:00
PostgreSQL Daemon 969685ad44 $Header: -> $PostgreSQL Changes ... 2003-11-29 19:52:15 +00:00
Peter Eisentraut feb4f44d29 Message editing: remove gratuitous variations in message wording, standardize
terms, add some clarifications, fix some untranslatable attempts at dynamic
message building.
2003-09-25 06:58:07 +00:00
Tatsuo Ishii 0c9f978c0c Fix GB18030 to UTF-8 mapping table 2003-08-25 01:46:16 +00:00
Tatsuo Ishii b4ab39ff05 Fix GB18030 to UTF-8 mapping table 2003-08-24 05:18:04 +00:00
Peter Eisentraut 200b7d11af Fix uninstall target. 2003-08-23 04:22:34 +00:00
Tom Lane f65643771b Conversion functions must be STRICT to prevent them from getting null inputs. 2003-08-08 14:31:12 +00:00
Tom Lane 2f9c859ea1 Fix some copyright notices that weren't updated. Improve copyright tool
so it won't miss 'em again.
2003-08-04 23:59:41 +00:00
Bruce Momjian f3c3deb7d0 Update copyrights to 2003. 2003-08-04 02:40:20 +00:00
Bruce Momjian 089003fb46 pgindent run. 2003-08-04 00:43:34 +00:00
Tom Lane b6a1d25b0a Error message editing in utils/adt. Again thanks to Joe Conway for doing
the bulk of the heavy lifting ...
2003-07-27 04:53:12 +00:00
Tom Lane 689eb53e47 Error message editing in backend/utils (except /adt). 2003-07-25 20:18:01 +00:00
Bruce Momjian b14295cfe4 Attached is the complete diff against current CVS.
Compiles on BCC 5.5 and VC++ 6.0 (with warnings).

Karl Waclawek
2003-06-12 08:15:29 +00:00
Bruce Momjian dc4ee8a833 Back out patch that got bundled into another patch. 2003-06-12 08:11:07 +00:00
Bruce Momjian a647e30ba3 New patch with corrected README attached.
Also quickly added mention that it may be a qualified schema name.

Rod Taylor
2003-06-12 08:02:57 +00:00
Bruce Momjian 12c9423832 Allow Win32 to compile under MinGW. Major changes are:
Win32 port is now called 'win32' rather than 'win'
        add -lwsock32 on Win32
        make gethostname() be only used when kerberos4 is enabled
        use /port/getopt.c
        new /port/opendir.c routines
        disable GUC unix_socket_group on Win32
        convert some keywords.c symbols to KEYWORD_P to prevent conflict
        create new FCNTL_NONBLOCK macro to turn off socket blocking
        create new /include/port.h file that has /port prototypes, move
          out of c.h
        new /include/port/win32_include dir to hold missing include files
        work around ERROR being defined in Win32 includes
2003-05-15 16:35:30 +00:00
Tom Lane 351372e585 Department of second thoughts: probably still need an IsTransactionState
test in there...
2003-04-27 18:01:46 +00:00
Tom Lane 5f15fa8d06 Clean up some problems in SetClientEncoding: failed to honor doit flag
in all cases, leaked TopMemoryContext memory in others.  Make the
interaction between SetClientEncoding and InitializeClientEncoding
cleaner and better documented.  I suspect these changes should be
back-patched into 7.3, but will wait on Tatsuo's verification.
2003-04-27 17:31:25 +00:00
Tatsuo Ishii 35a0995992 Fix encoding conversion function bug.
See following posting for more details.

Subject: Re: [HACKERS] [BUGS] Bug #943: Server-Encoding from EUC_TW to UTF-8 doesn't
From: Tatsuo Ishii <t-ishii@sra.co.jp>
To: michael.enke@wincor-nixdorf.com, pgsql-bugs@postgresql.org
Cc: pgsql-hackers@postgresql.org
Date: Sat, 12 Apr 2003 10:51:45 +0900 (JST)
2003-04-12 07:53:57 +00:00
Tom Lane 1d650da2e5 This is a derived file and should never have been added to CVS. 2003-04-02 00:58:08 +00:00
Bruce Momjian 4b0b8dadd2 Add new files. 2003-03-27 16:53:15 +00:00
Tom Lane e4704001ea This patch fixes a bunch of spelling mistakes in comments throughout the
PostgreSQL source code.

Neil Conway
2003-03-10 22:28:22 +00:00
Tatsuo Ishii e2a618fe25 Fix for GUC client_encoding variable not being handled
correctly. See following thread for more details.

Subject: [HACKERS] client_encoding directive is ignored in postgresql.conf
From: Tatsuo Ishii <t-ishii@sra.co.jp>
Date: Wed, 29 Jan 2003 22:24:04 +0900 (JST)
2003-02-19 14:31:26 +00:00
Tom Lane b8add56ed0 Fix array subscript overruns identified by Yichen Xie. 2003-01-29 01:01:05 +00:00
Tatsuo Ishii 38535f8e32 Fix typo in an error message 2003-01-11 06:55:11 +00:00
Peter Eisentraut 4ed6be54e2 Fix Latin9/Unicode conversion by selecting the right table. 2002-12-09 19:47:21 +00:00
Bruce Momjian ceab6f7283 As far as I figured from the source code this function only deals with
cleaning up locale names and nothing else. Since all the locale names
are in plain  ASCII I think it will be safe to use ASCII-only lower-case
conversion.

Nicolai Tufar
2002-12-05 23:21:07 +00:00
Tatsuo Ishii ac47950238 Guard against 0 length string encoding conversion case. 2002-11-26 02:22:29 +00:00
Tatsuo Ishii 90a06dba16 Fix broken GB18030 <--> UTF-8 conversion map 2002-11-12 11:33:40 +00:00
Tom Lane 5123139210 Remove encoding lookups from grammar stage, push them back to places
where it's safe to do database access.  Along the way, fix core dump
for 'DEFAULT' parameters to CREATE DATABASE.  initdb forced due to
change in pg_proc entry.
2002-11-02 18:41:22 +00:00
Tom Lane 3518fbe86f Add missing semicolons to a few PG_FUNCTION_INFO_V1 calls. 2002-10-26 15:01:01 +00:00
Peter Eisentraut 8c3ab663ab Tweak conversion names to follow the established naming scheme, and
document that scheme.
2002-09-24 20:14:59 +00:00
Tatsuo Ishii 4b23f05c4f Fix bug in encoding conversion map. 2002-09-18 02:10:10 +00:00
Tatsuo Ishii 4c0bdd1ba8 Update Japanese README so that it reflects the changes made to the
conversion function interface.
2002-09-18 01:21:28 +00:00
Tatsuo Ishii 3357577247 Change Assert(len > 0) to Assert(len >= 0)
Change PG_RETURN_INT32(0) to PG_RETURN_VOID()
2002-09-13 06:41:18 +00:00
Peter Eisentraut 337da0678a Assorted fixes for Cygwin:
Eliminate the mysterious games that the Cygwin build plays with the linker
flag variables.  DLLLIBS is gone, use SHLIB_LINK like everyone else.
Detect cygipc in configure, after the linker flags are set up, otherwise
configure might not work at all.

Make sure everything is covered by make clean.

Fix the build of the new conversion procedure modules.

Add new DLLIMPORT markers where required.

Finally, the compiler complains if we use an explicit
-I/usr/local/include, so don't do that.  Curiously, -L/usr/local/lib is
still necessary.
2002-09-05 18:28:46 +00:00
Bruce Momjian e50f52a074 pgindent run. 2002-09-04 20:31:48 +00:00
Tom Lane 07c67187bf Avoid multiple scans of utils/mb/conversion_procs/ subdirectories during
'make install'; there are enough of 'em that this slowed down the make
noticeably.  Ensure that 'all' is the default make target in all these
directories (defaulting to 'make install' is surprising and dangerous
IMHO).  Fix a couple small typos.
2002-09-04 15:45:50 +00:00
Tatsuo Ishii 97592e6a6c Refrect the changes to src/test/regress/sql/conversion.sql By Tom. 2002-09-04 02:42:34 +00:00
Peter Eisentraut 77f7763b55 Remove all traces of multibyte and locale options. Clean up comments
referring to "multibyte" where it really means character encoding.
2002-09-03 21:45:44 +00:00
Tatsuo Ishii ed7baeaf4d Remove #ifdef MULTIBYTE per hackers list discussion. 2002-08-29 07:22:30 +00:00
Bruce Momjian ff1793f036 Remove erroneous character from Makefile due to editor error. 2002-08-22 02:18:45 +00:00
Tom Lane b663f3443b Add a bunch of pseudo-types to replace the behavior formerly associated
with OPAQUE, as per recent pghackers discussion.  I still want to do some
more work on the 'cstring' pseudo-type, but I'm going to commit the bulk
of the changes now before the tree starts shifting under me ...
2002-08-22 00:01:51 +00:00
Bruce Momjian d46e3dc00f Changes made so new conversion Makefiles will build out of the source tree. 2002-08-21 21:33:55 +00:00
Tatsuo Ishii 10b374aecf Fix bug in pg_convert() per report from MaC.Yui.
It pfree() wrong pointer.
2002-08-19 04:08:08 +00:00
Tatsuo Ishii 538b101595 Fix memory leak in SetClientEncoding(). 2002-08-14 05:33:34 +00:00
Tatsuo Ishii 969e0246ed Add Cyrillic and other encodings for encoding conversion.
Patches submitted by Kaori Inaba (i-kaori@sra.co.jp).
2002-08-14 02:45:10 +00:00
Tatsuo Ishii 697b472099 Address build problems on cygwin and (hopefully) AIX. 2002-08-08 07:47:43 +00:00
Tatsuo Ishii 3c63578a7e Load and keep conversion function info when SET CLIENT_ENCODING TO is
executed to prevent database access while performing encoding
conversion.
2002-08-08 06:35:26 +00:00
Tatsuo Ishii 6206a880cf Add SQL99 CONVERT() function. 2002-08-06 05:40:47 +00:00
Tatsuo Ishii 0345f58496 Implement DROP CONVERSION
Add regression test
2002-07-25 10:07:13 +00:00
Tatsuo Ishii 19a20e04bd Add Japanese README explaining how to add new conversion.
English README will come soon...
2002-07-24 07:05:41 +00:00
Tatsuo Ishii 86270024ff Oops. Too much ifdef out. 2002-07-19 11:09:25 +00:00
Tatsuo Ishii 248cbb5796 Temporary ifdef out migrating functions to avoid compiler warnings. 2002-07-19 00:22:24 +00:00
Peter Eisentraut 85d2a629c6 Create directory before installing files. 2002-07-18 22:58:08 +00:00
Tatsuo Ishii eb335a034b I have committed many support files for CREATE CONVERSION. Default
conversion procs and conversions are added in initdb. Currently
supported conversions are:

UTF-8(UNICODE) <--> SQL_ASCII, ISO-8859-1 to 16, EUC_JP, EUC_KR,
		    EUC_CN, EUC_TW, SJIS, BIG5, GBK, GB18030, UHC,
		    JOHAB, TCVN

EUC_JP <--> SJIS
EUC_TW <--> BIG5
MULE_INTERNAL <--> EUC_JP, SJIS, EUC_TW, BIG5

Note that initial contents of pg_conversion system catalog are created
in the initdb process. So doing initdb required is ideal, it's
possible to add them to your databases by hand, however. To accomplish
this:

psql -f your_postgresql_install_path/share/conversion_create.sql your_database

So I did not bump up the version in cataversion.h.

TODO:
Add more conversion procs
Add [CASCADE|RESTRICT] to DROP CONVERSION
Add tuples to pg_depend
Add regression tests
Write docs
Add SQL99 CONVERT command?
--
Tatsuo Ishii
2002-07-18 02:02:30 +00:00
Tatsuo Ishii 3c7798f068 Add conversion procs for CREATE CONVERSION 2002-07-16 09:25:06 +00:00
Tatsuo Ishii 15378a53f8 Add support for GB18030 2002-06-14 03:30:56 +00:00
Tatsuo Ishii 14f72b9a4d Add GB18030 support. Contributed by Bill Huang <bill_huanghb@ybb.ne.jp>
(ODBC support has not been committed yet. left for Hiroshi...)
2002-06-13 08:30:22 +00:00
Bruce Momjian e6227fd0ec Add missing Unicode multibyte files. 2002-03-06 06:12:59 +00:00
Bruce Momjian 92288a1cf9 Change made to elog:
o  Change all current CVS messages of NOTICE to WARNING.  We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.

o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.

o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.

o Remove INFO from the client_min_messages options and add NOTICE.

Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.

Regression passed.
2002-03-06 06:10:59 +00:00
Bruce Momjian a8bd7e1c6e > Tatsuo Ishii wrote:
> > > > It was made to cope with encoding such as an Asian bloc in 7.2Beta2.
> > > >
> > > > Added ServerEncoding
> > > >         Korean (JOHAB), Thai (WIN874),
> > > >         Vietnamese (TCVN), Arabic (WIN1256)
> > > >
> > > > Added ClientEncoding
> > > >         Simplified Chinese (GBK), Korean (UHC)
> > > >
> > > >
> > > >
> http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2b2.newencoding.diff.tar.gz
> > > > (608K)
> > >
> > > Looks good.  I need some people to review this for me.
> >
> > For me they look good too. The only missing part is a
> > documentation. I will ask him to write it up. If he couldn't, I will
> > do it for him.
> > > The diff is 3mb
> > > but appears to address only additions to multibyte.  I have attached a
> > > list of files it modifies.  Also, look at the sizes of the mb/
> > > directory.  It is getting large:
> > >
> > >   4       ./CVS
> > >   6       ./Unicode/CVS
> > >   3433    ./Unicode
> > >   6197    .
> >
> > Yes. We definitely need the on-the-fly encoding addition capability:
> > i.e. CREATE CHRACTER SET in the future...
> > --
> > Tatsuo Ishii
> >
> >

Address chainge.

http://www.sankyo-unyu.co.jp/Pool/postgresql-7.2.newencoding.diff.gz

Add PsqlODBC and document ...etc patch.

Eiji Tokuya
2002-03-05 05:52:50 +00:00
Tatsuo Ishii 933761e7b1 Simplify pg_convert() in that it calls pg_convert2 using new fmgr interface. 2001-11-20 01:32:29 +00:00
Tatsuo Ishii 5590d5fe99 Fix nasty bugs in pg_convert() and pg_convert2().
o they sometimes returns a result garbage string appended.
    o they do not work if client encoding is different from server
      encoding
2001-11-19 06:48:39 +00:00
Bruce Momjian ea08e6cd55 New pgindent run with fixes suggested by Tom. Patch manually reviewed,
initdb/regression tests pass.
2001-11-05 17:46:40 +00:00
Bruce Momjian 6783b2372e Another pgindent run. Fixes enum indenting, and improves #endif
spacing.  Also adds space for one-line comments.
2001-10-28 06:26:15 +00:00
Bruce Momjian b81844b173 pgindent run on all C files. Java run to follow. initdb/regression
tests pass.
2001-10-25 05:50:21 +00:00
Tatsuo Ishii cfe01796e6 Ok, here is the modified encoding table (column1 is the standard name,
2 is our "official" name, and 3 is alias). If there's no objection, I
will change them.

ASCII		SQL_ASCII
UTF-8		UNICODE		UTF_8
MULE-INTERNAL	MULE_INTERNAL
ISO-8859-1	LATIN1		ISO_8859_1
ISO-8859-2	LATIN2		ISO_8859_2
ISO-8859-3	LATIN3		ISO_8859_3
ISO-8859-4	LATIN4		ISO_8859_4
ISO-8859-5	ISO_8859_5
ISO-8859-6	ISO_8859_6
ISO-8859-7	ISO_8859_7
ISO-8859-8	ISO_8859_8
ISO-8859-9	LATIN5		ISO_8859_9
ISO-8859-10	LATIN6		ISO_8859_10
ISO-8859-13	LATIN7		ISO_8859_13
ISO-8859-14	LATIN8		ISO_8859_14
ISO-8859-15	LATIN9		ISO_8859_15
ISO-8859-16	LATIN10		ISO_8859_16
2001-10-16 10:09:17 +00:00
Tatsuo Ishii d07bacd54a Add UTF-8 char >= 0x10000 check 2001-10-15 01:19:15 +00:00
Tatsuo Ishii f426465ba9 Add a new function "pg_client_encoding" which returns the current client
side encoding name. This is necessary for client API's such as JDBC
to perform correct encoding conversions. See my email "[HACKERS]
pg_client_encoding" 10 Sep 2001.
2001-10-12 02:08:34 +00:00
Tatsuo Ishii 51053d3216 Add support for ISO-8859-6 to 16 2001-10-11 14:20:35 +00:00
Tatsuo Ishii 1b20315008 Fix bug in mic2ascii(). It does not handle correctly if none ASCII
chars are in the input.
2001-09-25 01:27:03 +00:00
Tatsuo Ishii be629abfc8 Add pg_database_encoding_max_length() function. 2001-09-23 10:59:45 +00:00
Tatsuo Ishii 8ebdac0ed5 Remove test drivers
Also fix comment in conv.c.
2001-09-22 08:44:49 +00:00