From pgsql-patches-owner+M16987@postgresql.org Thu Aug 4 17:35:52 2005 Return-path: Received: from svr1.postgresql.org (svr1.postgresql.org [200.46.204.71]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j74LZot06577 for ; Thu, 4 Aug 2005 17:35:50 -0400 (EDT) Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id ED433529DB; Thu, 4 Aug 2005 18:35:44 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 70225-08; Thu, 4 Aug 2005 21:35:44 +0000 (GMT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by svr1.postgresql.org (Postfix) with ESMTP id 0ECB452998; Thu, 4 Aug 2005 18:35:44 -0300 (ADT) X-Original-To: pgsql-patches-postgresql.org@localhost.postgresql.org Received: from localhost (unknown [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id AFAF752840 for ; Thu, 4 Aug 2005 18:33:25 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 70920-01 for ; Thu, 4 Aug 2005 21:33:13 +0000 (GMT) Received: from mx-2.sollentuna.net (mx-2.sollentuna.net [195.84.163.199]) by svr1.postgresql.org (Postfix) with ESMTP id E1C935280D for ; Thu, 4 Aug 2005 18:33:11 -0300 (ADT) Received: from ALGOL.sollentuna.se (janus.sollentuna.se [62.65.68.67]) by mx-2.sollentuna.net (Postfix) with ESMTP id 7F5A08F289 for ; Thu, 4 Aug 2005 23:33:13 +0200 (CEST) content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----_=_NextPart_001_01C5993C.1E1CB100" Subject: [PATCHES] FW: Win32 unicode vs ICU X-MimeOLE: Produced By Microsoft Exchange V6.5.7226.0 Date: Thu, 4 Aug 2005 23:33:12 +0200 Message-ID: <6BCB9D8A16AC4241919521715F4D8BCE094656@algol.sollentuna.se> X-MS-Has-Attach: yes Thread-Topic: Win32 unicode vs ICU Thread-Index: AcWVyKf6M9GrCqZAQGeSvCZNg9n/jgDc1WwQ From: "Magnus Hagander" To: "PostgreSQL-patches" X-Virus-Scanned: by amavisd-new at hub.org X-Mailing-List: pgsql-patches List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-patches-owner@postgresql.org X-Virus-Scanned: by amavisd-new at hub.org Status: ORr This is a multi-part message in MIME format. ------_=_NextPart_001_01C5993C.1E1CB100 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable I just realised this mail didn't go through. Probably because it was too large for -hackers. So: repost to -patches. Sorry about that. If it's a duplicate, even more sorry, but I couldn't find it in the archives. (This may explain that nobody answered me :P) //Magnus =20 > -----Original Message----- > From: Magnus Hagander=20 > Sent: Sunday, July 31, 2005 2:09 PM > To: PostgreSQL-development > Cc: pgsql-hackers-win32@postgresql.org > Subject: Win32 unicode vs ICU >=20 > Hi! >=20 > I've been working with Palles ICU patch to make it work on=20 > win32, and I believe I have it done. While doing it I noticed=20 > that ICU basically converts to UTF16 and back - I previously=20 > thought it worked on UTF8 strings. Based on this I also tried=20 > out an implementation for the win32-unicode problem that does=20 > *not* require ICU. It uses the win32 native functions to map=20 > to utf16 and back, and then to process the text there. And I=20 > got through with much less code than the ICU version, while=20 > doing the same thing. >=20 > I am unsure of how to proceed. As I see it there are three paths: > 1) Use native win32 functionality only on win32 > 2) Use ICU functionality only on win32 > 3) Allow both ICU and native functionality, compile time=20 > switch --with-icu (same as unix with the ICU patch) >=20 >=20 > The main downsides of ICU vs the native ones are: > * ICU does not accept win32 locale names. When doing=20 > setlocale("sv_se"), for example, win32 will return this in=20 > later calls as "Swedish_Sweden.1252". To get around this in=20 > the ICU patch, I had to implement a lookup map that converts=20 > it back to sv_se for ICU. >=20 > * ICU is yet another build and runtime dependency, and a=20 > large one (comes in at 11Mb for the DLL files alone in the=20 > win32 download) >=20 >=20 > I guess that the main upside of it is that we'd get=20 > constistent behaviour - in case there are issues with either=20 > ICU or win32 native they'd otherwise differ. And only one new=20 > codepath. But we already live with the platform-inconsistency today... >=20 > Another upside is that it handles more encodings in ICU - my=20 > native implementation does *only* UTF8 and relies on existing=20 > functionality to deal with other encodings. It could of=20 > course be extended if necessary, but from what I can tell=20 > UTF8 is the big one. >=20 >=20 >=20 > I have attached both patches. For the native version, only=20 > win32_utf8.patch is required. For the ICU version,=20 > icu_win32.patch is needed and also the files=20 > localemap.c,localemap.pl, iso639 and iso3166 needs to go in=20 > src/backend/port/win32. (the localemap needs to be updated to=20 > do a better-than-linear search, but I wanted to include an example) >=20 >=20 > Thoughts on the options? >=20 >=20 > And anohter question - my native patch touches the same=20 > functions as the ICU patch. Can somebody who knows the=20 > internals confirm or deny that these are all the required=20 > locations, or do we need to modify more? >=20 > (I have run simple tests in swedish locale and both behave=20 > the same and correct, but I'm unsure of exactly how much=20 > would be affected) >=20 > Finally, the win32 patch also changes the normal path to use=20 > strncoll(). The comment above the function states that we'd=20 > like to use strncoll but it's not available. Well, on win32=20 > it is, so it should provide a speedup on win32. It is=20 > currently not included in the ICU patch, but should probably=20 > be included whichever path we'd chose. >=20 >=20 > //Magnus >=20 ------_=_NextPart_001_01C5993C.1E1CB100 Content-Type: application/octet-stream; name="win32_utf8.patch" Content-Transfer-Encoding: base64 Content-Description: win32_utf8.patch Content-Disposition: attachment; filename="win32_utf8.patch" SW5kZXg6IHNyYy9iYWNrZW5kL3V0aWxzL2FkdC9vcmFjbGVfY29tcGF0LmMN Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvcHJvamVjdHMv Y3Zzcm9vdC9wZ3NxbC9zcmMvYmFja2VuZC91dGlscy9hZHQvb3JhY2xlX2Nv bXBhdC5jLHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS42MA0KZGlmZiAtYyAt cjEuNjAgb3JhY2xlX2NvbXBhdC5jDQoqKiogc3JjL2JhY2tlbmQvdXRpbHMv YWR0L29yYWNsZV9jb21wYXQuYwk3IE1heSAyMDA1IDE1OjE4OjE3IC0wMDAw CTEuNjANCi0tLSBzcmMvYmFja2VuZC91dGlscy9hZHQvb3JhY2xlX2NvbXBh dC5jCTMxIEp1bCAyMDA1IDExOjExOjI4IC0wMDAwDQoqKioqKioqKioqKioq KioNCioqKiAxNDksMTU0ICoqKioNCi0tLSAxNDksMjA5IC0tLS0NCiAgI2Vu ZGlmICAgLyogVVNFX1dJREVfVVBQRVJfTE9XRVIgKi8NCiAgDQogIA0KKyAj aWZkZWYgV0lOMzINCisgLyogV2luMzIgVVRGOC9VVEYxNiBjb252ZXJzaW9u IGhlbHBlcnMgKi8NCisgc3RhdGljIHdjaGFyX3QgKndpbjMyX3V0Zjh0ZXh0 dG91dGYxNihjb25zdCB0ZXh0ICp0eHQpDQorIHsNCisgCWludAkJCW5ieXRl cyA9IFZBUlNJWkUodHh0KSAtIFZBUkhEUlNaOwkNCisgCXdjaGFyX3QgICAg KnJlc3VsdDsNCisgCWludCAgICAgICAgIHI7DQorIA0KKyAJaWYgKG5ieXRl cyA8IDAgfHwNCisgCQkJbmJ5dGVzID4gKGludCkgKElOVF9NQVggLyBzaXpl b2Yod2NoYXJfdCkpIC0xKQ0KKyAJCWVyZXBvcnQoRVJST1IsDQorIAkJCQko ZXJyY29kZShFUlJDT0RFX09VVF9PRl9NRU1PUlkpLA0KKyAJCQkJIGVycm1z Zygib3V0IG9mIG1lbW9yeSIpKSk7DQorIA0KKyAJLyogT3V0cHV0IHdvcmtz cGFjZSBjYW5ub3QgaGF2ZSBtb3JlIGNvZGVzIHRoYW4gaW5wdXQgYnl0ZXMg Ki8NCisgCXJlc3VsdCA9ICh3Y2hhcl90ICopIHBhbGxvYygobmJ5dGVzICsg MSkgKiBzaXplb2Yod2NoYXJfdCkpOw0KKyANCisgCS8qIERvIHRoZSBjb252 ZXJzaW9uICovDQorIAlyID0gTXVsdGlCeXRlVG9XaWRlQ2hhcihDUF9VVEY4 LCAwLCBWQVJEQVRBKHR4dCksICBuYnl0ZXMsDQorIAkJCQlyZXN1bHQsIG5i eXRlcyAqIHNpemVvZih3Y2hhcl90KSk7DQorIAlpZiAoIXIpDQorIAkJZXJl cG9ydChFUlJPUiwNCisgCQkJCShlcnJjb2RlKEVSUkNPREVfQ0hBUkFDVEVS X05PVF9JTl9SRVBFUlRPSVJFKSwNCisgCQkJCSBlcnJtc2coImludmFsaWQg bXVsdGlieXRlIGNoYXJhY3RlciBmb3IgbG9jYWxlIiksDQorIAkJCQkgZXJy aGludCgiVGhlIHNlcnZlcidzIExDX0NUWVBFIGxvY2FsZSBpcyBwcm9iYWJs eSBpbmNvbXBhdGlibGUgd2l0aCB0aGUgZGF0YWJhc2UgZW5jb2RpbmcuIikp KTsNCisgDQorIAlyZXN1bHRbcl0gPSAwOwkNCisgCXJldHVybiByZXN1bHQ7 DQorIH0NCisgDQorIHN0YXRpYyB0ZXh0ICp3aW4zMl91dGYxNnRvdXRmOHRl eHQoY29uc3Qgd2NoYXJfdCAqdHh0KQ0KKyB7DQorIAl0ZXh0CQkqcmVzdWx0 Ow0KKyAJaW50CQkJIG5ieXRlczsNCisgCWludAkJCSByOw0KKyAJDQorIAlu Ynl0ZXMgPSBXaWRlQ2hhclRvTXVsdGlCeXRlKENQX1VURjgsIDAsIHR4dCwg LTEsIE5VTEwsIDAsIE5VTEwsIE5VTEwpOw0KKyAJaWYgKG5ieXRlcyA9PSAw KQ0KKyAJCWVyZXBvcnQoRVJST1IsDQorIAkJCQkoZXJyY29kZShFUlJDT0RF X0NIQVJBQ1RFUl9OT1RfSU5fUkVQRVJUT0lSRSksDQorIAkJCQkgZXJybXNn KCJpbnZhbGlkIHV0ZjE2IGNoYXJhY3RlciBmb3IgbG9jYWxlOiAlbHUiLEdl dExhc3RFcnJvcigpKSkpOw0KKyANCisgCXJlc3VsdCA9IHBhbGxvYyhuYnl0 ZXMrVkFSSERSU1opOw0KKyANCisgCXIgPSBXaWRlQ2hhclRvTXVsdGlCeXRl KENQX1VURjgsIDAsIHR4dCwgLTEsIFZBUkRBVEEocmVzdWx0KSwgbmJ5dGVz LCBOVUxMLCBOVUxMKTsNCisgCWlmIChyID09IDApDQorIAkJZXJlcG9ydChF UlJPUiwNCisgCQkJCShlcnJjb2RlKEVSUkNPREVfQ0hBUkFDVEVSX05PVF9J Tl9SRVBFUlRPSVJFKSwNCisgCQkJCSBlcnJtc2coImludmFsaWQgdXRmMTYg Y2hhcmFjdGVyIGZvciBsb2NhbGU6ICVsdSIsR2V0TGFzdEVycm9yKCkpKSk7 DQorIA0KKyAJVkFSQVRUX1NJWkVQKHJlc3VsdCkgPSBuYnl0ZXMgKyBWQVJI RFJTWiAtIDE7IC8qIC0xID0gaWdub3JlIG51bGwgKi8NCisgCXJldHVybiBy ZXN1bHQ7DQorIH0NCisgI2VuZGlmIC8qIFdJTjMyICovDQorIA0KICAvKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqKioNCiAgICoNCiAgICogbG93ZXINCioqKioq KioqKioqKioqKg0KKioqIDE3OSwxODQgKioqKg0KLS0tIDIzNCwyNTEgLS0t LQ0KICAJCXdjaGFyX3QgICAgKndvcmtzcGFjZTsNCiAgCQlpbnQJCQlpOw0K ICANCisgI2lmZGVmIFdJTjMyDQorIAkJLyogV2luMzIgZG9lcyBub3QgaGF2 ZSBVVEYtOCwgc28gd2UgbmVlZCB0byBtYXAgdG8gVVRGLTE2ICovDQorIAkJ aWYgKEdldERhdGFiYXNlRW5jb2RpbmcoKSA9PSBQR19VVEY4KQ0KKyAJCXsN CisgCQkJd29ya3NwYWNlID0gd2luMzJfdXRmOHRleHR0b3V0ZjE2KHN0cmlu Zyk7DQorIAkJCV93Y3Nsd3Iod29ya3NwYWNlKTsNCisgCQkJcmVzdWx0ID0g d2luMzJfdXRmMTZ0b3V0Zjh0ZXh0KHdvcmtzcGFjZSk7DQorIAkJCXBmcmVl KHdvcmtzcGFjZSk7DQorIAkJCVBHX1JFVFVSTl9URVhUX1AocmVzdWx0KTsN CisgCQl9DQorICNlbmRpZg0KKyAJCQ0KICAJCXdvcmtzcGFjZSA9IHRleHR0 b3djcyhzdHJpbmcpOw0KICANCiAgCQlmb3IgKGkgPSAwOyB3b3Jrc3BhY2Vb aV0gIT0gMDsgaSsrKQ0KKioqKioqKioqKioqKioqDQoqKiogMjQ1LDI1MCAq KioqDQotLS0gMzEyLDMyOSAtLS0tDQogIAkJd2NoYXJfdCAgICAqd29ya3Nw YWNlOw0KICAJCWludAkJCWk7DQogIA0KKyAjaWZkZWYgV0lOMzINCisgCQkv KiBXaW4zMiBkb2VzIG5vdCBoYXZlIFVURi04LCBzbyB3ZSBuZWVkIHRvIG1h cCB0byBVVEYtMTYgKi8NCisgCQlpZiAoR2V0RGF0YWJhc2VFbmNvZGluZygp ID09IFBHX1VURjgpDQorIAkJew0KKyAJCQl3b3Jrc3BhY2UgPSB3aW4zMl91 dGY4dGV4dHRvdXRmMTYoc3RyaW5nKTsNCisgCQkJX3djc3Vwcih3b3Jrc3Bh Y2UpOw0KKyAJCQlyZXN1bHQgPSB3aW4zMl91dGYxNnRvdXRmOHRleHQod29y a3NwYWNlKTsNCisgCQkJcGZyZWUod29ya3NwYWNlKTsNCisgCQkJUEdfUkVU VVJOX1RFWFRfUChyZXN1bHQpOw0KKyAJCX0NCisgI2VuZGlmDQorIA0KICAJ CXdvcmtzcGFjZSA9IHRleHR0b3djcyhzdHJpbmcpOw0KICANCiAgCQlmb3Ig KGkgPSAwOyB3b3Jrc3BhY2VbaV0gIT0gMDsgaSsrKQ0KKioqKioqKioqKioq KioqDQoqKiogMzE1LDMyMCAqKioqDQotLS0gMzk0LDQwNSAtLS0tDQogIAkJ aW50CQkJd2FzYWxudW0gPSAwOw0KICAJCWludAkJCWk7DQogIA0KKyAjaWZk ZWYgV0lOMzINCisgCQkvKiBXaW4zMiBkb2VzIG5vdCBoYXZlIFVURi04LCBz byB3ZSBuZWVkIHRvIG1hcCB0byBVVEYtMTYgKi8NCisgCQlpZiAoR2V0RGF0 YWJhc2VFbmNvZGluZygpID09IFBHX1VURjgpDQorIAkJCXdvcmtzcGFjZSA9 IHdpbjMyX3V0Zjh0ZXh0dG91dGYxNihzdHJpbmcpOw0KKyAJCWVsc2UNCisg I2VuZGlmDQogIAkJd29ya3NwYWNlID0gdGV4dHRvd2NzKHN0cmluZyk7DQog IA0KICAJCWZvciAoaSA9IDA7IHdvcmtzcGFjZVtpXSAhPSAwOyBpKyspDQoq KioqKioqKioqKioqKioNCioqKiAzMjYsMzMxICoqKioNCi0tLSA0MTEsNDIy IC0tLS0NCiAgCQkJd2FzYWxudW0gPSBpc3dhbG51bSh3b3Jrc3BhY2VbaV0p Ow0KICAJCX0NCiAgDQorICNpZmRlZiBXSU4zMg0KKyAJCS8qIE1hcCBiYWNr IHRvIFVURi04ICovDQorIAkJaWYgKEdldERhdGFiYXNlRW5jb2RpbmcoKSA9 PSBQR19VVEY4KQ0KKyAJCQlyZXN1bHQgPSB3aW4zMl91dGYxNnRvdXRmOHRl eHQod29ya3NwYWNlKTsNCisgCQllbHNlDQorICNlbmRpZg0KICAJCXJlc3Vs dCA9IHdjc3RvdGV4dCh3b3Jrc3BhY2UsIGkpOw0KICANCiAgCQlwZnJlZSh3 b3Jrc3BhY2UpOw0KSW5kZXg6IHNyYy9iYWNrZW5kL3V0aWxzL2FkdC92YXJs ZW5hLmMNCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0NClJDUyBmaWxlOiAvcHJv amVjdHMvY3Zzcm9vdC9wZ3NxbC9zcmMvYmFja2VuZC91dGlscy9hZHQvdmFy bGVuYS5jLHYNCnJldHJpZXZpbmcgcmV2aXNpb24gMS4xMzANCmRpZmYgLWMg LXIxLjEzMCB2YXJsZW5hLmMNCioqKiBzcmMvYmFja2VuZC91dGlscy9hZHQv dmFybGVuYS5jCTI5IEp1bCAyMDA1IDAzOjE3OjU1IC0wMDAwCTEuMTMwDQot LS0gc3JjL2JhY2tlbmQvdXRpbHMvYWR0L3ZhcmxlbmEuYwkzMSBKdWwgMjAw NSAxMToxNTo0OSAtMDAwMA0KKioqKioqKioqKioqKioqDQoqKiogODQ4LDg1 MyAqKioqDQotLS0gODQ4LDkwMSAtLS0tDQogIAkJY2hhcgkJYTJidWZbU1RB Q0tCVUZMRU5dOw0KICAJCWNoYXIJICAgKmExcCwNCiAgCQkJCSAgICphMnA7 DQorICNpZmRlZiBXSU4zMg0KKyAJCS8qIFdpbjMyIGRvZXMgbm90IGhhdmUg VVRGLTgsIHNvIHdlIG5lZWQgdG8gbWFwIHRvIFVURi0xNiAqLw0KKyAJCWlm IChHZXREYXRhYmFzZUVuY29kaW5nKCkgPT0gUEdfVVRGOCkNCisgCQl7DQor IAkJCWludCBhMWxlbiA9IFNUQUNLQlVGTEVOOw0KKyAJCQlpbnQgYTJsZW4g PSBTVEFDS0JVRkxFTjsNCisgDQorIAkJCWlmIChsZW4xID49IFNUQUNLQlVG TEVOLzIpDQorIAkJCXsNCisgCQkJCWExbGVuID0gbGVuMSAqIDIgKyAyOw0K KyAJCQkJYTFwID0gcGFsbG9jKGExbGVuKTsNCisgCQkJfQ0KKyAJCQllbHNl DQorIAkJCQlhMXAgPSBhMWJ1ZjsNCisgCQkJDQorIAkJCWlmIChsZW4yID49 IFNUQUNLQlVGTEVOLzIpDQorIAkJCXsNCisgCQkJCWEybGVuID0gbGVuMiAq IDIgKyAyOw0KKyAJCQkJYTJwID0gcGFsbG9jKGEybGVuKTsNCisgCQkJfQ0K KyAJCQllbHNlDQorIAkJCQlhMnAgPSBhMmJ1ZjsNCisgDQorIAkJCWlmICgh TXVsdGlCeXRlVG9XaWRlQ2hhcihDUF9VVEY4LCAwLCBhcmcxLCAtMSwgKExQ V1NUUilhMXAsIGExbGVuLzIpKQ0KKyAJCQkJZXJlcG9ydChFUlJPUiwNCisg CQkJCQkJKGVycm1zZygiZmFpbGVkIHRvIGNvbnZlcnQgc3RyaW5nIHRvIFVU RjE2OiAlbHUiLA0KKyAJCQkJCQkJR2V0TGFzdEVycm9yKCkpKSk7DQorIAkJ CWlmICghTXVsdGlCeXRlVG9XaWRlQ2hhcihDUF9VVEY4LCAwLCBhcmcyLCAt MSwgKExQV1NUUilhMnAsIGEybGVuLzIpKQ0KKyAJCQkJZXJlcG9ydChFUlJP UiwNCisgCQkJCQkJKGVycm1zZygiZmFpbGVkIHRvIGNvbnZlcnQgc3RyaW5n IHRvIFVURjE2OiAlbHUiLA0KKyAJCQkJCQkJR2V0TGFzdEVycm9yKCkpKSk7 DQorIAkJCQ0KKyAJCQllcnJubz0wOw0KKyAJCQlyZXN1bHQgPSB3Y3Njb2xs KChMUFdTVFIpYTFwLCAoTFBXU1RSKWEycCk7DQorIAkJCWlmIChyZXN1bHQg PT0gMjE0NzQ4MzY0NykgLyogX05MU0NNUEVSUk9SOyBtaXNzaW5nIGZyb20g bWluZ3cgaGVhZGVycyAqLw0KKyAJCQkJZXJlcG9ydChFUlJPUiwNCisgCQkJ CQkJKGVycm1zZygiZmFpbGVkIHRvIGNvbXBhcmUgdW5pY29kZSBzdHJpbmdz OiAlaSIsIGVycm5vKSkpOw0KKyAJCQlpZiAoYTFsZW4gIT0gU1RBQ0tCVUZM RU4pDQorIAkJCQlwZnJlZShhMXApOw0KKyAJCQlpZiAoYTJsZW4gIT0gU1RB Q0tCVUZMRU4pDQorIAkJCQlwZnJlZShhMnApOw0KKyAJCQkNCisgCQkJcmV0 dXJuIHJlc3VsdDsNCisgCQl9DQorIA0KKyAJCS8qIFdpbjMyIGhhcyBzdHJu Y29sbCgpLCBzbyB1c2UgaXQgdGhlcmUgdG8gYXZvaWQgY29weWluZyAqLw0K KyAJCXJldHVybiBfc3RybmNvbGwoYXJnMSwgYXJnMiwgKGxlbjE+bGVuMik/ bGVuMjpsZW4xICk7DQorICNlbmRpZg0KICANCiAgCQlpZiAobGVuMSA+PSBT VEFDS0JVRkxFTikNCiAgCQkJYTFwID0gKGNoYXIgKikgcGFsbG9jKGxlbjEg KyAxKTsNCg== ------_=_NextPart_001_01C5993C.1E1CB100 Content-Type: application/octet-stream; name="icu_win32.patch" Content-Transfer-Encoding: base64 Content-Description: icu_win32.patch Content-Disposition: attachment; filename="icu_win32.patch" SW5kZXg6IGNvbmZpZ3VyZS5pbg0KPT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0K UkNTIGZpbGU6IC9wcm9qZWN0cy9jdnNyb290L3Bnc3FsL2NvbmZpZ3VyZS5p bix2DQpyZXRyaWV2aW5nIHJldmlzaW9uIDEuNDE3DQpkaWZmIC1jIC1yMS40 MTcgY29uZmlndXJlLmluDQoqKiogY29uZmlndXJlLmluCTYgSnVsIDIwMDUg MjE6MDQ6MTMgLTAwMDAJMS40MTcNCi0tLSBjb25maWd1cmUuaW4JMjggSnVs IDIwMDUgMTc6MDE6NDcgLTAwMDANCioqKioqKioqKioqKioqKg0KKioqIDQ2 Nyw0NzIgKioqKg0KLS0tIDQ2Nyw0ODAgLS0tLQ0KICBBQ19NU0dfUkVTVUxU KFskd2l0aF9vcGVuc3NsXSkNCiAgQUNfU1VCU1Qod2l0aF9vcGVuc3NsKQ0K ICANCisgIw0KKyAjIElDVQ0KKyAjDQorIEFDX01TR19DSEVDS0lORyhbd2hl dGhlciB0byBidWlsZCB3aXRoIElDVSBzdXBwb3J0XSkNCisgUEdBQ19BUkdf Qk9PTCh3aXRoLCBpY3UsIG5vLCBbICAtLXdpdGgtaWN1ICAgICAgICAgICAg ICBidWlsZCB3aXRoIElDVSBzdXBwb3J0XSwNCisgICAgICAgICAgICAgICBb QUNfREVGSU5FKFtVU0VfSUNVXSwgMSwgW0RlZmluZSB0byBidWlsZCB3aXRo IElDVSBzdXBwb3J0LiAoLS13aXRoLWljdSldKV0pDQorIEFDX01TR19SRVNV TFQoWyR3aXRoX2ljdV0pDQorIEFDX1NVQlNUKHdpdGhfaWN1KQ0KICANCiAg Iw0KICAjIFJlYWRsaW5lDQoqKioqKioqKioqKioqKioNCioqKiA2NzQsNjc5 ICoqKioNCi0tLSA2ODIsNjk4IC0tLS0NCiAgICBmaQ0KICBmaQ0KICANCisg aWYgdGVzdCAiJHdpdGhfaWN1IiA9IHllcyA7IHRoZW4NCisgICBpZiB0ZXN0 ICIkUE9SVE5BTUUiICE9ICJ3aW4zMiI7IHRoZW4NCisgICAgIEFDX0NIRUNL X0xJQihpY3VpMThuLCB1Y29sX29wZW5fM18yLCBbXSwgW0FDX01TR19FUlJP UihbbGlicmFyeSAnaWN1aTE4bicgaXMgcmVxdWlyZWQgZm9yIElDVV0pXSkN CisgICAgIEFDX0NIRUNLX0xJQihpY3V1YywgIHVfdG9sb3dlcl8zXzIsIFtd LCBbQUNfTVNHX0VSUk9SKFtsaWJyYXJ5ICdpY3V1YycgaXMgcmVxdWlyZWQg Zm9yIElDVV0pXSkNCisgICAgIEFDX0NIRUNLX0xJQihpY3VkYXRhLCBpY3Vk dDMyX2RhdCwgW10sIFtBQ19NU0dfRVJST1IoW2xpYnJhcnkgJ2ljdWRhdGEn IGlzIHJlcXVpcmVkIGZvciBJQ1VdKV0pDQorICAgZWxzZQ0KKyAgICAgQUNf Q0hFQ0tfTElCKGljdWluLCB1Y29sX29wZW5fM18yLCBbXSwgW0FDX01TR19F UlJPUihbbGlicmFyeSAnaWN1aW4nIGlzIHJlcXVpcmVkIGZvciBJQ1VdKV0p DQorICAgICBBQ19DSEVDS19MSUIoaWN1dWMsICB1X3RvbG93ZXJfM18yLCBb XSwgW0FDX01TR19FUlJPUihbbGlicmFyeSAnaWN1dWMnIGlzIHJlcXVpcmVk IGZvciBJQ1VdKV0pDQorICAgZmkNCisgZmkNCisgICANCiAgaWYgdGVzdCAi JHdpdGhfcGFtIiA9IHllcyA7IHRoZW4NCiAgICBBQ19DSEVDS19MSUIocGFt LCAgICBwYW1fc3RhcnQsIFtdLCBbQUNfTVNHX0VSUk9SKFtsaWJyYXJ5ICdw YW0nIGlzIHJlcXVpcmVkIGZvciBQQU1dKV0pDQogIGZpDQoqKioqKioqKioq KioqKioNCioqKiA3NDgsNzUzICoqKioNCi0tLSA3NjcsNzc2IC0tLS0NCiAg ICBBQ19DSEVDS19IRUFERVIob3BlbnNzbC9lcnIuaCwgW10sIFtBQ19NU0df RVJST1IoW2hlYWRlciBmaWxlIDxvcGVuc3NsL2Vyci5oPiBpcyByZXF1aXJl ZCBmb3IgT3BlblNTTF0pXSkNCiAgZmkNCiAgDQorIGlmIHRlc3QgIiR3aXRo X2ljdSIgPSB5ZXMgOyB0aGVuDQorICAgQUNfQ0hFQ0tfSEVBREVSKHVuaWNv ZGUvdXR5cGVzLmgsIFtdLCBbQUNfTVNHX0VSUk9SKFtoZWFkZXIgZmlsZSA8 dW5pY29kZS91dHlwZXMuaD4gaXMgcmVxdWlyZWQgZm9yIElDVV0pXSkNCisg ZmkNCisgIA0KICBpZiB0ZXN0ICIkd2l0aF9wYW0iID0geWVzIDsgdGhlbg0K ICAgIEFDX0NIRUNLX0hFQURFUlMoc2VjdXJpdHkvcGFtX2FwcGwuaCwgW10s DQogICAgICAgICAgICAgICAgICAgICBbQUNfQ0hFQ0tfSEVBREVSUyhwYW0v cGFtX2FwcGwuaCwgW10sDQpJbmRleDogc3JjL2JhY2tlbmQvcG9ydC93aW4z Mi9NYWtlZmlsZQ0KPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQ0KUkNTIGZpbGU6 IC9wcm9qZWN0cy9jdnNyb290L3Bnc3FsL3NyYy9iYWNrZW5kL3BvcnQvd2lu MzIvTWFrZWZpbGUsdg0KcmV0cmlldmluZyByZXZpc2lvbiAxLjYNCmRpZmYg LWMgLXIxLjYgTWFrZWZpbGUNCioqKiBzcmMvYmFja2VuZC9wb3J0L3dpbjMy L01ha2VmaWxlCTI5IEF1ZyAyMDA0IDAwOjM4OjAzIC0wMDAwCTEuNg0KLS0t IHNyYy9iYWNrZW5kL3BvcnQvd2luMzIvTWFrZWZpbGUJMzAgSnVsIDIwMDUg MTc6NTI6NDcgLTAwMDANCioqKioqKioqKioqKioqKg0KKioqIDEyLDI5ICoq KioNCiAgdG9wX2J1aWxkZGlyID0gLi4vLi4vLi4vLi4NCiAgaW5jbHVkZSAk KHRvcF9idWlsZGRpcikvc3JjL01ha2VmaWxlLmdsb2JhbA0KICANCiEgT0JK UyA9IHNlbWEubyBzaG1lbS5vIHRpbWVyLm8gc29ja2V0Lm8gc2lnbmFsLm8g c2VjdXJpdHkubyBlcnJvci5vDQogIA0KICBhbGw6IFNVQlNZUy5vDQogIA0K ICBTVUJTWVMubzogJChPQkpTKQ0KICAJJChMRCkgJChMRFJFTCkgJChMRE9V VCkgU1VCU1lTLm8gJChPQkpTKQ0KICANCiAgZGVwZW5kIGRlcDoNCiAgCSQo Q0MpIC1NTSAkKENGTEFHUykgKi5jID5kZXBlbmQNCiAgDQogIGNsZWFuOiAN CiEgCXJtIC1mIFNVQlNZUy5vICQoT0JKUykNCiAgDQogIGlmZXEgKGRlcGVu ZCwkKHdpbGRjYXJkIGRlcGVuZCkpDQogIGluY2x1ZGUgZGVwZW5kDQotLS0g MTIsMzQgLS0tLQ0KICB0b3BfYnVpbGRkaXIgPSAuLi8uLi8uLi8uLg0KICBp bmNsdWRlICQodG9wX2J1aWxkZGlyKS9zcmMvTWFrZWZpbGUuZ2xvYmFsDQog IA0KISBPQkpTID0gc2VtYS5vIHNobWVtLm8gdGltZXIubyBzb2NrZXQubyBz aWduYWwubyBzZWN1cml0eS5vIGVycm9yLm8gbG9jYWxlbWFwLm8NCiAgDQog IGFsbDogU1VCU1lTLm8NCiAgDQogIFNVQlNZUy5vOiAkKE9CSlMpDQogIAkk KExEKSAkKExEUkVMKSAkKExET1VUKSBTVUJTWVMubyAkKE9CSlMpDQogIA0K KyBsb2NhbGVtYXAubzogbG9jYWxlbWFwLmMgbG9jYWxlbWFwLmgNCisgDQor IGxvY2FsZW1hcC5oOiBpc282MzkgaXNvMzE2NiBsb2NhbGVtYXAucGwNCisg CSQoUEVSTCkgbG9jYWxlbWFwLnBsID4gbG9jYWxlbWFwLmgNCisgDQogIGRl cGVuZCBkZXA6DQogIAkkKENDKSAtTU0gJChDRkxBR1MpICouYyA+ZGVwZW5k DQogIA0KICBjbGVhbjogDQohIAlybSAtZiBTVUJTWVMubyAkKE9CSlMpIGxv Y2FsZW1hcC5oDQogIA0KICBpZmVxIChkZXBlbmQsJCh3aWxkY2FyZCBkZXBl bmQpKQ0KICBpbmNsdWRlIGRlcGVuZA0KSW5kZXg6IHNyYy9iYWNrZW5kL3V0 aWxzL2FkdC9vcmFjbGVfY29tcGF0LmMNCj09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT0NClJDUyBmaWxlOiAvcHJvamVjdHMvY3Zzcm9vdC9wZ3NxbC9zcmMvYmFj a2VuZC91dGlscy9hZHQvb3JhY2xlX2NvbXBhdC5jLHYNCnJldHJpZXZpbmcg cmV2aXNpb24gMS42MA0KZGlmZiAtYyAtcjEuNjAgb3JhY2xlX2NvbXBhdC5j DQoqKiogc3JjL2JhY2tlbmQvdXRpbHMvYWR0L29yYWNsZV9jb21wYXQuYwk3 IE1heSAyMDA1IDE1OjE4OjE3IC0wMDAwCTEuNjANCi0tLSBzcmMvYmFja2Vu ZC91dGlscy9hZHQvb3JhY2xlX2NvbXBhdC5jCTMwIEp1bCAyMDA1IDE0OjU5 OjI5IC0wMDAwDQoqKioqKioqKioqKioqKioNCioqKiAzMiwzNyAqKioqDQot LS0gMzIsNDMgLS0tLQ0KICAjaW5jbHVkZSAidXRpbHMvcGdfbG9jYWxlLmgi DQogICNpbmNsdWRlICJtYi9wZ193Y2hhci5oIg0KICANCisgI2lmZGVmIFVT RV9JQ1UNCisgI2luY2x1ZGUgPHVuaWNvZGUvdXR5cGVzLmg+ICAgLyogQmFz aWMgSUNVIGRhdGEgdHlwZXMgKi8NCisgI2luY2x1ZGUgPHVuaWNvZGUvdWNu di5oPiAgICAgLyogQyAgIENvbnZlcnRlciBBUEkgICAgKi8NCisgI2luY2x1 ZGUgPHVuaWNvZGUvdXN0cmluZy5oPg0KKyAjZW5kaWYgLyogVVNFX0lDVSAq Lw0KKyANCiAgDQogIC8qDQogICAqIElmIHRoZSBzeXN0ZW0gcHJvdmlkZXMg dGhlIG5lZWRlZCBmdW5jdGlvbnMgZm9yIHdpZGUtY2hhcmFjdGVyIG1hbmlw dWxhdGlvbg0KKioqKioqKioqKioqKioqDQoqKiogNTMsNTggKioqKg0KLS0t IDU5LDEwNiAtLS0tDQogIAkgICBib29sIGRvbHRyaW0sIGJvb2wgZG9ydHJp bSk7DQogIA0KICANCisgI2lmZGVmIFVTRV9JQ1UNCisgc3RhdGljIFVDb252 ZXJ0ZXIgKmNvbnYgPSBOVUxMOw0KKyANCisgc3RhdGljIHRleHQgKg0KKyBV Q2hhcnRvdGV4dChjb25zdCBVQ2hhciAqc3RyLCBpbnQgbmNvZGVzKQ0KKyB7 DQorIAl0ZXh0CSAgICpyZXN1bHQ7DQorIAlzaXplX3QJCW5ieXRlcywgcmVz dWx0c2l6ZTsNCisgDQorIAlVRXJyb3JDb2RlIHN0YXR1cyA9IFVfWkVST19F UlJPUjsNCisgDQorIAkvKiBPdmVyZmxvdyBwYXJhbm9pYSAqLw0KKyAJaWYg KG5jb2RlcyA8IDAgfHwNCisgCQluY29kZXMgPiAoaW50KSAoKElOVF9NQVgg LSBWQVJIRFJTWikgLyBzaXplb2YoVUNoYXIpKSAtIDEpDQorIAkJZXJlcG9y dChFUlJPUiwNCisgCQkJCShlcnJjb2RlKEVSUkNPREVfT1VUX09GX01FTU9S WSksDQorIAkJCQkgZXJybXNnKCJvdXQgb2YgbWVtb3J5IikpKTsNCisgDQor IAkvKiBNYWtlIHdvcmtzcGFjZSBjZXJ0YWlubHkgbGFyZ2UgZW5vdWdoIGZv ciByZXN1bHQgKi8NCisgCXJlc3VsdHNpemUgPSBVQ05WX0dFVF9NQVhfQllU RVNfRk9SX1NUUklORyhuY29kZXMsIHVjbnZfZ2V0TWF4Q2hhclNpemUoY29u dikpOw0KKyAJcmVzdWx0ID0gKHRleHQgKikgcGFsbG9jKHJlc3VsdHNpemUg KyBWQVJIRFJTWik7DQorIA0KKyAJbmJ5dGVzID0gdWNudl9mcm9tVUNoYXJz KGNvbnYsIFZBUkRBVEEocmVzdWx0KSwgcmVzdWx0c2l6ZSwNCisgCQkJCQkJ CSBzdHIsIG5jb2RlcywgJnN0YXR1cyk7DQorIA0KKyAJaWYgKFVfRkFJTFVS RShzdGF0dXMpKQ0KKyAJew0KKyAJCS8qIEludmFsaWQgbXVsdGlieXRlIGNo YXJhY3RlciBlbmNvdW50ZXJlZCAuLi4gc2hvdWxkbid0IGhhcHBlbiAqLw0K KyAJCWVyZXBvcnQoRVJST1IsDQorIAkJCQkoZXJyY29kZShFUlJDT0RFX0NI QVJBQ1RFUl9OT1RfSU5fUkVQRVJUT0lSRSksDQorIAkJCQkgZXJybXNnKCJp bnZhbGlkIG11bHRpYnl0ZSBjaGFyYWN0ZXIgZm9yIGxvY2FsZSIpKSk7DQor IAl9DQorIA0KKyAJQXNzZXJ0KG5ieXRlcyA8PSAoc2l6ZV90KSAobmNvZGVz ICogc2l6ZW9mKFVDaGFyKSkpOw0KKyANCisgCVZBUkFUVF9TSVpFUChyZXN1 bHQpID0gbmJ5dGVzICsgVkFSSERSU1o7DQorIA0KKyAJcmV0dXJuIHJlc3Vs dDsNCisgfQ0KKyANCisgDQorICNlbHNlDQogICNpZmRlZiBVU0VfV0lERV9V UFBFUl9MT1dFUg0KICANCiAgLyoNCioqKioqKioqKioqKioqKg0KKioqIDE0 NywxNTIgKioqKg0KLS0tIDE5NSwyMDEgLS0tLQ0KICAJcmV0dXJuIHJlc3Vs dDsNCiAgfQ0KICAjZW5kaWYgICAvKiBVU0VfV0lERV9VUFBFUl9MT1dFUiAq Lw0KKyAjZW5kaWYgICAvKiBVU0VfSUNVICovDQogIA0KICANCiAgLyoqKioq KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq KioqKioqKioqKioqKioqKioqDQoqKioqKioqKioqKioqKioNCioqKiAxNjYs MTcxICoqKioNCi0tLSAyMTUsMjg2IC0tLS0NCiAgRGF0dW0NCiAgbG93ZXIo UEdfRlVOQ1RJT05fQVJHUykNCiAgew0KKyAjaWZkZWYgVVNFX0lDVQ0KKyAj ZGVmaW5lIFNUQUNLQlVGTEVOCQkxMDI0IC8gc2l6ZW9mKFVDaGFyKQ0KKyAJ LyogdXNlIElDVSBvbmx5IHdoZW4gbWF4IGVuY29kaW5nIGxlbmd0aCA+IG9u ZSAqLw0KKyAJaWYgKHBnX2RhdGFiYXNlX2VuY29kaW5nX21heF9sZW5ndGgo KSA+IDEpDQorIAl7DQorIAkJdGV4dAkgICAqc3RyaW5nID0gUEdfR0VUQVJH X1RFWFRfUCgwKTsNCisgCQl0ZXh0CSAgICpyZXN1bHQ7DQorIAkJVUNoYXIg ICAgICAgc291cmNlYnVmW1NUQUNLQlVGTEVOXSwgZGVzdGJ1ZltTVEFDS0JV RkxFTl07DQorIAkJVUNoYXIgICAgICAqc291cmNlLCAqZGVzdDsNCisgCQlp bnQJCQlidWZsZW4sDQorIAkJCQkJYXJnbGVuID0gVkFSU0laRShzdHJpbmcp IC0gVkFSSERSU1o7DQorIAkJVUVycm9yQ29kZSAgc3RhdHVzID0gVV9aRVJP X0VSUk9SOw0KKyANCisgCQlpZiAoY29udiA9PSBOVUxMKQ0KKyAJCXsNCisg CQkJY29udiA9IHVjbnZfb3BlbihOVUxMLCAmc3RhdHVzKTsNCisgCQkJaWYg KFVfRkFJTFVSRShzdGF0dXMpKQ0KKyAJCQl7DQorIAkJCQllcmVwb3J0KEVS Uk9SLA0KKyAJCQkJCQkoZXJyY29kZShzdGF0dXMpLA0KKyAJCQkJCQkgZXJy bXNnKCJJQ1UgZXJyb3I6IG9yYWNsZV9jb21wYXQuYywgY291bGQgbm90IGdl dCBjb252ZXJ0ZXIgZm9yIFwiJXNcIiIsIHVjbnZfZ2V0RGVmYXVsdE5hbWUo KSkpKTsNCisgCQkJfQ0KKyAJCX0NCisgDQorIAkJaWYgKGFyZ2xlbiA+PSBT VEFDS0JVRkxFTiAvIHNpemVvZihVQ2hhcikpDQorIAkJew0KKyAJCQlidWZs ZW4gPSBhcmdsZW4gKiBzaXplb2YoVUNoYXIpICsgMTsNCisgIAkJCXNvdXJj ZSA9IHBhbGxvYyhidWZsZW4pOw0KKyAJCQlkZXN0ID0gcGFsbG9jKGJ1Zmxl bik7DQorIAkJfQ0KKyAJCWVsc2UNCisgCQl7DQorIAkJCWJ1ZmxlbiA9IFNU QUNLQlVGTEVOOw0KKyAJCQlzb3VyY2UgPSBzb3VyY2VidWY7DQorIAkJCWRl c3QgPSBkZXN0YnVmOw0KKyAJCX0NCisgCQkvLyBjb252ZXJ0IHRvIFVURi0x Ng0KKyAJCXVjbnZfdG9VQ2hhcnMoY29udiwgc291cmNlLCBidWZsZW4sIFZB UkRBVEEoc3RyaW5nKSwgYXJnbGVuLCAmc3RhdHVzKTsNCisgCQlpZiAoVV9G QUlMVVJFKHN0YXR1cykpDQorIAkJew0KKyAJCQllcmVwb3J0KEVSUk9SLA0K KyAJCQkJCQkoZXJyY29kZShzdGF0dXMpLA0KKyAJCQkJCQkgZXJybXNnKCJJ Q1UgZXJyb3I6IENvdWxkIG5vdCBjb252ZXJ0IHN0cmluZyIpKSk7DQorIAkJ fQ0KKyAJCQ0KKyAJCS8vIHJ1biBkZXNpcmVkIGZ1bmN0aW9uDQorIAkJYnVm bGVuID0gdV9zdHJUb0xvd2VyKGRlc3QsIGJ1Zmxlbiwgc291cmNlLCAtMSwg TlVMTCwgJnN0YXR1cyk7DQorIAkJaWYgKFVfRkFJTFVSRShzdGF0dXMpKQ0K KyAJCXsNCisgCQkJZXJlcG9ydChFUlJPUiwNCisgCQkJCQkJKGVycmNvZGUo c3RhdHVzKSwNCisgCQkJCQkJIGVycm1zZygiSUNVIGVycm9yOiBDb3VsZCBu b3QgbW9kaWZ5IGNhc2UiKSkpOw0KKyAJCX0NCisgDQorIAkJLy8gYW5kIGNv bnZlcnQgbW9kaWZpZWQgdXRmLTE2IHN0cmluZyBiYWNrIHRvIHRleHQNCisg CQlyZXN1bHQgPSBVQ2hhcnRvdGV4dChkZXN0LCBidWZsZW4pOw0KKyANCisg CQlpZiAoYXJnbGVuID49IFNUQUNLQlVGTEVOIC8gc2l6ZW9mKFVDaGFyKSkN CisgCQl7DQorIAkJCXBmcmVlKHNvdXJjZSk7DQorIAkJCXBmcmVlKGRlc3Qp Ow0KKyAJCX0NCisgCQlQR19SRVRVUk5fVEVYVF9QKHJlc3VsdCk7DQorIAl9 DQorIAllbHNlDQorICNlbHNlDQogICNpZmRlZiBVU0VfV0lERV9VUFBFUl9M T1dFUg0KICAJLyoNCiAgCSAqCVVzZSB3aWRlIGNoYXIgY29kZSBvbmx5IHdo ZW4gbWF4IGVuY29kaW5nIGxlbmd0aCA+IDEgYW5kIGN0eXBlICE9IEMuDQoq KioqKioqKioqKioqKioNCioqKiAxOTIsMTk3ICoqKioNCi0tLSAzMDcsMzEz IC0tLS0NCiAgCX0NCiAgCWVsc2UNCiAgI2VuZGlmICAgLyogVVNFX1dJREVf VVBQRVJfTE9XRVIgKi8NCisgI2VuZGlmICAgLyogVVNFX0lDVSAqLw0KICAJ ew0KICAJCXRleHQJICAgKnN0cmluZyA9IFBHX0dFVEFSR19URVhUX1BfQ09Q WSgwKTsNCiAgCQljaGFyCSAgICpwdHI7DQoqKioqKioqKioqKioqKioNCioq KiAyMzIsMjM3ICoqKioNCi0tLSAzNDgsNDE4IC0tLS0NCiAgRGF0dW0NCiAg dXBwZXIoUEdfRlVOQ1RJT05fQVJHUykNCiAgew0KKyAjaWZkZWYgVVNFX0lD VQ0KKyAJLyogdXNlIElDVSBvbmx5IHdoZW4gbWF4IGVuY29kaW5nIGxlbmd0 aCA+IG9uZSAqLw0KKyAJaWYgKHBnX2RhdGFiYXNlX2VuY29kaW5nX21heF9s ZW5ndGgoKSA+IDEpDQorIAl7DQorIAkJdGV4dAkgICAqc3RyaW5nID0gUEdf R0VUQVJHX1RFWFRfUCgwKTsNCisgCQl0ZXh0CSAgICpyZXN1bHQ7DQorIAkJ VUNoYXIgICAgICAgc291cmNlYnVmW1NUQUNLQlVGTEVOXSwgZGVzdGJ1ZltT VEFDS0JVRkxFTl07DQorIAkJVUNoYXIgICAgICAqc291cmNlLCAqZGVzdDsN CisgCQlpbnQzMl90ICAgICBidWZsZW4sIGFyZ2xlbjsNCisgDQorIAkJVUVy cm9yQ29kZSAgc3RhdHVzID0gVV9aRVJPX0VSUk9SOw0KKyANCisgCQlpZiAo Y29udiA9PSBOVUxMKQ0KKyAJCXsNCisgCQkJY29udiA9IHVjbnZfb3BlbihO VUxMLCAmc3RhdHVzKTsNCisgCQkJaWYgKFVfRkFJTFVSRShzdGF0dXMpKQ0K KyAJCQl7DQorIAkJCQllcmVwb3J0KEVSUk9SLA0KKyAJCQkJCQkoZXJyY29k ZShzdGF0dXMpLA0KKyAJCQkJCQkgZXJybXNnKCJJQ1UgZXJyb3I6IENvdWxk IG5vdCBnZXQgY29udmVydGVyIGZvciBcIiVzXCIiLCB1Y252X2dldERlZmF1 bHROYW1lKCkpKSk7DQorIAkJCX0NCisgCQl9DQorIAkJYXJnbGVuID0gVkFS U0laRShzdHJpbmcpIC0gVkFSSERSU1o7DQorIAkJaWYgKGFyZ2xlbiAqIHNp emVvZihVQ2hhcikgPj0gU1RBQ0tCVUZMRU4pDQorIAkJew0KKyAJCQlidWZs ZW4gPSBhcmdsZW4gKiBzaXplb2YoVUNoYXIpICsgMTsNCisgIAkJCXNvdXJj ZSA9IHBhbGxvYyhidWZsZW4pOw0KKyAJCQlkZXN0ID0gcGFsbG9jKGJ1Zmxl bik7DQorIAkJfQ0KKyAJCWVsc2UNCisgCQl7DQorIAkJCWJ1ZmxlbiA9IFNU QUNLQlVGTEVOOw0KKyAJCQlzb3VyY2UgPSBzb3VyY2VidWY7DQorIAkJCWRl c3QgPSBkZXN0YnVmOw0KKyAJCX0NCisgCQkvLyBjb252ZXJ0IHRvIFVURi0x Ng0KKyAJCXVjbnZfdG9VQ2hhcnMoY29udiwgc291cmNlLCBidWZsZW4sIFZB UkRBVEEoc3RyaW5nKSwgYXJnbGVuLCAmc3RhdHVzKTsNCisgCQlpZiAoVV9G QUlMVVJFKHN0YXR1cykpDQorIAkJew0KKyAJCQllcmVwb3J0KEVSUk9SLA0K KyAJCQkJCQkoZXJyY29kZShzdGF0dXMpLA0KKyAJCQkJCQkgZXJybXNnKCJJ Q1UgZXJyb3I6IENvdWxkIG5vdCBjb252ZXJ0IHN0cmluZyIpKSk7DQorIAkJ fQ0KKyANCisgCQkvLyBydW4gZGVzaXJlZCBmdW5jdGlvbg0KKyAJCWJ1Zmxl biA9IHVfc3RyVG9VcHBlcihkZXN0LCBidWZsZW4sIHNvdXJjZSwgLTEsIE5V TEwsICZzdGF0dXMpOw0KKyAJCWlmIChVX0ZBSUxVUkUoc3RhdHVzKSkNCisg CQl7DQorIAkJCWVyZXBvcnQoRVJST1IsDQorIAkJCQkJCShlcnJjb2RlKHN0 YXR1cyksDQorIAkJCQkJCSBlcnJtc2coIklDVSBlcnJvcjogQ291bGQgbm90 IG1vZGlmeSBjYXNlIikpKTsNCisgCQl9DQorIA0KKyAJCS8vIGFuZCBjb252 ZXJ0IG1vZGlmaWVkIHV0Zi0xNiBzdHJpbmcgYmFjayB0byB0ZXh0DQorIAkJ cmVzdWx0ID0gVUNoYXJ0b3RleHQoZGVzdCwgYnVmbGVuKTsNCisgDQorIAkJ aWYgKGFyZ2xlbiAqIHNpemVvZihVQ2hhcikgPj0gU1RBQ0tCVUZMRU4pDQor IAkJew0KKyAJCQlwZnJlZShzb3VyY2UpOw0KKyAJCQlwZnJlZShkZXN0KTsN CisgCQl9DQorIAkJUEdfUkVUVVJOX1RFWFRfUChyZXN1bHQpOw0KKyAJfQ0K KyAJZWxzZQ0KKyAjZWxzZQ0KICAjaWZkZWYgVVNFX1dJREVfVVBQRVJfTE9X RVINCiAgCS8qDQogIAkgKglVc2Ugd2lkZSBjaGFyIGNvZGUgb25seSB3aGVu IG1heCBlbmNvZGluZyBsZW5ndGggPiAxIGFuZCBjdHlwZSAhPSBDLg0KKioq KioqKioqKioqKioqDQoqKiogMjU4LDI2MyAqKioqDQotLS0gNDM5LDQ0NSAt LS0tDQogIAl9DQogIAllbHNlDQogICNlbmRpZiAgIC8qIFVTRV9XSURFX1VQ UEVSX0xPV0VSICovDQorICNlbmRpZiAgIC8qIFVTRV9JQ1UgKi8NCiAgCXsN CiAgCQl0ZXh0CSAgICpzdHJpbmcgPSBQR19HRVRBUkdfVEVYVF9QX0NPUFko MCk7DQogIAkJY2hhcgkgICAqcHRyOw0KKioqKioqKioqKioqKioqDQoqKiog MzAxLDMwNiAqKioqDQotLS0gNDgzLDU1MyAtLS0tDQogIERhdHVtDQogIGlu aXRjYXAoUEdfRlVOQ1RJT05fQVJHUykNCiAgew0KKyAjaWZkZWYgVVNFX0lD VQ0KKyAJLyogdXNlIElDVSBvbmx5IHdoZW4gbWF4IGVuY29kaW5nIGxlbmd0 aCA+IG9uZSAqLw0KKyAJaWYgKHBnX2RhdGFiYXNlX2VuY29kaW5nX21heF9s ZW5ndGgoKSA+IDEpDQorIAl7DQorIAkJdGV4dAkgICAqc3RyaW5nID0gUEdf R0VUQVJHX1RFWFRfUCgwKTsNCisgCQl0ZXh0CSAgICpyZXN1bHQ7DQorIAkJ VUNoYXIgICAgICAgc291cmNlYnVmW1NUQUNLQlVGTEVOXSwgZGVzdGJ1ZltT VEFDS0JVRkxFTl07DQorIAkJVUNoYXIgICAgICAqc291cmNlLCAqZGVzdDsN CisgCQlpbnQzMl90ICAgICBidWZsZW4sIGFyZ2xlbjsNCisgDQorIAkJVUVy cm9yQ29kZSAgc3RhdHVzID0gVV9aRVJPX0VSUk9SOw0KKyANCisgCQlpZiAo Y29udiA9PSBOVUxMKQ0KKyAJCXsNCisgCQkJY29udiA9IHVjbnZfb3BlbihO VUxMLCAmc3RhdHVzKTsNCisgCQkJaWYgKFVfRkFJTFVSRShzdGF0dXMpKQ0K KyAJCQl7DQorIAkJCQllcmVwb3J0KEVSUk9SLA0KKyAJCQkJCQkoZXJyY29k ZShzdGF0dXMpLA0KKyAJCQkJCQkgZXJybXNnKCJJQ1UgZXJyb3I6IENvdWxk IG5vdCBnZXQgY29udmVydGVyIGZvciBcIiVzXCIiLCB1Y252X2dldERlZmF1 bHROYW1lKCkpKSk7DQorIAkJCX0NCisgCQl9DQorIAkJYXJnbGVuID0gVkFS U0laRShzdHJpbmcpIC0gVkFSSERSU1o7DQorIAkJaWYgKGFyZ2xlbiAqIHNp emVvZihVQ2hhcikgPj0gU1RBQ0tCVUZMRU4pDQorIAkJew0KKyAJCQlidWZs ZW4gPSBhcmdsZW4gKiBzaXplb2YoVUNoYXIpICsgMTsNCisgIAkJCXNvdXJj ZSA9IHBhbGxvYyhidWZsZW4pOw0KKyAJCQlkZXN0ID0gcGFsbG9jKGJ1Zmxl bik7DQorIAkJfQ0KKyAJCWVsc2UNCisgCQl7DQorIAkJCWJ1ZmxlbiA9IFNU QUNLQlVGTEVOOw0KKyAJCQlzb3VyY2UgPSBzb3VyY2VidWY7DQorIAkJCWRl c3QgPSBkZXN0YnVmOw0KKyAJCX0NCisgCQkvLyBjb252ZXJ0IHRvIFVURi0x Ng0KKyAJCXVjbnZfdG9VQ2hhcnMoY29udiwgc291cmNlLCBidWZsZW4sIFZB UkRBVEEoc3RyaW5nKSwgYXJnbGVuLCAmc3RhdHVzKTsNCisgCQlpZiAoVV9G QUlMVVJFKHN0YXR1cykpDQorIAkJew0KKyAJCQllcmVwb3J0KEVSUk9SLA0K KyAJCQkJCQkoZXJyY29kZShzdGF0dXMpLA0KKyAJCQkJCQkgZXJybXNnKCJJ Q1UgZXJyb3I6IENvdWxkIG5vdCBjb252ZXJ0IHN0cmluZyIpKSk7DQorIAkJ fQ0KKyANCisgCQkvLyBydW4gZGVzaXJlZCBmdW5jdGlvbg0KKyAJCWJ1Zmxl biA9IHVfc3RyVG9UaXRsZShkZXN0LCBidWZsZW4sIHNvdXJjZSwgLTEsIE5V TEwsIE5VTEwsICZzdGF0dXMpOw0KKyAJCWlmIChVX0ZBSUxVUkUoc3RhdHVz KSkNCisgCQl7DQorIAkJCWVyZXBvcnQoRVJST1IsDQorIAkJCQkJCShlcnJj b2RlKHN0YXR1cyksDQorIAkJCQkJCSBlcnJtc2coIklDVSBlcnJvcjogQ291 bGQgbm90IG1vZGlmeSBjYXNlIikpKTsNCisgCQl9DQorIA0KKyAJCS8vIGFu ZCBjb252ZXJ0IG1vZGlmaWVkIHV0Zi0xNiBzdHJpbmcgYmFjayB0byB0ZXh0 DQorIAkJcmVzdWx0ID0gVUNoYXJ0b3RleHQoZGVzdCwgYnVmbGVuKTsNCisg DQorIAkJaWYgKGFyZ2xlbiAqIHNpemVvZihVQ2hhcikgPj0gU1RBQ0tCVUZM RU4pDQorIAkJew0KKyAJCQlwZnJlZShzb3VyY2UpOw0KKyAJCQlwZnJlZShk ZXN0KTsNCisgCQl9DQorIAkJUEdfUkVUVVJOX1RFWFRfUChyZXN1bHQpOw0K KyAJfQ0KKyAJZWxzZQ0KKyAjZWxzZQ0KICAjaWZkZWYgVVNFX1dJREVfVVBQ RVJfTE9XRVINCiAgCS8qDQogIAkgKglVc2Ugd2lkZSBjaGFyIGNvZGUgb25s eSB3aGVuIG1heCBlbmNvZGluZyBsZW5ndGggPiAxIGFuZCBjdHlwZSAhPSBD Lg0KKioqKioqKioqKioqKioqDQoqKiogMzM0LDMzOSAqKioqDQotLS0gNTgx LDU4NyAtLS0tDQogIAl9DQogIAllbHNlDQogICNlbmRpZiAgIC8qIFVTRV9X SURFX1VQUEVSX0xPV0VSICovDQorICNlbmRpZiAgIC8qIFVTRV9JQ1UgKi8N CiAgCXsNCiAgCQl0ZXh0CSAgICpzdHJpbmcgPSBQR19HRVRBUkdfVEVYVF9Q X0NPUFkoMCk7DQogIAkJaW50CQkJd2FzYWxudW0gPSAwOw0KSW5kZXg6IHNy Yy9iYWNrZW5kL3V0aWxzL2FkdC92YXJsZW5hLmMNCj09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT0NClJDUyBmaWxlOiAvcHJvamVjdHMvY3Zzcm9vdC9wZ3NxbC9z cmMvYmFja2VuZC91dGlscy9hZHQvdmFybGVuYS5jLHYNCnJldHJpZXZpbmcg cmV2aXNpb24gMS4xMzANCmRpZmYgLWMgLXIxLjEzMCB2YXJsZW5hLmMNCioq KiBzcmMvYmFja2VuZC91dGlscy9hZHQvdmFybGVuYS5jCTI5IEp1bCAyMDA1 IDAzOjE3OjU1IC0wMDAwCTEuMTMwDQotLS0gc3JjL2JhY2tlbmQvdXRpbHMv YWR0L3ZhcmxlbmEuYwkzMCBKdWwgMjAwNSAxNzo0NjozNCAtMDAwMA0KKioq KioqKioqKioqKioqDQoqKiogMzAsMzUgKioqKg0KLS0tIDMwLDQxIC0tLS0N CiAgI2luY2x1ZGUgInV0aWxzL3BnX2xvY2FsZS5oIg0KICAjaW5jbHVkZSAi cmVnZXgvcmVnZXguaCINCiAgDQorICNpZmRlZiBVU0VfSUNVDQorICNpbmNs dWRlIDx1bmljb2RlL3V0eXBlcy5oPiAgIC8qIEJhc2ljIElDVSBkYXRhIHR5 cGVzICovDQorICNpbmNsdWRlIDx1bmljb2RlL3VjbnYuaD4gICAgIC8qIEMg ICBDb252ZXJ0ZXIgQVBJICAgICovDQorICNpbmNsdWRlIDx1bmljb2RlL3Vj b2wuaD4NCisgI2luY2x1ZGUgPHVuaWNvZGUvdWxvYy5oPg0KKyAjZW5kaWYg LyogVVNFX0lDVSAqLw0KICANCiAgdHlwZWRlZiBzdHJ1Y3QgdmFybGVuYSB1 bmtub3duOw0KICANCioqKioqKioqKioqKioqKg0KKioqIDg0NCw4NzQgKioq Kg0KICANCiAgCWlmICghbGNfY29sbGF0ZV9pc19jKCkpDQogIAl7DQohIAkJ Y2hhcgkJYTFidWZbU1RBQ0tCVUZMRU5dOw0KISAJCWNoYXIJCWEyYnVmW1NU QUNLQlVGTEVOXTsNCiEgCQljaGFyCSAgICphMXAsDQohIAkJCQkgICAqYTJw Ow0KICANCiEgCQlpZiAobGVuMSA+PSBTVEFDS0JVRkxFTikNCiEgCQkJYTFw ID0gKGNoYXIgKikgcGFsbG9jKGxlbjEgKyAxKTsNCiEgCQllbHNlDQohIAkJ CWExcCA9IGExYnVmOw0KISAJCWlmIChsZW4yID49IFNUQUNLQlVGTEVOKQ0K ISAJCQlhMnAgPSAoY2hhciAqKSBwYWxsb2MobGVuMiArIDEpOw0KICAJCWVs c2UNCiEgCQkJYTJwID0gYTJidWY7DQogIA0KISAJCW1lbWNweShhMXAsIGFy ZzEsIGxlbjEpOw0KISAJCWExcFtsZW4xXSA9ICdcMCc7DQohIAkJbWVtY3B5 KGEycCwgYXJnMiwgbGVuMik7DQohIAkJYTJwW2xlbjJdID0gJ1wwJzsNCiEg DQohIAkJcmVzdWx0ID0gc3RyY29sbChhMXAsIGEycCk7DQohIA0KISAJCWlm IChsZW4xID49IFNUQUNLQlVGTEVOKQ0KISAJCQlwZnJlZShhMXApOw0KISAJ CWlmIChsZW4yID49IFNUQUNLQlVGTEVOKQ0KISAJCQlwZnJlZShhMnApOw0K ICAJfQ0KICAJZWxzZQ0KICAJew0KLS0tIDg1MCw5ODYgLS0tLQ0KICANCiAg CWlmICghbGNfY29sbGF0ZV9pc19jKCkpDQogIAl7DQohICNpZmRlZiBVU0Vf SUNVDQohICNkZWZpbmUgVVNUQUNLQlVGTEVOCQlTVEFDS0JVRkxFTiAvIHNp emVvZihVQ2hhcikNCiEgCQlpZiAocGdfZGF0YWJhc2VfZW5jb2RpbmdfbWF4 X2xlbmd0aCgpID4gMSkNCiEgCQl7DQohIAkJCVVDaGFyCQlhMWJ1ZltVU1RB Q0tCVUZMRU5dLA0KISAJCSAgICAgICAgIAkJYTJidWZbVVNUQUNLQlVGTEVO XTsNCiEgCQkJaW50CQkJYTFsZW4gPSBVU1RBQ0tCVUZMRU4sDQohIAkJCSAg CQkJYTJsZW4gPSBVU1RBQ0tCVUZMRU47CQ0KISAJCQlVQ2hhcgkgICAqYTFw LA0KISAJCQkgICAgICAgICAgICphMnA7DQohIA0KISAJCQlzdGF0aWMgVUNv bGxhdG9yICogY29sbGF0b3IgPSBOVUxMOw0KISAJCQlzdGF0aWMgVUNvbnZl cnRlciAqIGNvbnYgPSBOVUxMOw0KISAJCQlVRXJyb3JDb2RlICBzdGF0dXMg PSBVX1pFUk9fRVJST1I7DQogIA0KISAJCQlpZiAoY29udiA9PSBOVUxMKQ0K ISAJCQl7DQohIAkJCQljb252ID0gdWNudl9vcGVuKE5VTEwsICZzdGF0dXMp Ow0KISAJCQkJaWYgKFVfRkFJTFVSRShzdGF0dXMpIHx8IGNvbnYgPT0gTlVM TCkNCiEgCQkJCXsNCiEgCQkJCQllcmVwb3J0KEVSUk9SLA0KISAJCQkJCQkJ KGVycmNvZGUoc3RhdHVzKSwNCiEgCQkJCQkJCSBlcnJtc2coIklDVSBlcnJv cjogdmFybGVuYS5jLCBjb3VsZCBub3QgZ2V0IGNvbnZlcnRlciBmb3IgXCIl c1wiIiwgdWNudl9nZXREZWZhdWx0TmFtZSgpKSkpOw0KISAJCQkJfQ0KISAJ CQl9DQohIA0KISAJCQkvKiBXZSBrZWVwIGEgc3RhdGljIGNvbGxhdG9yICJm b3JldmVyIiwgc2luY2UgaXQgaXMgaGFyZA0KISAJCQkgKiBjb2RlZCBpbnRv IHRoZSBkYXRhYmFzZSBjbHVzdGVyIGF0IGluaXRkYiB0aW1lDQohIAkJCSAq IGFueXdheS4gQ3JlYXRlIGl0IGZpcnN0IHRpbWUgd2UgZ2V0IGhlcmUuICov DQohIAkJCWlmIChjb2xsYXRvciA9PSBOVUxMKQ0KISAJCQl7DQohIAkJCQkv KiBFeHBlY3QgTENfQ09MTEFURSB0byBiZSBzZXQgdG8gc29tZXRoaW5nIHRo YXQgSUNVDQohIAkJCQkgKiB3aWxsIHVuZGVyc3RhbmQuIFRoaXMgaXMgcXVp dGUgcHJvYmFibGUsIHNpbmNlIElDVQ0KISAJCQkJICogZG9lcyBhIGxvdCBv ZiBoZXVyaXN0aWNzIHdpdGggdGhpcyBhcmd1bWVudC4gSSdkDQohIAkJCQkg KiByYXRoZXIgc2V0IHRoaXMgaW4geGxvZy5jLCBidXQgaXQgc2VlbXMgSUNV IGZvcmdldHMNCiEgCQkJCSAqIGl0Pz8/ICovDQohICNpZm5kZWYgV0lOMzIN CiEgCQkJCXVsb2Nfc2V0RGVmYXVsdChzZXRsb2NhbGUoTENfQ09MTEFURSwg TlVMTCksICZzdGF0dXMpOw0KISAjZWxzZQ0KISAJCQkJLyogV2luMzIgbG9j YWxlIG5hbWVzIGFyZSBjb21wbGV0ZWx5IGRpZmZlcmVudCBmcm9tIHdoYXQg SUNVIGV4cGVjdHMsIHNvIA0KISAJCQkJICAgd2UgbmVlZCB0byBkbyBzb21l IGNvbnZlcnNpb24gKi8NCiEgCQkJCXVsb2Nfc2V0RGVmYXVsdChwZ3dpbjMy X2xvY2FsZW1hcChzZXRsb2NhbGUoTENfQ09MTEFURSwgTlVMTCkpLCAmc3Rh dHVzKTsNCiEgI2VuZGlmDQohIAkJCQlpZihVX0ZBSUxVUkUoc3RhdHVzKSkN CiEgCQkJCXsNCiEgCQkJCQllcmVwb3J0KFdBUk5JTkcsDQohIAkJCQkJCQko ZXJyY29kZShzdGF0dXMpLA0KISAJCQkJCQkJIGVycm1zZygiSUNVIEVycm9y OiB2YXJsZW5hLmMsIGNvdWxkIG5vdCBzZXQgZGVmYXVsdCBsY19jb2xsYXRl IikpKTsNCiEgCQkJCX0NCiEgCQkJCWNvbGxhdG9yID0gdWNvbF9vcGVuKE5V TEwsICZzdGF0dXMpOw0KISAJCQkJaWYgKFVfRkFJTFVSRShzdGF0dXMpKQ0K ISAJCQkJew0KISAJCQkJCWVyZXBvcnQoV0FSTklORywNCiEgCQkJCQkJCShl cnJjb2RlKHN0YXR1cyksDQohIAkJCQkJCQkgZXJybXNnKCJJQ1UgRXJyb3I6 IHZhcmxlbmEuYywgY291bGQgbm90IG9wZW4gY29sbGF0b3IiKSkpOw0KISAJ CQkJfQ0KISAJCQl9DQohIAkJCQ0KISAJCQlpZiAobGVuMSA+PSBVU1RBQ0tC VUZMRU4gLyBzaXplb2YoVUNoYXIpKQ0KISAJCQl7DQohIAkJCQlhMWxlbiA9 IGxlbjEgKiBzaXplb2YoVUNoYXIpICsgMjsNCiEgCQkJCWExcCA9IChVQ2hh ciAqKSBwYWxsb2MoYTFsZW4pOwkNCiEgCQkJfQ0KISAJCQllbHNlDQohIAkJ CQlhMXAgPSBhMWJ1ZjsNCiEgDQohIAkJCWlmIChsZW4yID49IFVTVEFDS0JV RkxFTiAvIHNpemVvZihVQ2hhcikpDQohIAkJCXsNCiEgCQkJCWEybGVuID0g bGVuMiAqIHNpemVvZihVQ2hhcikgKyAyOw0KISAJCQkJYTJwID0gKFVDaGFy ICopIHBhbGxvYyhhMmxlbik7DQohIAkJCX0NCiEgCQkJZWxzZQ0KISAJCQkJ YTJwID0gYTJidWY7DQohIA0KISAJCQl1Y252X3RvVUNoYXJzKGNvbnYsIGEx cCwgYTFsZW4sIGFyZzEsIGxlbjEsICZzdGF0dXMpOw0KISAJCQlpZihVX0ZB SUxVUkUoc3RhdHVzKSkNCiEgCQkJew0KISAJCQkJZXJlcG9ydChXQVJOSU5H LA0KISAJCQkJCQkoZXJyY29kZShzdGF0dXMpLA0KISAJCQkJCQkgZXJybXNn KCJJQ1UgRXJyb3I6IHZhcmxlbmEuYywgY291bGQgbm90IGNvbnZlcnQgdG8g VUNoYXJzIikpKTsNCiEgCQkJfQ0KISAJCQl1Y252X3RvVUNoYXJzKGNvbnYs IGEycCwgYTJsZW4sIGFyZzIsIGxlbjIsICZzdGF0dXMpOw0KISAJCQlpZihV X0ZBSUxVUkUoc3RhdHVzKSkNCiEgCQkJew0KISAJCQkJZXJlcG9ydChXQVJO SU5HLA0KISAJCQkJCQkoZXJyY29kZShzdGF0dXMpLA0KISAJCQkJCQkgZXJy bXNnKCJJQ1UgRXJyb3I6IHZhcmxlbmEuYywgY291bGQgbm90IGNvbnZlcnQg dG8gVUNoYXJzIikpKTsNCiEgCQkJfQ0KISANCiEgCQkJcmVzdWx0ID0gdWNv bF9zdHJjb2xsKGNvbGxhdG9yLCBhMXAsIC0xLCBhMnAsIC0xKTsNCiEgCQkJ aWYoVV9GQUlMVVJFKHN0YXR1cykpDQohIAkJCXsNCiEgCQkJCWVyZXBvcnQo V0FSTklORywNCiEgCQkJCQkJKGVycmNvZGUoc3RhdHVzKSwNCiEgCQkJCQkJ IGVycm1zZygiSUNVIEVycm9yOiB2YXJsZW5hLmMsIGNvdWxkIG5vdCBjb2xs YXRlIikpKTsNCiEgCQkJfQ0KISANCiEgCQkJaWYgKGxlbjEgKiBzaXplb2Yo VUNoYXIpID49IFVTVEFDS0JVRkxFTikNCiEgCQkJCXBmcmVlKGExcCk7DQoh IAkJCWlmIChsZW4yICogc2l6ZW9mKFVDaGFyKSA+PSBVU1RBQ0tCVUZMRU4p DQohIAkJCQlwZnJlZShhMnApOw0KISAJCX0NCiAgCQllbHNlDQohICNlbmRp ZiAvKiBVU0VfSUNVICovDQohIAkJew0KISAJCQljaGFyCQlhMWJ1ZltTVEFD S0JVRkxFTl07DQohIAkJCWNoYXIJCWEyYnVmW1NUQUNLQlVGTEVOXTsNCiEg CQkJY2hhcgkgICAqYTFwLA0KISAJCQkJCSAgICphMnA7DQogIA0KISAJCQlp ZiAobGVuMSA+PSBTVEFDS0JVRkxFTikNCiEgCQkJCWExcCA9IChjaGFyICop IHBhbGxvYyhsZW4xICsgMSk7DQohIAkJCWVsc2UNCiEgCQkJCWExcCA9IGEx YnVmOw0KISAJCQlpZiAobGVuMiA+PSBTVEFDS0JVRkxFTikNCiEgCQkJCWEy cCA9IChjaGFyICopIHBhbGxvYyhsZW4yICsgMSk7DQohIAkJCWVsc2UNCiEg CQkJCWEycCA9IGEyYnVmOw0KISAJCQkNCiEgCQkJbWVtY3B5KGExcCwgYXJn MSwgbGVuMSk7DQohIAkJCWExcFtsZW4xXSA9ICdcMCc7DQohIAkJCW1lbWNw eShhMnAsIGFyZzIsIGxlbjIpOw0KISAJCQlhMnBbbGVuMl0gPSAnXDAnOw0K ISAJCQkNCiEgCQkJcmVzdWx0ID0gc3RyY29sbChhMXAsIGEycCk7DQohIAkJ CQ0KISAJCQlpZiAobGVuMSA+PSBTVEFDS0JVRkxFTikNCiEgCQkJCXBmcmVl KGExcCk7DQohIAkJCWlmIChsZW4yID49IFNUQUNLQlVGTEVOKQ0KISAJCQkJ cGZyZWUoYTJwKTsNCiEgCQl9DQogIAl9DQogIAllbHNlDQogIAl7DQpJbmRl eDogc3JjL2JhY2tlbmQvdXRpbHMvbWIvZW5jbmFtZXMuYw0KPT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PQ0KUkNTIGZpbGU6IC9wcm9qZWN0cy9jdnNyb290L3Bn c3FsL3NyYy9iYWNrZW5kL3V0aWxzL21iL2VuY25hbWVzLmMsdg0KcmV0cmll dmluZyByZXZpc2lvbiAxLjI1DQpkaWZmIC1jIC1yMS4yNSBlbmNuYW1lcy5j DQoqKiogc3JjL2JhY2tlbmQvdXRpbHMvbWIvZW5jbmFtZXMuYwkxNCBNYXIg MjAwNSAxODozMToyMCAtMDAwMAkxLjI1DQotLS0gc3JjL2JhY2tlbmQvdXRp bHMvbWIvZW5jbmFtZXMuYwkyOCBKdWwgMjAwNSAxNzoxNjo1MiAtMDAwMA0K KioqKioqKioqKioqKioqDQoqKiogMzc1LDM4MCAqKioqDQotLS0gMzc1LDQ5 NiAtLS0tDQogIAl9DQogIH07DQogIA0KKyAjaWZkZWYgVVNFX0lDVQ0KKyAv Kg0KKyAgKiBUcnkgdG8gbWFwIG1vc3QgaW50ZXJuYWwgY2hhcmFjdGVyIGVu Y29kaW5ncyB0byB0aGUgcHJvcGVyIGFuZA0KKyAgKiBwcmVmZXJyZWQgSUFO QSBzdHJpbmcuIFVzZSB0aGlzIGluIG1idXRpbHMuYyB0byBmZWVkIElDVSBp bmZvIGFib3V0DQorICAqIHRoZSBkYXRhYmFzZSdzIGNoYXJhY3RlciBlbmNv ZGluZy4NCisgICoNCisgICogUGFsbGUgR2lyZ2Vuc29obiwgMjAwNQ0KKyAg Ki8NCisgDQorIHBnX2VuYzJuYW1lIHBnX2VuYzJpYW5hbmFtZV90YmxbXSA9 DQorIHsNCisgCXsNCisgCQkiVVMtQVNDSUkiLCBQR19TUUxfQVNDSUkNCisg CX0sDQorIAl7DQorIAkJIkVVQy1KUCIsIFBHX0VVQ19KUA0KKyAJfSwNCisg CXsNCisgCQkiR0IyMzEyIiwgUEdfRVVDX0NODQorIAl9LA0KKyAJew0KKyAJ CSJFVUMtS1IiLCBQR19FVUNfS1INCisgCX0sDQorIAl7DQorIAkJIklTTy0y MDIyLUNOIiwgUEdfRVVDX1RXDQorIAl9LA0KKyAJew0KKyAJCSJLU19DXzU2 MDEtMTk4NyIsIFBHX0pPSEFCICAvKiBlaXRoZXIgS1NfQ181NjAxLTE5ODcg b3IgSVNPLTIwMjItS1IgPz8/ICovDQorIAl9LA0KKyAJew0KKyAJCSJVVEYt OCIsIFBHX1VURjgNCisgCX0sDQorIAl7DQorIAkJIk1VTEVfSU5URVJOQUwi LCBQR19NVUxFX0lOVEVSTkFMICAvKiBpcyBub3QgZm9yIHJlYWwgKi8NCisg CX0sDQorIAl7DQorIAkJIklTTy04ODU5LTEiLCBQR19MQVRJTjENCisgCX0s DQorIAl7DQorIAkJIklTTy04ODU5LTIiLCBQR19MQVRJTjINCisgCX0sDQor IAl7DQorIAkJIklTTy04ODU5LTMiLCBQR19MQVRJTjMNCisgCX0sDQorIAl7 DQorIAkJIklTTy04ODU5LTQiLCBQR19MQVRJTjQNCisgCX0sDQorIAl7DQor IAkJIklTTy04ODU5LTkiLCBQR19MQVRJTjUNCisgCX0sDQorIAl7DQorIAkJ IklTTy04ODU5LTEwIiwgUEdfTEFUSU42DQorIAl9LA0KKyAJew0KKyAJCSJJ U08tODg1OS0xMyIsIFBHX0xBVElONw0KKyAJfSwNCisgCXsNCisgCQkiSVNP LTg4NTktMTQiLCBQR19MQVRJTjgNCisgCX0sDQorIAl7DQorIAkJIklTTy04 ODU5LTE1IiwgUEdfTEFUSU45DQorIAl9LA0KKyAJew0KKyAJCSJJU08tODg1 OS0xNiIsIFBHX0xBVElOMTANCisgCX0sDQorIAl7DQorIAkJIndpbmRvd3Mt MTI1NiIsIFBHX1dJTjEyNTYNCisgCX0sDQorIAl7DQorIAkJIndpbmRvd3Mt MTI1OCIsIFBHX1dJTjEyNTgNCisgCX0sDQorIAl7DQorIAkJIndpbmRvd3Mt ODc0IiwgUEdfV0lOODc0DQorIAl9LA0KKyAJew0KKyAJCSJLT0k4LVIiLCBQ R19LT0k4Ug0KKyAJfSwNCisgCXsNCisgCQkid2luZG93cy0xMjUxIiwgUEdf V0lOMTI1MQ0KKyAJfSwNCisgCXsNCisgCQkiSUJNODY2IiwgUEdfV0lOODY2 DQorIAl9LA0KKyAJew0KKyAJCSJJU08tODg1OS01IiwgUEdfSVNPXzg4NTlf NQ0KKyAJfSwNCisgCXsNCisgCQkiSVNPLTg4NTktNiIsIFBHX0lTT184ODU5 XzYNCisgCX0sDQorIAl7DQorIAkJIklTTy04ODU5LTciLCBQR19JU09fODg1 OV83DQorIAl9LA0KKyAJew0KKyAJCSJJU08tODg1OS04IiwgUEdfSVNPXzg4 NTlfOA0KKyAJfSwNCisgCXsNCisgCQkid2luZG93cy0xMjUwIiwgUEdfV0lO MTI1MA0KKyAJfSwNCisgCXsNCisgCQkiU2hpZnRfSklTIiwgUEdfU0pJUw0K KyAJfSwNCisgCXsNCisgCQkiQmlnNSIsIFBHX0JJRzUNCisgCX0sDQorIAl7 DQorIAkJIkdCSyIsIFBHX0dCSw0KKyAJfSwNCisgCXsNCisgCQkiY3A5NDki LCBQR19VSEMNCisgCX0sDQorIAl7DQorIAkJIkdCMTgwMzAiLCBQR19HQjE4 MDMwDQorIAl9DQorIH07DQorICNlbmRpZiAvKiBVU0VfSUNVICovDQorIA0K ICAvKiAtLS0tLS0tLS0tDQogICAqIEVuY29kaW5nIGNoZWNrcywgZm9yIGVy cm9yIHJldHVybnMgLTEgZWxzZSBlbmNvZGluZyBpZA0KICAgKiAtLS0tLS0t LS0tDQpJbmRleDogc3JjL2JhY2tlbmQvdXRpbHMvbWIvbWJ1dGlscy5jDQo9 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09DQpSQ1MgZmlsZTogL3Byb2plY3RzL2N2 c3Jvb3QvcGdzcWwvc3JjL2JhY2tlbmQvdXRpbHMvbWIvbWJ1dGlscy5jLHYN CnJldHJpZXZpbmcgcmV2aXNpb24gMS41MA0KZGlmZiAtYyAtcjEuNTAgbWJ1 dGlscy5jDQoqKiogc3JjL2JhY2tlbmQvdXRpbHMvbWIvbWJ1dGlscy5jCTEw IEp1bCAyMDA1IDIxOjEzOjU5IC0wMDAwCTEuNTANCi0tLSBzcmMvYmFja2Vu ZC91dGlscy9tYi9tYnV0aWxzLmMJMjggSnVsIDIwMDUgMTY6MzY6NTQgLTAw MDANCioqKioqKioqKioqKioqKg0KKioqIDE1LDIwICoqKioNCi0tLSAxNSwy MyAtLS0tDQogICNpbmNsdWRlICJ1dGlscy9tZW11dGlscy5oIg0KICAjaW5j bHVkZSAidXRpbHMvc3lzY2FjaGUuaCINCiAgI2luY2x1ZGUgImNhdGFsb2cv bmFtZXNwYWNlLmgiDQorICNpZmRlZiBVU0VfSUNVDQorICNpbmNsdWRlIDx1 bmljb2RlL3VjbnYuaD4NCisgI2VuZGlmIC8qIFVTRV9JQ1UgKi8NCiAgDQog IC8qDQogICAqIFdlIGhhbmRsZSBmb3IgYWN0dWFsIEZFIGFuZCBCRSBlbmNv ZGluZyBzZXR0aW5nIGVuY29kaW5nLWlkZW50aWZpY2F0b3INCioqKioqKioq KioqKioqKg0KKioqIDU3Niw1ODEgKioqKg0KLS0tIDU3OSw1ODcgLS0tLQ0K ICANCiAgCURhdGFiYXNlRW5jb2RpbmcgPSAmcGdfZW5jMm5hbWVfdGJsW2Vu Y29kaW5nXTsNCiAgCUFzc2VydChEYXRhYmFzZUVuY29kaW5nLT5lbmNvZGlu ZyA9PSBlbmNvZGluZyk7DQorICNpZmRlZiBVU0VfSUNVDQorIAl1Y252X3Nl dERlZmF1bHROYW1lKCgmcGdfZW5jMmlhbmFuYW1lX3RibFtlbmNvZGluZ10p LT5uYW1lKTsNCisgI2VuZGlmDQogIH0NCiAgDQogIHZvaWQNCkluZGV4OiBz cmMvaW5jbHVkZS9wZ19jb25maWcuaC5pbg0KPT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PQ0KUkNTIGZpbGU6IC9wcm9qZWN0cy9jdnNyb290L3Bnc3FsL3NyYy9p bmNsdWRlL3BnX2NvbmZpZy5oLmluLHYNCnJldHJpZXZpbmcgcmV2aXNpb24g MS44Ng0KZGlmZiAtYyAtcjEuODYgcGdfY29uZmlnLmguaW4NCioqKiBzcmMv aW5jbHVkZS9wZ19jb25maWcuaC5pbgkxIEp1bCAyMDA1IDE4OjE3OjMxIC0w MDAwCTEuODYNCi0tLSBzcmMvaW5jbHVkZS9wZ19jb25maWcuaC5pbgkyOCBK dWwgMjAwNSAxNjozNjo1NCAtMDAwMA0KKioqKioqKioqKioqKioqDQoqKiog NjQyLDY0NyAqKioqDQotLS0gNjQyLDY1MCAtLS0tDQogIC8qIERlZmluZSB0 byBidWlsZCB3aXRoIChPcGVuKVNTTCBzdXBwb3J0LiAoLS13aXRoLW9wZW5z c2wpICovDQogICN1bmRlZiBVU0VfU1NMDQogIA0KKyAvKiBEZWZpbmUgdG8g YnVpbGQgd2l0aCBJQ1Ugc3VwcG9ydC4gKC0td2l0aC1pY3UpICovDQorICN1 bmRlZiBVU0VfSUNVDQorIA0KICAvKiBEZWZpbmUgdG8gc2VsZWN0IFN5c1Yt c3R5bGUgc2VtYXBob3Jlcy4gKi8NCiAgI3VuZGVmIFVTRV9TWVNWX1NFTUFQ SE9SRVMNCiAgDQpJbmRleDogc3JjL2luY2x1ZGUvbWIvcGdfd2NoYXIuaA0K PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PQ0KUkNTIGZpbGU6IC9wcm9qZWN0cy9j dnNyb290L3Bnc3FsL3NyYy9pbmNsdWRlL21iL3BnX3djaGFyLmgsdg0KcmV0 cmlldmluZyByZXZpc2lvbiAxLjU5DQpkaWZmIC1jIC1yMS41OSBwZ193Y2hh ci5oDQoqKiogc3JjL2luY2x1ZGUvbWIvcGdfd2NoYXIuaAkxNSBKdW4gMjAw NSAwMDoxNTowOCAtMDAwMAkxLjU5DQotLS0gc3JjL2luY2x1ZGUvbWIvcGdf d2NoYXIuaAkyOCBKdWwgMjAwNSAxNjozNjo1NCAtMDAwMA0KKioqKioqKioq KioqKioqDQoqKiogMjM1LDI0MCAqKioqDQotLS0gMjM1LDI0MSAtLS0tDQog IH0gcGdfZW5jMm5hbWU7DQogIA0KICBleHRlcm4gcGdfZW5jMm5hbWUgcGdf ZW5jMm5hbWVfdGJsW107DQorIGV4dGVybiBwZ19lbmMybmFtZSBwZ19lbmMy aWFuYW5hbWVfdGJsW107DQogIA0KICBleHRlcm4gcGdfZW5jbmFtZSAqcGdf Y2hhcl90b19lbmNuYW1lX3N0cnVjdChjb25zdCBjaGFyICpuYW1lKTsNCiAg DQpJbmRleDogc3JjL2luY2x1ZGUvcG9ydC93aW4zMi5oDQo9PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09DQpSQ1MgZmlsZTogL3Byb2plY3RzL2N2c3Jvb3QvcGdz cWwvc3JjL2luY2x1ZGUvcG9ydC93aW4zMi5oLHYNCnJldHJpZXZpbmcgcmV2 aXNpb24gMS40Ng0KZGlmZiAtYyAtcjEuNDYgd2luMzIuaA0KKioqIHNyYy9p bmNsdWRlL3BvcnQvd2luMzIuaAkxNiBKdW4gMjAwNSAxNzo1Mzo1NCAtMDAw MAkxLjQ2DQotLS0gc3JjL2luY2x1ZGUvcG9ydC93aW4zMi5oCTMwIEp1bCAy MDA1IDE3OjM5OjM2IC0wMDAwDQoqKioqKioqKioqKioqKioNCioqKiAyNTYs MjU4ICoqKioNCi0tLSAyNTYsMjYxIC0tLS0NCiAgDQogIC8qIGluIGJhY2tl bmQvcG9ydC93aW4zMi9lcnJvci5jICovDQogIGV4dGVybiB2b2lkIF9kb3Nt YXBlcnIodW5zaWduZWQgbG9uZyk7DQorIA0KKyAvKiBpbiBiYWNrZW5kL3Bv cnQvd2luMzIvbG9jYWxlbWFwLmMgKi8NCisgZXh0ZXJuIGNoYXIgKnBnd2lu MzJfbG9jYWxlbWFwKGNoYXIgKndpbmxvY2FsZSk7DQo= ------_=_NextPart_001_01C5993C.1E1CB100 Content-Type: application/octet-stream; name="localemap.pl" Content-Transfer-Encoding: base64 Content-Description: localemap.pl Content-Disposition: attachment; filename="localemap.pl" IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMNCiMgbG9jYWxlbWFwLnBs IC0tIEJ1aWxkIGxvY2FsZSBtYXAgYmFzZWQgb24gc291cmNlIGRhdGEgZmls ZXMNCiMNCiMgQ29weXJpZ2h0IChjKSAyMDA1LCBQb3N0Z3JlU1FMIEdsb2Jh bCBEZXZlbG9wbWVudCBHcm91cA0KIw0KIyAkUG9zdGdyZVNRTCQNCiMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMj IyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjDQojDQojIFRoZSBmaWxlIGlzbzYz OSBjb250YWlucyBJU08tNjM5IGxhbmd1YWdlIGNvZGVzLCBjdXQtYW5kLXBh c3RlIGZyb20NCiMgaHR0cDovL3d3dy53My5vcmcvV0FJL0VSL0lHL2VydC9p c282MzkuaHRtLg0KIw0KIyBUaGUgZmlsZSBpc28zMTY2IGNvbnRhaW5zIElT Ty0zMTY2IGNvdW50cnkgY29kZXMsIGN1dC1hbmQtcGFzdGUgZnJvbQ0KIyBo dHRwOi8vd3d3Lm9hc2lzLW9wZW4ub3JnL2NvdmVyL2NvdW50cnkzMTY2Lmh0 bWwsDQojDQoNCm9wZW4oSSwiaXNvNjM5IikgfHwgZGllICJDb3VsZCBub3Qg b3BlbiBpc282MzlcbiI7DQp3aGlsZSAoPEk+KSB7DQoJY2hvbXA7DQoJZGll ICJNYWxmb3JtYXR0ZWQgbGluZSBpbiBpc282MzkiIHVubGVzcyAvXihbQS1a XXsyfSkgXCIoLispXCIkLzsNCgkkY29kZSA9IGxjICQxOw0KCSRsYW5nID0g bGMgJDI7DQoJaWYgKCRsYW5nID1+IC9eKC4rKVwiIC8pIHsgJGxhbmcgPSAk MTsgfQ0KCSRpc282Mzl7JGxhbmd9PSRjb2RlOw0KfQ0KY2xvc2UoSSk7DQoN Cm9wZW4oSSwiaXNvMzE2NiIpIHx8IGRpZSAiQ291bGQgbm90IG9wZW4gaXNv MzE2NlxuIjsNCndoaWxlICg8ST4pIHsNCgljaG9tcDsNCglkaWUgIk1hbGZv cm1hdHRlZCBsaW5lIGluIGlzbzMxNjYiIHVubGVzcyAvXihbQS1aXXsyfSlc cysoLispJC87DQoJJGNvZGUgPSBsYyAkMTsNCgkkY291bnRyeSA9IGxjICQy Ow0KCSRpc28zMTY2eyRjb3VudHJ5fSA9ICRjb2RlOw0KfQ0KY2xvc2UoSSk7 DQoNCnByaW50IDw8RU9GOw0KdHlwZWRlZiBzdHJ1Y3Qgew0KCWNoYXIgKmtl eTsNCgljaGFyICp2YWw7DQp9IF9rZXl2YWw7DQoNCnN0YXRpYyBfa2V5dmFs IGlzbzYzOVtdID0gew0KRU9GDQpmb3IgJGsgKHNvcnQga2V5cyAlaXNvNjM5 KSB7DQoJcHJpbnQgIiB7XCIka1wiLFwiJGlzbzYzOXska31cIn0sXG4iOw0K fQ0KcHJpbnQgIiB7TlVMTCxOVUxMfVxufTtcblxuIjsNCg0KcHJpbnQgInN0 YXRpYyBfa2V5dmFsIGlzbzMxNjZbXSA9IHtcbiI7DQpmb3IgJGsgKHNvcnQg a2V5cyAlaXNvMzE2Nikgew0KCXByaW50ICIge1wiJGtcIixcIiRpc28zMTY2 eyRrfVwifSxcbiI7DQp9DQoNCnByaW50ICIge05VTEwsTlVMTH1cbn07XG5c biI7DQo= ------_=_NextPart_001_01C5993C.1E1CB100 Content-Type: application/octet-stream; name="localemap.c" Content-Transfer-Encoding: base64 Content-Description: localemap.c Content-Disposition: attachment; filename="localemap.c" LyoNCiAqIGxvY2FsZW1hcCAtIG1hcCB3aW4zMiBsb2NhbGUgbmFtZXMgdG8g SUNVIHN0YW5kYXJkIG5hbWVzDQogKg0KICogQ29weXJpZ2h0IChDKSAyMDA1 LCBQb3N0Z3JlU1FMIEdsb2JhbCBEZXZlbG9wbWVudCBHcm91cA0KICoNCiAq ICRQb3N0Z3JlU1FMJA0KICovDQojaW5jbHVkZSAicG9zdGdyZXMuaCIgDQoN CiNpbmNsdWRlICJsb2NhbGVtYXAuaCINCg0KDQpzdGF0aWMgY2hhciAqbWF0 Y2hfbGFuZ3VhZ2UoY2hhciAqd2lubGFuZ3VhZ2UpDQp7DQoJX2tleXZhbCAq a3Y7DQoNCglmb3IgKGt2ID0gaXNvNjM5OyBrdi0+a2V5OyBrdisrKSB7DQoJ CWlmICghc3RyY2FzZWNtcChrdi0+a2V5LCB3aW5sYW5ndWFnZSkpDQoJCQly ZXR1cm4ga3YtPnZhbDsNCgl9DQoJZXJlcG9ydChFUlJPUiwNCgkJCShlcnJt c2coImNvdWxkIG5vdCBtYXRjaCBsYW5ndWFnZSBmb3IgJXMiLCB3aW5sYW5n dWFnZSkpKTsNCglyZXR1cm4gTlVMTDsNCn0NCg0Kc3RhdGljIGNoYXIgKm1h dGNoX2NvdW50cnkoY2hhciAqd2luY291bnRyeSkNCnsNCglfa2V5dmFsICpr djsNCg0KCWZvciAoa3YgPSBpc28zMTY2OyBrdi0+a2V5OyBrdisrKSB7DQoJ CWlmICghc3RyY2FzZWNtcChrdi0+a2V5LCB3aW5jb3VudHJ5KSkNCgkJCXJl dHVybiBrdi0+dmFsOw0KCX0NCglyZXR1cm4gTlVMTDsNCn0NCg0KLyoNCiAq IEF0dGVtcHQgdG8gY29udmVydCBhIHdpbjMyIGxvY2FsZSAoU3dlZGlzaF9T d2VkZW4uMTI1MikNCiAqIHRvIElDVSBmb3JtYXQgKHN2X3NlKQ0KICovDQpz dGF0aWMgY2hhciBvdXRidWZbMTZdOw0KY2hhciAqcGd3aW4zMl9sb2NhbGVt YXAoY2hhciAqd2lubG9jYWxlKQ0Kew0KCWNoYXIgYnVmWzI1Nl07DQoJY2hh ciAqdW5kZXJzY29yZTsNCgljaGFyICpkb3Q7DQoJY2hhciAqY291bnRyeTsN Cg0KCWlmIChzdHJsZW4od2lubG9jYWxlKT4yNTUpIA0KCQllcmVwb3J0KEVS Uk9SLA0KCQkJCShlcnJtc2dfaW50ZXJuYWwoImxvY2FsZSBuYW1lIHRvbyBs b25nIikpKTsNCglzdHJjcHkoYnVmLHdpbmxvY2FsZSk7DQoNCgl1bmRlcnNj b3JlID0gc3RyY2hyKGJ1ZiwnXycpOw0KCWlmICghdW5kZXJzY29yZSkNCgkJ LyogT25seSBsYW5ndWFnZSBuYW1lICovDQoJCXJldHVybiBtYXRjaF9sYW5n dWFnZShidWYpOw0KDQoJKnVuZGVyc2NvcmUgPSAwOw0KCXVuZGVyc2NvcmUr KzsNCg0KCWRvdCA9IHN0cmNocih1bmRlcnNjb3JlLCcuJyk7DQoJaWYgKGRv dCkNCgkJLyogSWYgY29kZXBhZ2UgaXMgaW5jbHVkZWQsIHdlIGp1c3QgaWdu b3JlIGl0LiAqLw0KCQkqZG90ID0gMDsNCgkNCgljb3VudHJ5ID0gbWF0Y2hf Y291bnRyeSh1bmRlcnNjb3JlKTsNCglpZiAoIWNvdW50cnkpDQoJCXJldHVy biBtYXRjaF9sYW5ndWFnZShidWYpOw0KDQoJc3ByaW50ZihvdXRidWYsIiVz XyVzIixtYXRjaF9sYW5ndWFnZShidWYpLGNvdW50cnkpOw0KCXJldHVybiBv dXRidWY7DQp9DQo= ------_=_NextPart_001_01C5993C.1E1CB100 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 ---------------------------(end of broadcast)--------------------------- TIP 4: Have you searched our list archives? http://archives.postgresql.org ------_=_NextPart_001_01C5993C.1E1CB100-- From pgsql-hackers-owner+M72011@postgresql.org Wed Aug 24 15:37:17 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j7OJbH103190 for ; Wed, 24 Aug 2005 15:37:17 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id D5152B21F6B; Wed, 24 Aug 2005 19:37:12 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 34C03D7357 for ; Wed, 24 Aug 2005 15:53:55 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 85477-03 for ; Wed, 24 Aug 2005 18:53:52 +0000 (GMT) Received: from rodrick.geeknet.com.au (ns1.geeknet.com.au [220.244.63.182]) by svr1.postgresql.org (Postfix) with ESMTP id 7C774D78F9 for ; Wed, 24 Aug 2005 15:53:48 -0300 (ADT) Subject: Re: [HACKERS] FreeBSD ICU was Win32 unicode vs ICU MIME-Version: 1.0 Date: Thu, 25 Aug 2005 04:53:47 +1000 Content-Type: text/plain; charset="us-ascii" Message-ID: <5066E5A966339E42AA04BA10BA706AE50A938F@rodrick.geeknet.com.au> content-class: urn:content-classes:message X-MimeOLE: Produced By Microsoft Exchange V6.5.6944.0 Thread-Topic: [HACKERS] FreeBSD ICU was Win32 unicode vs ICU Thread-Index: AcWo0l6W575cyph2Twi7Fhfaa+w/tAACptMA From: "John Hansen" To: "Kevin McArthur" , X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.017 required=5 tests=[AWL=0.017] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from quoted-printable to 8bit by candle.pha.pa.us id j7OJbH103190 Status: OR Kevin McArthur Wrote: > Should the postgresql project also be looking at CLDR for > cross-platform unicode support? Afaict, from the ICU website, ICU too uses CLDR. Why reinvent the wheel? ... John ---------------------------(end of broadcast)--------------------------- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M72581@postgresql.org Sat Sep 3 16:47:42 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j83Klg129281 for ; Sat, 3 Sep 2005 16:47:42 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id CA484B21F00; Sat, 3 Sep 2005 20:47:36 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 354A7D8B35 for ; Sat, 3 Sep 2005 17:35:06 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 55971-07 for ; Sat, 3 Sep 2005 20:34:58 +0000 (GMT) Received: from svana.org (svana.org [203.20.62.76]) by svr1.postgresql.org (Postfix) with ESMTP id 4AFECD8AE7 for ; Sat, 3 Sep 2005 17:34:56 -0300 (ADT) Received: from kleptog by svana.org with local (Exim 3.35 #1 (Debian)) id 1EBeim-0002xI-00; Sun, 04 Sep 2005 06:34:40 +1000 Date: Sat, 3 Sep 2005 22:34:40 +0200 From: Martijn van Oosterhout To: Tom Lane cc: Greg Stark , pgsql-hackers@postgresql.org Subject: Locale implementation questions (was: [HACKERS] Proof of concept COLLATE support with patch) Message-ID: <20050903203434.GA4281@svana.org> Reply-To: Martijn van Oosterhout References: <20050902130420.GA15466@svana.org> <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mP3DRpeJDSE+ciuQ" Content-Disposition: inline In-Reply-To: <14696.1125675741@sss.pgh.pa.us> User-Agent: Mutt/1.3.28i X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6 X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522 48A7 6394 F08A 0DC6 7BE6 X-PGP-Key-URL: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.003 required=5 tests=[AWL=0.003] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR --mP3DRpeJDSE+ciuQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Sep 02, 2005 at 11:42:21AM -0400, Tom Lane wrote: > The objection is fundamentally that a platform-specific implementation > cannot be our long-term goal, and so expending effort on creating one > seems like a diversion. If there were a plan put forward showing how > this is just a useful way-station, and we could see how we'd later get > rid of the glibc dependency without throwing away the work already done, > then it would be a different story. Well, my patch showed that useful locale work can be acheived with precisely two functions: newlocale and strxfrm_l. I'm going to talk about two things: one, the code from Apple. Two, how we present locale support to users. --- Now, it would be really nice to take Apple's implementation in Darwin and use that. What I don't understand is the licence of the code in Darwin. My interpretation is that stuff in: http://darwinsource.opendarwin.org/10.4.2/Libc-391/locale/ is Apple stuff under APSL, useless to us. And that stuff in: http://darwinsource.opendarwin.org/10.4.2/Libc-391/locale/FreeBSD/ are just patches to FreeBSD and this under the normal BSD license (no big header claiming the licence change). The good news is that the majority of what we need is in patch form. The bad news is that the hub of the good stuff (newlocale, duplocale, freelocale) is under a big fat APSL licence. Does anyone know if this code can be used at all by BSD projects or did they blanket relicence everything? --- Now, I want to bring up some points relating to including a locale library in PostgreSQL. Given that none of the BSDs seem really interested in fixing the issue we'll have to do it ourselves (I don't see anyone else doing it). We can save ourselves effort by basing it on FreeBSDs locale code, because then we can use their datafiles, which we *definitly* don't want to maintain ourselves. Now: 1. FreeBSDs locale list is short, some 48 compared with glibc's 217. Hopefully Apple can expand on that in a way we can use. But given the difference we should probably give people a way of falling back to the system libraries in case there's a locale we don't support. On the other hand, lots of locales are similar so maybe people can find ones close enough to work. No, glibc and FreeBSD use different file formats, so you can't copy them. Do we want this locale data just for collation, or do we want to be able to use it for formatting monetary amounts too? This is even more info to store. Lots of languages use ISO/IEC 14651 for order. 2. Locale data needs to be combined with a charset and compiled to work with the library. PostgreSQL supports at least 15 charsets but we don't want to ship compiled versions of all of these (Debian learnt that the hard way). So, how do we generate the files people need. a. Auto-compile on demand. First time a locale is referenced spawn the compiler to create the locale, then continue. (Ugh) b. Add a CREATE LOCALE english AS 'en_US' WITH CHARSET 'utf8'. Then require the COLLATE clause to refer to this identifier. This has some appeal, seperating the system names from the PostgreSQL names. It also gives some info regarding charsets. c. Should users be allowed to define new locales? d. Should admins be required to create the external files using a program, say pg_createlocale. Remember, if you use a latin1 locale to sort utf8 you'll get the wrong result, so we want to avoid that. 3. Compiled locale files are large. One UTF-8 locale datafile can exceed a megabyte. Do we want the option of disabling it for small systems? 4. Do we want the option of running system locale in parallel with the internal ones? 5. I think we're going to have to deal with the very real possibility that our locale database will not be as good as some of the system provided ones. The question is how. This is quite unlike timezones which are quite standardized and rarely change. That database is quite well maintained. Would people object to a configure option that selected: --with-locales=3Dinternal (use pg database) --with-locales=3Dsystem (use system database for win32, glibc or Ma= cOS X) --with-locales=3Dnone (what we support now, which is neither) I don't think it will be much of an issue to support this, all the functions take the same parameters and have almost the same names. 6. Locales for SQL_ASCII. Seems to me you have two options, either reject COLLATE altogether unless they specify a charset, or don't care and let the user shoot themselves in the foot if they wish... BTW, this MacOS locale supports seems to be new for 10.4.2 according to the CVS log info, can anyone confirm this? Anyway, I hope this post didn't bore too much. Locale support has been one of those things that has bugged me for a long time and it would be nice if there could be some real movement. Have a nice weekend, --=20 Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. --mP3DRpeJDSE+ciuQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQFDGgjYIB7bNG8LQkwRAr6OAJ9uqNDKDQKWUAY4KiPAazHJ1TsVWwCeJ7sq 7hcILjdgZTQ4LjyPAhWnJwQ= =5MT9 -----END PGP SIGNATURE----- --mP3DRpeJDSE+ciuQ-- From pgsql-hackers-owner+M72583@postgresql.org Sat Sep 3 17:54:58 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j83Lsv108133 for ; Sat, 3 Sep 2005 17:54:57 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 3325AB21F00; Sat, 3 Sep 2005 21:54:51 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id DA9AED8B14 for ; Sat, 3 Sep 2005 18:44:52 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 88603-07 for ; Sat, 3 Sep 2005 21:44:51 +0000 (GMT) Received: from stark.xeocode.com (stark.xeocode.com [216.58.44.227]) by svr1.postgresql.org (Postfix) with ESMTP id 01864D8B50 for ; Sat, 3 Sep 2005 18:44:50 -0300 (ADT) Received: from localhost ([127.0.0.1] helo=stark.xeocode.com) by stark.xeocode.com with smtp (Exim 3.36 #1 (Debian)) id 1EBfog-0004U8-00; Sat, 03 Sep 2005 17:44:50 -0400 To: Martijn van Oosterhout cc: Tom Lane , Greg Stark , pgsql-hackers@postgresql.org Subject: Re: Locale implementation questions (was: [HACKERS] Proof of concept COLLATE support with patch) References: <20050902130420.GA15466@svana.org> <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <20050903203434.GA4281@svana.org> In-Reply-To: <20050903203434.GA4281@svana.org> From: Greg Stark Organization: The Emacs Conspiracy; member since 1992 Date: 03 Sep 2005 17:44:50 -0400 Message-ID: <87oe79ygfh.fsf@stark.xeocode.com> Lines: 39 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.005 required=5 tests=[AWL=0.005] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Martijn van Oosterhout writes: > 2. Locale data needs to be combined with a charset and compiled to work > with the library. PostgreSQL supports at least 15 charsets but we don't > want to ship compiled versions of all of these (Debian learnt that the > hard way). So, how do we generate the files people need. That's just one of many lessons learned the hard way by distributions. Nor will it be the last innovation in this area. I really find this instinct of wanting to reimplement large swaths of the OS inside Postgres (and annoying detail-ridden swaths that are hard to get right and continually evolving too) to be a bad idea. I can't believe it's harder to maintain an #ifdef HAVE_STRCOL_L #else #endif than it is to try to maintain an entire independent locale library. Nor is it simpler for sysadmins to have to maintain an entirely separate set of locales independently from the system locales. If you really are unhappy enough with OS setlocale implementations to want to try to do this then it would be more helpful to do it outside of Postgres. Package up the Apple setlocale library as a separate package that anyone can install on Solaris, BSD, Linux or whatever. Then Postgres can just say "it works fine with your OS library but your OS library might be very slow. Here's a third-party library that you can install that is fast and may relieve any problems you have with collation performance." But I think that's getting ahead of things. Until Postgres even supports collations using the OS libraries you won't even know if that's even necessary. -- greg ---------------------------(end of broadcast)--------------------------- TIP 4: Have you searched our list archives? http://archives.postgresql.org From pgsql-hackers-owner+M72585@postgresql.org Sun Sep 4 08:44:01 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84Ci0127486 for ; Sun, 4 Sep 2005 08:44:01 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 8ED1DB21F00; Sun, 4 Sep 2005 12:43:55 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id D87D4D6E38 for ; Sun, 4 Sep 2005 09:31:31 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 80362-07 for ; Sun, 4 Sep 2005 12:31:29 +0000 (GMT) Received: from svana.org (svana.org [203.20.62.76]) by svr1.postgresql.org (Postfix) with ESMTP id C0F8DD6D8D for ; Sun, 4 Sep 2005 09:31:24 -0300 (ADT) Received: from kleptog by svana.org with local (Exim 3.35 #1 (Debian)) id 1EBteM-0005wf-00; Sun, 04 Sep 2005 22:31:06 +1000 Date: Sun, 4 Sep 2005 14:31:05 +0200 From: Martijn van Oosterhout To: Greg Stark cc: Tom Lane , pgsql-hackers@postgresql.org Subject: Re: Locale implementation questions (was: [HACKERS] Proof of concept COLLATE support with patch) Message-ID: <20050904123103.GA21198@svana.org> Reply-To: Martijn van Oosterhout References: <20050902130420.GA15466@svana.org> <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <20050903203434.GA4281@svana.org> <87oe79ygfh.fsf@stark.xeocode.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="EeQfGwPcQSOJBaQU" Content-Disposition: inline In-Reply-To: <87oe79ygfh.fsf@stark.xeocode.com> User-Agent: Mutt/1.3.28i X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6 X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522 48A7 6394 F08A 0DC6 7BE6 X-PGP-Key-URL: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.003 required=5 tests=[AWL=0.003] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR --EeQfGwPcQSOJBaQU Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Sep 03, 2005 at 05:44:50PM -0400, Greg Stark wrote: > [...] Nor is it > simpler for sysadmins to have to maintain an entirely separate set of loc= ales > independently from the system locales. Indeed, I was already coming up with mechanisms to determine what locales the system uses and try to autogenerate them. I agree though, it's not useful for systems that already have complete locale support. Why add to the burden? Anyway, my reading of the specs says that we must support the syntax. It doesn't say we need to support any orderings other than the default (ie what we do now). > If you really are unhappy enough with OS setlocale implementations to wan= t to > try to do this then it would be more helpful to do it outside of Postgres. > Package up the Apple setlocale library as a separate package that anyone = can > install on Solaris, BSD, Linux or whatever. Then Postgres can just say "it > works fine with your OS library but your OS library might be very slow. H= ere's > a third-party library that you can install that is fast and may relieve a= ny > problems you have with collation performance." That's why I asked about the patches and files that Apple wrote. What are the licence restrictions? Would we be able to download the, what, 20 files and distribute it as a library. Being APSL we couldn't include it in the tarball, but it could be a pgfoundry project or something. If somebody knows a reason why this could not be done, speak up now because my reading of the APSL licence tells me it's fine. > But I think that's getting ahead of things. Until Postgres even supports > collations using the OS libraries you won't even know if that's even > necessary. Well, I added COLLATE support for ORDER BY and CREATE INDEX and it worked in under 200 lines. I'm thinking ahead and I don't think the COLLATE rules are that hard. Implementing them seems a bit fiddly. It may be easiest to consider COLLATE a non-associative operator. I'm still unsure if I should turn the string comparison operators into three-argument functions. Anyway, I'll look into the library issue first. --=20 Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. --EeQfGwPcQSOJBaQU Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQFDGukEIB7bNG8LQkwRAkeaAJ9rj5Tz7WPZpp+wWYWjTdjR68o8DwCfZzcZ uB57noHmcnvefVoaw27bQ5Q= =nLio -----END PGP SIGNATURE----- --EeQfGwPcQSOJBaQU-- From pgsql-hackers-owner+M72582@postgresql.org Sat Sep 3 17:46:57 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j83Lkr106932 for ; Sat, 3 Sep 2005 17:46:57 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 6E192B21F00; Sat, 3 Sep 2005 21:46:47 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 7CF09D72B7 for ; Sat, 3 Sep 2005 18:36:14 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 77080-09 for ; Sat, 3 Sep 2005 21:36:13 +0000 (GMT) Received: from stark.xeocode.com (stark.xeocode.com [216.58.44.227]) by svr1.postgresql.org (Postfix) with ESMTP id 85FC0D6FEB for ; Sat, 3 Sep 2005 18:36:09 -0300 (ADT) Received: from localhost ([127.0.0.1] helo=stark.xeocode.com) by stark.xeocode.com with smtp (Exim 3.36 #1 (Debian)) id 1EBfgA-0004RS-00; Sat, 03 Sep 2005 17:36:02 -0400 To: Tom Lane cc: Greg Stark , Martijn van Oosterhout , pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Proof of concept COLLATE support with patch References: <20050902130420.GA15466@svana.org> <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> In-Reply-To: <14696.1125675741@sss.pgh.pa.us> From: Greg Stark Organization: The Emacs Conspiracy; member since 1992 Date: 03 Sep 2005 17:36:02 -0400 Message-ID: <87u0h1ygu5.fsf@stark.xeocode.com> Lines: 36 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.005 required=5 tests=[AWL=0.005] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Tom Lane writes: > Greg Stark writes: > > I still doesn't get where the hostility towards this functionality comes from. > > We're not really willing to say "here is a piece of syntax REQUIRED > BY THE SQL SPEC which we only support on some platforms". readline, > O_DIRECT, and the like are a completely inappropriate analogy, because > those are inherently platform-dependent (and not in the spec). But that's not the case at all. The syntax can be supported everywhere it would just be somewhat faster on some platforms than others. It's already reasonably fast on any platform that caches locale information which includes glibc and presumably other free software libcs. It would be slightly faster if there are _l functions. And much slower if the libc setlocale implementation is braindead. But there's nothing wrong with saying "it's slow because your libc is slow. Compile with this freely available library which has a better implementation". The programming syntax would still be exactly 100% the same. > The objection is fundamentally that a platform-specific implementation > cannot be our long-term goal, and so expending effort on creating one > seems like a diversion. If there were a plan put forward showing how > this is just a useful way-station, and we could see how we'd later get > rid of the glibc dependency without throwing away the work already done, > then it would be a different story. It's not like the actual calls to setlocale are going to be much code. One day presumably some variant of these _l functions will become entirely standard. In which case you're talking about potentially "throwing away" 50 lines of code. The bulk of the code is going to be parsing and implementing the actual syntax and behaviour of the SQL spec. And in any case I wouldn't expect it to ever get thrown away. There will be people compiling on RH9 or similar vintage systems for a long time. -- greg ---------------------------(end of broadcast)--------------------------- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq From pgsql-hackers-owner+M72589@postgresql.org Sun Sep 4 13:20:26 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84HKQ121839 for ; Sun, 4 Sep 2005 13:20:26 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id E4A85B21FBB; Sun, 4 Sep 2005 17:20:23 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id C4CA7D7AEA for ; Sun, 4 Sep 2005 14:07:02 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 63550-04 for ; Sun, 4 Sep 2005 17:07:01 +0000 (GMT) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by svr1.postgresql.org (Postfix) with SMTP id 02A4FD7A6F for ; Sun, 4 Sep 2005 14:06:59 -0300 (ADT) Received: (qmail invoked by alias); 04 Sep 2005 17:06:59 -0000 Received: from dsl-082-083-230-040.arcor-ip.net (EHLO colt.pezone.net) [82.83.230.40] by mail.gmx.net (mp033) with SMTP; 04 Sep 2005 19:06:59 +0200 X-Authenticated: #495269 From: Peter Eisentraut To: pgsql-hackers@postgresql.org, Martijn van Oosterhout Subject: Re: [HACKERS] Proof of concept COLLATE support with patch Date: Sun, 4 Sep 2005 19:06:57 +0200 User-Agent: KMail/1.8.1 References: <20050902130420.GA15466@svana.org> In-Reply-To: <20050902130420.GA15466@svana.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-ID: <200509041906.57721.peter_e@gmx.net> X-Y-GMX-Trusted: 0 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.018 required=5 tests=[AWL=0.018] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Martijn van Oosterhout wrote: > This is just a proof of concept patch. I didn't send it to -patches > because as Tom pointed out, there's no hope of it getting in due to > platform dependant behaviour. I think it would be best if we defined an internal API for plugging in various kinds of locale support. Then you can hook in this "newlocale", the Windows variant, ICU, or plain-old POSIX locale support for backward compatibility. You already identified most of the API functions. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---------------------------(end of broadcast)--------------------------- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match From pgsql-hackers-owner+M72592@postgresql.org Sun Sep 4 18:15:47 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84MFk103432 for ; Sun, 4 Sep 2005 18:15:46 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id E623DB21FBF; Sun, 4 Sep 2005 22:15:40 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id D671BD83B4 for ; Sun, 4 Sep 2005 19:06:26 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 56030-04 for ; Sun, 4 Sep 2005 22:06:23 +0000 (GMT) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id 44141D6F63 for ; Sun, 4 Sep 2005 19:06:21 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id j84M6Boq008947; Sun, 4 Sep 2005 18:06:12 -0400 (EDT) To: Greg Stark cc: Martijn van Oosterhout , pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Proof of concept COLLATE support with patch In-Reply-To: <87u0h1ygu5.fsf@stark.xeocode.com> References: <20050902130420.GA15466@svana.org> <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <87u0h1ygu5.fsf@stark.xeocode.com> Comments: In-reply-to Greg Stark message dated "03 Sep 2005 17:36:02 -0400" Date: Sun, 04 Sep 2005 18:06:11 -0400 Message-ID: <8946.1125871571@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.006 required=5 tests=[AWL=0.006] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Greg Stark writes: > But there's nothing wrong with saying "it's slow because your > libc is slow. Compile with this freely available library which has a better > implementation". The hole in that argument is the assumption that there *is* a freely available library that can be used (where freely == BSD license). We wouldn't be having this discussion if we knew of one. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M72596@postgresql.org Sun Sep 4 19:27:26 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84NRQ109968 for ; Sun, 4 Sep 2005 19:27:26 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id A427211ED4E0; Sun, 4 Sep 2005 23:27:20 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 64BFED8B2F for ; Sun, 4 Sep 2005 20:15:22 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 08469-05 for ; Sun, 4 Sep 2005 23:15:21 +0000 (GMT) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id CE6D9D8AD4 for ; Sun, 4 Sep 2005 20:15:19 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id j84NFLVH009399; Sun, 4 Sep 2005 19:15:21 -0400 (EDT) To: Peter Eisentraut cc: pgsql-hackers@postgresql.org, Martijn van Oosterhout Subject: Re: [HACKERS] Proof of concept COLLATE support with patch In-Reply-To: <200509041906.57721.peter_e@gmx.net> References: <20050902130420.GA15466@svana.org> <200509041906.57721.peter_e@gmx.net> Comments: In-reply-to Peter Eisentraut message dated "Sun, 04 Sep 2005 19:06:57 +0200" Date: Sun, 04 Sep 2005 19:15:21 -0400 Message-ID: <9398.1125875721@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.006 required=5 tests=[AWL=0.006] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Peter Eisentraut writes: > I think it would be best if we defined an internal API for plugging in > various kinds of locale support. Agreed ... > Then you can hook in this > "newlocale", the Windows variant, ICU, or plain-old POSIX locale > support for backward compatibility. If plain old POSIX actually did what we needed, we likely wouldn't be having this discussion at all. POSIX doesn't give us enough visibility of the locale's properties (in particular, which character set encoding it wants). The performance penalties it imposes are pretty bad also, though arguably secondary. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M72599@postgresql.org Sun Sep 4 20:01:47 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j8501k113218 for ; Sun, 4 Sep 2005 20:01:46 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 5FCFD11ED4E1; Mon, 5 Sep 2005 00:01:40 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 3CE03D8A88 for ; Sun, 4 Sep 2005 20:53:09 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 14738-01 for ; Sun, 4 Sep 2005 23:53:08 +0000 (GMT) Received: from gateway.pjmodos.bounceme.net (gprsplusa185.isp.t-mobile.cz [62.141.20.185]) by svr1.postgresql.org (Postfix) with ESMTP id B5606D8A4E for ; Sun, 4 Sep 2005 20:53:03 -0300 (ADT) Received: from modos ([10.12.0.96] ident=PJMODOS) by gateway.pjmodos.bounceme.net with esmtp (Exim 3.35 #1 (Debian)) id 1EC4IE-0004Le-00; Mon, 05 Sep 2005 01:52:58 +0200 Message-ID: <431B88CD.8050904@seznam.cz> Date: Mon, 05 Sep 2005 01:52:45 +0200 From: Petr Jelinek User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) X-Accept-Language: cs, en, en-us MIME-Version: 1.0 To: Tom Lane cc: PostgreSQL-development Subject: Re: [HACKERS] Proof of concept COLLATE support with patch References: <20050902130420.GA15466@svana.org> <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <87u0h1ygu5.fsf@stark.xeocode.com> <8946.1125871571@sss.pgh.pa.us> In-Reply-To: <8946.1125871571@sss.pgh.pa.us> Content-Type: text/plain; charset=windows-1250; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=1.173 required=5 tests=[AWL=-0.253, DNS_FROM_RFC_POST=1.376, FORGED_RCVD_HELO=0.05] X-Spam-Level: * X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Tom Lane wrote: > > The hole in that argument is the assumption that there *is* a freely > available library that can be used (where freely == BSD license). > We wouldn't be having this discussion if we knew of one. I see this discussion as another reason to use ICU, I mean complete rewrite of locale handling to use ICU on all platforms. I know it's big project but it's doable for 8.2 and it would virtually solve all locale problems and could be base for new unicode/locale features. I am not sure if this is the way postgres wants to go tho (having dependency on such a big and uncommon library). -- Regards Petr Jelinek (PJMODOS) ---------------------------(end of broadcast)--------------------------- TIP 6: explain analyze is your friend From pgsql-hackers-owner+M72602@postgresql.org Mon Sep 5 04:02:37 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j8582Z126563 for ; Mon, 5 Sep 2005 04:02:36 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 6640811EB014; Mon, 5 Sep 2005 08:02:28 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 1FC36D7841 for ; Mon, 5 Sep 2005 04:52:11 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 09506-02 for ; Mon, 5 Sep 2005 07:52:06 +0000 (GMT) Received: from svana.org (svana.org [203.20.62.76]) by svr1.postgresql.org (Postfix) with ESMTP id 0DC42D77AE for ; Mon, 5 Sep 2005 04:52:06 -0300 (ADT) Received: from kleptog by svana.org with local (Exim 3.35 #1 (Debian)) id 1ECBlg-0001TS-00; Mon, 05 Sep 2005 17:51:52 +1000 Date: Mon, 5 Sep 2005 09:51:52 +0200 From: Martijn van Oosterhout To: Tom Lane cc: Greg Stark , pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Proof of concept COLLATE support with patch Message-ID: <20050905075152.GA5278@svana.org> Reply-To: Martijn van Oosterhout References: <20050902130420.GA15466@svana.org> <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <87u0h1ygu5.fsf@stark.xeocode.com> <8946.1125871571@sss.pgh.pa.us> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="MGYHOYXEY6WxJCY8" Content-Disposition: inline In-Reply-To: <8946.1125871571@sss.pgh.pa.us> User-Agent: Mutt/1.3.28i X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6 X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522 48A7 6394 F08A 0DC6 7BE6 X-PGP-Key-URL: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.003 required=5 tests=[AWL=0.003] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR --MGYHOYXEY6WxJCY8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 04, 2005 at 06:06:11PM -0400, Tom Lane wrote: > Greg Stark writes: > > But there's nothing wrong with saying "it's slow because your > > libc is slow. Compile with this freely available library which has a be= tter > > implementation". >=20 > The hole in that argument is the assumption that there *is* a freely > available library that can be used (where freely =3D=3D BSD license). > We wouldn't be having this discussion if we knew of one. 1. Use the one in Darwin just for the *BSDs and Solaris at least. It's not great but it would probably work. 2. Long term, transition to ICU (http://icu.sourceforge.net/) which is the cross-platform internationalisation library used by Java. Looks like Mono and Gnome/GTK are going to use this (or at least allow use of) soon also. It uses the X licence AFAICS. It's a big pill right now but it a year it could be installed standard on most linux systems. It's at least avaiable everywhere now. Note, it's not compatable with POSIX locales so if we go there it'll be an all or nothing switch. But if we intend to go there eventually, it makes fiddling on our own library a waste of time. Incidently, I played with the code in Darwin and getting it to compile on a system that already has extended locale support is, uh, tricky to say the least. Lots of conflicting definitions. --=20 Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. --MGYHOYXEY6WxJCY8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQFDG/kXIB7bNG8LQkwRAijSAJ4xv5JiigSwEe4jgBmxvqHN+GuOgQCfSEHp C5tpavd58A8VEnf/BBUgfzo= =gsNf -----END PGP SIGNATURE----- --MGYHOYXEY6WxJCY8-- From pgsql-hackers-owner+M72586@postgresql.org Sun Sep 4 09:37:44 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84Dbh100551 for ; Sun, 4 Sep 2005 09:37:43 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 824CCB21F00; Sun, 4 Sep 2005 13:37:37 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 413CCD7015 for ; Sun, 4 Sep 2005 10:26:10 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 20529-10 for ; Sun, 4 Sep 2005 13:26:07 +0000 (GMT) Received: from sraigw.sra.co.jp (sraigw.sra.co.jp [202.32.10.2]) by svr1.postgresql.org (Postfix) with ESMTP id 12EF2D6F75 for ; Sun, 4 Sep 2005 10:26:06 -0300 (ADT) Received: from srascb.sra.co.jp (srascb [133.137.8.65]) by sraigw.sra.co.jp (Postfix) with ESMTP id 08EB262112; Sun, 4 Sep 2005 22:26:09 +0900 (JST) Received: from srascb.sra.co.jp (localhost [127.0.0.1]) by localhost.sra.co.jp (Postfix) with ESMTP id D1D9610CD06; Sun, 4 Sep 2005 22:26:09 +0900 (JST) Received: from sranhm.sra.co.jp (sranhm [133.137.44.16]) by srascb.sra.co.jp (Postfix) with ESMTP id ABF1210CD04; Sun, 4 Sep 2005 22:26:09 +0900 (JST) Received: from localhost (sraihb-hub.sra.co.jp [133.137.8.6]) by sranhm.sra.co.jp (8.9.3+3.2W/3.7W-srambox) with ESMTP id WAA12274; Sun, 4 Sep 2005 22:26:09 +0900 Date: Sun, 04 Sep 2005 22:25:36 +0900 (JST) Message-ID: <20050904.222536.39155679.ishii@sraoss.co.jp> To: kleptog@svana.org cc: tgl@sss.pgh.pa.us, gsstark@mit.edu, pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Locale implementation questions From: Tatsuo Ishii In-Reply-To: <20050903203434.GA4281@svana.org> References: <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <20050903203434.GA4281@svana.org> X-Mailer: Mew version 2.3 on Emacs 20.7 / Mule 4.1 =?iso-2022-jp?B?KBskQjAqGyhCKQ==?= MIME-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.028 required=5 tests=[AWL=0.028] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR > 3. Compiled locale files are large. One UTF-8 locale datafile can > exceed a megabyte. Do we want the option of disabling it for small > systems? To avoid the problem, you could dynmically load the compiled tables. The charset conversion tables are handled similar way. Also I think it's important to allow user defined collate data. To implement the CREATE COLLATE syntax, we need to have that capability anyway. > 4. Do we want the option of running system locale in parallel with the > internal ones? > > 5. I think we're going to have to deal with the very real possibility > that our locale database will not be as good as some of the system > provided ones. The question is how. This is quite unlike timezones > which are quite standardized and rarely change. That database is quite > well maintained. > > Would people object to a configure option that selected: > --with-locales=internal (use pg database) > --with-locales=system (use system database for win32, glibc or MacOS X) > --with-locales=none (what we support now, which is neither) > > I don't think it will be much of an issue to support this, all the > functions take the same parameters and have almost the same names. To be honest, I don't understand why we have to rely on (often broken) system locales. I don't think building our own locale data is too hard, and once we make up it, the maintenace cost will be very small since it should not be changed regularly. Moreover we could enjoy the benefit that PostgreSQL handles collations in a corret manner on any platform which PostgreSQL supports. > 6. Locales for SQL_ASCII. Seems to me you have two options, either > reject COLLATE altogether unless they specify a charset, or don't care > and let the user shoot themselves in the foot if they wish... > > BTW, this MacOS locale supports seems to be new for 10.4.2 according to > the CVS log info, can anyone confirm this? > > Anyway, I hope this post didn't bore too much. Locale support has been > one of those things that has bugged me for a long time and it would be > nice if there could be some real movement. Right. We Japanese (and probably Chinese too) have been bugged by the broken mutibyte locales for long time. Using C locale help us to a certain extent, but for Unicode we need correct locale data, othewise the sorted data will be completely chaos. -- SRA OSS, Inc. Japan Tatsuo Ishii ---------------------------(end of broadcast)--------------------------- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly From pgsql-hackers-owner+M72587@postgresql.org Sun Sep 4 11:13:07 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84FD5107053 for ; Sun, 4 Sep 2005 11:13:06 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id D1EC5B21F00; Sun, 4 Sep 2005 15:13:04 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 8823DD723A for ; Sun, 4 Sep 2005 12:01:47 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 65135-01 for ; Sun, 4 Sep 2005 15:01:43 +0000 (GMT) Received: from svana.org (svana.org [203.20.62.76]) by svr1.postgresql.org (Postfix) with ESMTP id 5896DD70F7 for ; Sun, 4 Sep 2005 12:01:40 -0300 (ADT) Received: from kleptog by svana.org with local (Exim 3.35 #1 (Debian)) id 1EBvzd-0006eP-00; Mon, 05 Sep 2005 01:01:13 +1000 Date: Sun, 4 Sep 2005 17:01:13 +0200 From: Martijn van Oosterhout To: Tatsuo Ishii cc: tgl@sss.pgh.pa.us, gsstark@mit.edu, pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Locale implementation questions Message-ID: <20050904150055.GB21198@svana.org> Reply-To: Martijn van Oosterhout References: <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <20050903203434.GA4281@svana.org> <20050904.222536.39155679.ishii@sraoss.co.jp> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZfOjI3PrQbgiZnxM" Content-Disposition: inline In-Reply-To: <20050904.222536.39155679.ishii@sraoss.co.jp> User-Agent: Mutt/1.3.28i X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6 X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522 48A7 6394 F08A 0DC6 7BE6 X-PGP-Key-URL: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.003 required=5 tests=[AWL=0.003] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR --ZfOjI3PrQbgiZnxM Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sun, Sep 04, 2005 at 10:25:36PM +0900, Tatsuo Ishii wrote: > > 3. Compiled locale files are large. One UTF-8 locale datafile can > > exceed a megabyte. Do we want the option of disabling it for small > > systems? >=20 > To avoid the problem, you could dynmically load the compiled > tables. The charset conversion tables are handled similar way. That's not the point, ofcourse they are loaded dynamically. The question is, when do we create the files in the first place. There are 48*15 =3D 750 combinations which would amount to tens of megabytes of essentially useless data. *When* you create the files is an important question. Compile time is out. Charset conversion is completely different, there just arn't that many combinations. > Also I think it's important to allow user defined collate data. To > implement the CREATE COLLATE syntax, we need to have that capability > anyway. Most OS's allow you to create collate data yourself anyway, why do we need to implement this too? > To be honest, I don't understand why we have to rely on (often broken) > system locales. I don't think building our own locale data is too > hard, and once we make up it, the maintenace cost will be very small > since it should not be changed regularly. Moreover we could enjoy the > benefit that PostgreSQL handles collations in a corret manner on any > platform which PostgreSQL supports. You say building our own locale data is not hard. I disagree, it's a waste of time we can do without. Unless you know the language yourself you cannot check changes made by anybody else. If there's an error in locale ordering, take it up with your OS distributor. I also think we open ourselves to questions like: 1. My locale is supported by the system but not by PostgreSQL, why? 2. My locale was supported last release but not this one, why? 3. Why does PostgreSQL sort differently from 'sort' or any other app on my system? > Right. We Japanese (and probably Chinese too) have been bugged by the > broken mutibyte locales for long time. Using C locale help us to a > certain extent, but for Unicode we need correct locale data, othewise > the sorted data will be completely chaos. Ok, is glibc still wrong or are they just implementing the unicode standard and that's what's wrong. All I'm saying is that we need to allow use of system locales until our native locale support is mature. In the end something like ICU (http://icu.sourceforge.net/) will end up obsoleting us. Nobody (in free-software anyway) uses it yet, but eventually it may be viable to require that to allow system independant locales. --=20 Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. --ZfOjI3PrQbgiZnxM Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQFDGwwmIB7bNG8LQkwRAmKxAJ9MccMVTLVLqW7c9Lsa8FzqEMvNGgCdGFgg nsvl/ZIAK/Qe1XpkMOItRFM= =h0V5 -----END PGP SIGNATURE----- --ZfOjI3PrQbgiZnxM-- From pgsql-hackers-owner+M72590@postgresql.org Sun Sep 4 15:15:31 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84JFU107171 for ; Sun, 4 Sep 2005 15:15:31 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 950F5B21FBE; Sun, 4 Sep 2005 19:15:26 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id AA4C5D85BF for ; Sun, 4 Sep 2005 16:02:50 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 51227-06 for ; Sun, 4 Sep 2005 19:02:48 +0000 (GMT) Received: from stark.xeocode.com (stark.xeocode.com [216.58.44.227]) by svr1.postgresql.org (Postfix) with ESMTP id CC650D8580 for ; Sun, 4 Sep 2005 16:02:46 -0300 (ADT) Received: from localhost ([127.0.0.1] helo=stark.xeocode.com) by stark.xeocode.com with smtp (Exim 3.36 #1 (Debian)) id 1EBzlN-0003LY-00; Sun, 04 Sep 2005 15:02:45 -0400 To: Tatsuo Ishii cc: kleptog@svana.org, tgl@sss.pgh.pa.us, gsstark@mit.edu, pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Locale implementation questions References: <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <20050903203434.GA4281@svana.org> <20050904.222536.39155679.ishii@sraoss.co.jp> In-Reply-To: <20050904.222536.39155679.ishii@sraoss.co.jp> From: Greg Stark Organization: The Emacs Conspiracy; member since 1992 Date: 04 Sep 2005 15:02:45 -0400 Message-ID: <87fysky7u2.fsf@stark.xeocode.com> Lines: 22 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.005 required=5 tests=[AWL=0.005] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Tatsuo Ishii writes: > To be honest, I don't understand why we have to rely on (often broken) > system locales. I don't think building our own locale data is too > hard, and once we make up it, the maintenace cost will be very small > since it should not be changed regularly. Moreover we could enjoy the > benefit that PostgreSQL handles collations in a corret manner on any > platform which PostgreSQL supports. I think it's sheer madness to try to reproduce large swaths of the OS inside Postgres because you're unhappy with the quality of the OS implementation. You should be asking yourself why OS vendors have such a hard time getting this stuff right and why would Postgres do any better. Wouldn't that work be better spent improving the database functionality of Postgres? Or at least better spent improving the locale support for the entire OS? It would be positively awful if every application on my system had its own locale database each of which had its own set of bugs and its own feature set. -- greg ---------------------------(end of broadcast)--------------------------- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq From pgsql-hackers-owner+M72597@postgresql.org Sun Sep 4 19:28:34 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j84NSY110074 for ; Sun, 4 Sep 2005 19:28:34 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 7898611ED4E0; Sun, 4 Sep 2005 23:28:28 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id BE640D8910 for ; Sun, 4 Sep 2005 20:19:37 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 09430-04 for ; Sun, 4 Sep 2005 23:19:37 +0000 (GMT) Received: from sss.pgh.pa.us (sss.pgh.pa.us [66.207.139.130]) by svr1.postgresql.org (Postfix) with ESMTP id 7DE22D8864 for ; Sun, 4 Sep 2005 20:19:34 -0300 (ADT) Received: from sss2.sss.pgh.pa.us (tgl@localhost [127.0.0.1]) by sss.pgh.pa.us (8.13.1/8.13.1) with ESMTP id j84NJclc009441; Sun, 4 Sep 2005 19:19:38 -0400 (EDT) To: Greg Stark cc: Tatsuo Ishii , kleptog@svana.org, pgsql-hackers@postgresql.org Subject: Re: [HACKERS] Locale implementation questions In-Reply-To: <87fysky7u2.fsf@stark.xeocode.com> References: <87k6hzzems.fsf@stark.xeocode.com> <14696.1125675741@sss.pgh.pa.us> <20050903203434.GA4281@svana.org> <20050904.222536.39155679.ishii@sraoss.co.jp> <87fysky7u2.fsf@stark.xeocode.com> Comments: In-reply-to Greg Stark message dated "04 Sep 2005 15:02:45 -0400" Date: Sun, 04 Sep 2005 19:19:38 -0400 Message-ID: <9440.1125875978@sss.pgh.pa.us> From: Tom Lane X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.006 required=5 tests=[AWL=0.006] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR Greg Stark writes: > I think it's sheer madness to try to reproduce large swaths of the OS > inside Postgres because you're unhappy with the quality of the OS > implementation. You should be asking yourself why OS vendors have such > a hard time getting this stuff right In the case of the *BSDs, it's pretty obviously because they don't care. > and why would Postgres do any better In the first place, we do care, and in the second place, having to deal with only one set of locale bugs would in itself be a huge advance over where we are now. We went over to maintaining our own timezone code for more or less the same reasons, and in hindsight that was obviously the right decision. Locale support is a bigger chunk, no doubt about it, but we also have a lot of motivation. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 2: Don't 'kill -9' the postmaster From pgsql-hackers-owner+M72619@postgresql.org Mon Sep 5 18:14:28 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j85MER115087 for ; Mon, 5 Sep 2005 18:14:27 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 68D9F11EB015; Mon, 5 Sep 2005 22:14:19 +0000 (GMT) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 92268D6FB8 for ; Mon, 5 Sep 2005 18:59:42 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 29325-07 for ; Mon, 5 Sep 2005 21:59:38 +0000 (GMT) Received: from svana.org (svana.org [203.20.62.76]) by svr1.postgresql.org (Postfix) with ESMTP id 98C2CD7A97 for ; Mon, 5 Sep 2005 18:59:33 -0300 (ADT) Received: from kleptog by svana.org with local (Exim 3.35 #1 (Debian)) id 1ECP02-0004SQ-00; Tue, 06 Sep 2005 07:59:34 +1000 Date: Mon, 5 Sep 2005 23:59:33 +0200 From: Martijn van Oosterhout To: pgsql-hackers@postgresql.org Subject: [HACKERS] Install Darwin's locale library on your system :) Message-ID: <20050905215928.GB5278@svana.org> Reply-To: Martijn van Oosterhout MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hHWLQfXTYDoKhP50" Content-Disposition: inline User-Agent: Mutt/1.3.28i X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6 X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522 48A7 6394 F08A 0DC6 7BE6 X-PGP-Key-URL: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.003 required=5 tests=[AWL=0.003] X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR --hHWLQfXTYDoKhP50 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Well, it was pointed out the other day that the Darwin C library supports the non-standard extensions to the POSIX locale interface and that this might be ported to other systems so PostgreSQL could use it. So, I have written a few scripts which download the libc and locale library from darwinsource, shuffle some files around and build the result into a library called libdummylocale.so. It basically completely replaces your locale support on whatever system you use it on. It's all under the APSL, though some parts may be BSD licenced. Let me say right now, the locale support here sucks, no two ways about it. It doesn't support a single UTF-8 locale. Oh, it lets you specify them, but when you ask for the CHARSET it still says US-ASCII. It does support a number of other different charsets. (Not for collation though). So my challenge to those people who think maintaining a locale library is easy: make *one* locale in FreeBSD (or Darwin or this lib) support full UTF-8 collation in whatever locale and/or charset you choose. It's all downhill from there. While it builds simple programs, I don't think it's totally safe. You'd need to rename the headers at least. And building on Darwin will probably blow up due to the way it plays fast and loose with Darwin specific #defines. But it's a beginning if anyone is interested. It builds in my glibc system. I'm going to drop the idea of making a locale library, there's just nothing good enough. glibc is the only thing that comes close. From here on I'm going to work on COLLATE for systems that support xlocale, with an eye on ICU if/when it becomes standard enough. Download: http://svana.org/kleptog/pgsql/dummylocale.tar.gz Have a nice day, --=20 Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. --hHWLQfXTYDoKhP50 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQFDHL++IB7bNG8LQkwRAjvyAJ98Hwm6Ak7zSdEU7shrzxbvMJJp/QCfcJSq ZseeCwJGWAs+3MEPEr8u2dM= =rr4R -----END PGP SIGNATURE----- --hHWLQfXTYDoKhP50-- From pgsql-patches-owner+M17366@postgresql.org Wed Sep 7 12:16:02 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j87GG1104073 for ; Wed, 7 Sep 2005 12:16:01 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id D3FD711EB015; Wed, 7 Sep 2005 16:15:58 +0000 (GMT) X-Original-To: pgsql-patches-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 326F8D7B55 for ; Wed, 7 Sep 2005 13:11:41 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 66946-04 for ; Wed, 7 Sep 2005 16:11:25 +0000 (GMT) Received: from svana.org (svana.org [203.20.62.76]) by svr1.postgresql.org (Postfix) with ESMTP id D88A6D6E3B for ; Wed, 7 Sep 2005 13:11:20 -0300 (ADT) Received: from kleptog by svana.org with local (Exim 3.35 #1 (Debian)) id 1ED2W5-00052g-00; Thu, 08 Sep 2005 02:11:17 +1000 Date: Wed, 7 Sep 2005 18:11:17 +0200 From: Martijn van Oosterhout To: pgsql-patches@postgresql.org Subject: [PATCHES] For review: Initial support for COLLATE Message-ID: <20050907161112.GA10273@svana.org> Reply-To: Martijn van Oosterhout MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ibTvN161/egqYuK8" Content-Disposition: inline User-Agent: Mutt/1.3.28i X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6 X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522 48A7 6394 F08A 0DC6 7BE6 X-PGP-Key-URL: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.003 required=5 tests=[AWL=0.003] X-Mailing-List: pgsql-patches List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-patches-owner@postgresql.org Status: OR --ibTvN161/egqYuK8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable [Please CC any replies, thanks] This patch is the beginnings of support for COLLATE. I need to do some other work for a few days so I'm posting here to get some initial reviews. Various parts are marked [done] and [not done]. The steps involved are: - Add COLLATE to grammer as part of expression tree. New CollateClause node stores the relevent facts. [done] - parse_expr goes through tree and determines the appropriate COLLATE state for each node as per SQL spec. [done] Note: PostgreSQL doesn't really have a way to identify text-like types and there's no real need to anyway. The implementation allows any node to have a collate state. This can be used for other things, see below. - CREATE COLLATE statement [not done, currently added on the fly] - Two new datatypes, pg_locale_t and pg_localedata_t. The former represents a locale (and will eventually have an OID). The latter is an anonymous cookie that stores locale specific information and is passed to functions that need it. It may be that once done, this latter type will vanish again. [done] - Several utility functions in the new file pg_xlocale.c for use by the rest of the system. [done] - Add boolean column 'proislocalized' to pg_proc which indicates if the output of this function is affected by the LOCALE. eg textcat doesn't care but textle does. [done] This is for: a) So the parser can complain about functions that look at the locale/collate order but it's not clear from the arguments (state None per SQL spec) and it's not specified explicitly. [not done] b) So when a column or function is indexed, the index code knows if the locale is relevent to sorting order. This is particularly interesting for btree indexes on character strings. [not done] Currently I've marked 58 functions as being locale sensetive, but that list will need careful going over. To some extent it can be checked automatically by examining which backend functions use the new PG_GETLOCALE() macro. - Check for correct encoding in loaded locales [not done] - Check the partial indexes do the right thing when matching expressions. [not done] - Docs, regression, etc... - make check: right now I'm getting some regressions in the rules and plpgsql, very odd... Possibly due to the fact that rules get collate nodes with locales that don't persist across invokations. Goals: - This setup extends the SQL spec a bit, in the sense that COLLATE can be attached to anything. It is my intention to allow functions such as to_char() and to_timestamp() to be localized. eg: test=3D# select cash_out('1.00'::money collate 'nl_NL'), cash_out('1.00'::m= oney collate 'en_AU'); cash_out | cash_out=20 ----------+---------- EUR1,00 | $1.00 (1 row) - Should LOCALE be created as a synonym for COLLATE? It reads more naturally. - Currently LC_COLLATE is fixed at initdb and LC_NUMERIC and LC_CURRENCY can be altered. The idea is that eventually even LC_COLLATE can be altered anytime (from the users point view anyway), as any objects that care will store the collate order they want and can execute functions as appropriate. - Eventually once the transition to full locale support is complete, change the backend so it always runs under locale C and only functions on userdata are affected by locales. This means that unquoted identifiers when converted to lowercase will be lowered as per ASCII rules. Judging by [1] this is what people want, but feedback would be nice. [1] http://www.mail-archive.com/pgsql-hackers@postgresql.org/msg39742.html Hence, right now, names compare using normal strcmp, but text, varchar, etc use strcoll. - The patch as it currently stands won't compile at all without system xlocale support. The goal is to provide some level of backward compatability where the COLLATE clause can be used, but only to affect functions like to_char(). Changing COLLATE order would be forbidden (ie just like now). - Eventually, once the parts relevent to locales are sufficiently abstracted, look into ICU to plug it in. Unfortunatly, it's a completely different model (all utf-16) so that's another phase altogether. - The default type output functions should never be locale specific. This is to avoid issues with pgdump and frontends. Create a function 'localize(anyelement)' to "do the obvious" to force it to happen. Download:=20 Compressed 57K, uncompressed >500K but that's due to rewriting the whole of pg_proc. The important code is not so big. Against todays CVS. http://svana.org/kleptog/pgsql/collate2.patch.gz Examples: test=3D# SELECT text('a') < text('B') COLLATE 'C', text('a') < text('B') CO= LLATE 'en_US.UTF-8'; ?column? | ?column?=20 ----------+---------- f | t (1 row) test=3D# SELECT text('A') < text('b') COLLATE 'C', text('A') < text('b') CO= LLATE 'en_US.UTF-8'; ?column? | ?column?=20 ----------+---------- t | t (1 row) test=3D# SELECT text('A') COLLATE 'en_US.UTF-8' < text('b') COLLATE 'C'; ERROR: Conflicting COLLATE clauses Have a nice day, --=20 Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. --ibTvN161/egqYuK8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQFDHxEgIB7bNG8LQkwRAqTqAJ0TOH3RkxnYHJmCPhjjYha0idsEoACfTjY0 EP4a7CtIaZ0Ar0dyuPja3h8= =8zan -----END PGP SIGNATURE----- --ibTvN161/egqYuK8-- From pgsql-patches-owner+M17368@postgresql.org Wed Sep 7 16:31:39 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id j87KVd101436 for ; Wed, 7 Sep 2005 16:31:39 -0400 (EDT) Received: from postgresql.org (svr1.postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 847AF11EB01E; Wed, 7 Sep 2005 20:31:32 +0000 (GMT) X-Original-To: pgsql-patches-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by svr1.postgresql.org (Postfix) with ESMTP id 22823D7D5C for ; Wed, 7 Sep 2005 16:12:20 -0300 (ADT) Received: from svr1.postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 19251-09 for ; Wed, 7 Sep 2005 19:12:13 +0000 (GMT) Received: from mail.gmx.net (mail.gmx.de [213.165.64.20]) by svr1.postgresql.org (Postfix) with SMTP id 05BA7D7B7E for ; Wed, 7 Sep 2005 16:12:13 -0300 (ADT) Received: (qmail invoked by alias); 07 Sep 2005 19:12:14 -0000 Received: from dsl-082-083-195-234.arcor-ip.net (EHLO colt.pezone.net) [82.83.195.234] by mail.gmx.net (mp022) with SMTP; 07 Sep 2005 21:12:14 +0200 X-Authenticated: #495269 From: Peter Eisentraut To: Martijn van Oosterhout Subject: Re: [PATCHES] For review: Initial support for COLLATE Date: Wed, 7 Sep 2005 21:12:12 +0200 User-Agent: KMail/1.8.1 References: <20050907161112.GA10273@svana.org> In-Reply-To: <20050907161112.GA10273@svana.org> cc: pgsql-patches@postgresql.org MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-ID: <200509072112.12955.peter_e@gmx.net> X-Y-GMX-Trusted: 0 X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, hits=0.044 required=5 tests=[AWL=-0.006, FORGED_RCVD_HELO=0.05] X-Mailing-List: pgsql-patches List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-patches-owner@postgresql.org Status: OR Martijn van Oosterhout wrote: > - Should LOCALE be created as a synonym for COLLATE? It reads more > naturally. No, and in fact the terminology mixup in your patch and description concerns me. If you are talking about collation, then the data types, system catalog columns, etc. should talk about collation, not about "locale", because that encompasses a number of other things that can be handled independent of the collation order. -- Peter Eisentraut http://developer.postgresql.org/~petere/ ---------------------------(end of broadcast)--------------------------- TIP 9: In versions below 8.0, the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match From pgsql-hackers-owner+M77964=pgman=candle.pha.pa.us@postgresql.org Tue Dec 27 06:30:14 2005 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id jBRDUET16504 for ; Tue, 27 Dec 2005 08:30:14 -0500 (EST) Received: from postgresql.org (postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 3BCBA67A57F for ; Tue, 27 Dec 2005 09:30:14 -0400 (AST) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by postgresql.org (Postfix) with ESMTP id 1DB3B9DC859 for ; Tue, 27 Dec 2005 09:29:47 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 89101-04 for ; Tue, 27 Dec 2005 09:29:50 -0400 (AST) X-Greylist: from auto-whitelisted by SQLgrey- Received: from svana.org (svana.org [203.20.62.76]) by postgresql.org (Postfix) with ESMTP id 319839DC8A5 for ; Tue, 27 Dec 2005 09:29:44 -0400 (AST) Received: from kleptog by svana.org with local (Exim 3.35 #1 (Debian)) id 1ErEte-0001lm-00; Wed, 28 Dec 2005 00:29:46 +1100 Date: Tue, 27 Dec 2005 14:29:46 +0100 From: Martijn van Oosterhout To: pgsql-hackers@postgresql.org Subject: [HACKERS] Proposed COLLATE implementation Message-ID: <20051227132941.GA32404@svana.org> Reply-To: Martijn van Oosterhout MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="W/nzBZO5zC0uMSeA" Content-Disposition: inline User-Agent: Mutt/1.3.28i X-PGP-Key-ID: Length=1024; ID=0x0DC67BE6 X-PGP-Key-Fingerprint: 295F A899 A81A 156D B522 48A7 6394 F08A 0DC6 7BE6 X-PGP-Key-URL: X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0.08 required=5 tests=[AWL=0.080] X-Spam-Score: 0.08 X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR --W/nzBZO5zC0uMSeA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Greetings all, If you're not interested in COLLATE, operator classes or related things, stop now, this is quite a long email. Firstly, status. PostgreSQL doesn't really support collations at all. The order of strings is defined at initdb time by the locale then and cannot be changed later. We allow lists to be sorted in either ascending or decending order but that's about it. Whatever order there is is deduced from b-tree operator classes. The purpose of this patch is to raise collations to (reasonably) first class object. The idea is that you can define a collation across any type and that you will then be able to ORDER BY, GROUP BY and INDEX using that collation. A collation defines both order and equality. The SQL standard does define COLLATE although they only apply that to character strings. There are no predefined collations in the standard. There are rules about how collations and collation states propegate from the leaves of the parse tree all the way to the root. In its simplest form, columns and constants have defined collations which modify the behaviour of functions using these values. At any point in the parse tree the user can override the collation with an explicit . If there is ambiguity about what collation applies at any point for a function that needs to know, this is a error. All this is parse-time analysis. Proposed Implementation: NODES To implement the above, two new node types are created: CollateClause which represents the in SQL syntax, and CollateState which represents the actual state at any node. Currently the only nodes expected to require these are OpExpr, FuncExpr, Var and Const. Although I guess it may apply to any node that can be used in an expression. CATALOG CHANGES To track collations requires a new table in the catalog, which I have named pg_collations. It contains the following fields: Oid oid; -- OID for this collation Name collname; -- Name of the collation (for collate clause) bool collasc; -- Ascending or descending Oid collopclass; -- Implementing Operator Class int4 colltype; -- Currently, 0=3Dsimple, 1=3Duses locale Oid colllocale; -- Locale in pg_locales (Should we be identifying the type here? or is it ok to lookup the type via the operator class). The first few fields name the collation so it can be referred to by the user. Then the collasc field determines how to use the operator class as given in the collopclass field. If it indicates descending order, it will invert the sense of the operator class. For example, asking for the GT op for a reverse collation will actually return the LT operator for the operator class. The purpose of the colltype and colllocale fields are described further down. The important thing at this point is that by specifying a collation you are also specifying an operator class. At the moment the ascending and descending collations for each type are hard-coded for initdb. At the moment they have been allocated OIDs starting at 2800, which is the first large available block. Each column of a table has a default collation, which defaults to the default collation of the type but can be specified in the table declaration. To store this requires an additional column in pg_attribute (attcollate) which contains the OID of the collation for that column. When it is referenced in a query, this collation is copied to the CollateState node of the Var node, from whence it can affect the query. Finally, to allow the parser to complain about ambiguous CollateStates, we need to indicate which functions actually need a senseble collate state to function. To this end a single boolean field has been added to pg_proc (proneedcollate). If this is true, the parser should error out when the collation state is COLLATE_NONE. INDEXES Another place you will be allowed to use the collate clause is while creating indexes. If you declare an index using a particular collation, it can be used in queries that order by the same collation. Note, that the collate clause indicates the operator class, so you can either specify one or the other, but not both. So each column of an index will also have a collation. However, pg_attribute has already got an extra field to store the collation for columns so it makes sense to store the collation here. In the process the pg_index.indclass field becomes redundant as it can be inferred from the pg_attribute rows associated with the index. To make this work there also needs to be a notion of compatability between collations. For example, two collations which are the reverse of eachother are compatable in the sense that an index defined with one collation would be usable for the other simply by scanning in reverse. FUNCTIONS In particular for string comparison but also possibly for user-defined types, a function will need to know what collation it is operating under. For this purpose an extra field (fn_collate) is added to FmgrInfo which is filled in with the collation from the expression tree (if any) or wherever relevent (eg. from the pg_attribute column when doing statistics or creating indexes). A PG_GETCOLLATE() macro is added to facilitate user-functions retreiving this data. This function throws an error when no collation has been defined. This shouldn't happen in practice as issues should have been weeded out at parse-time. This macro returns the OID of the collation but in many cases it will not be necessary. In particular, functions should NOT invert their result if the collation is inverted. It is considered the responsibility of the caller to invert the result if necessary. The reasons for this are: 1. In most cases that matter (order comparison) the issue can be dealt with at parse time by the NEGATOR or COMMUTATOR options. 2. For index scans, we would just do a reverse scan instead (or forward if the index is inverted) 3. Requiring every function to check the collation for inversion is wasteful, since in many cases the case can be dealt with statically. DEFAULT COLLATIONS At this point I'm inclined to define a few collations to be built in or specially handled: COLLATE ASC - default collation for type, ascending (ie, what we do now) COLLATE DESC - default collation for type, inverted COLLATE POSIX - For strings, define a simple bytewise string comparison. Indeed, it is expected that by default, all columns involving strings in the system catalog will always use COLLATE POSIX. Additionaly, type "name" will always use that collation, even if the user changes the default (by a method to be specified). This is straightforwardly done at initdb time. The purpose of COLLATE DESC is to simplify index declarations. Saying CREATE INDEX foo ON bar( a COLLATE ASC, b COLLATE DESC ); would allow it to be used in a query using ORDER BY a, b DESC, without the user having to lookup the name of the collation. When it comes to naming collations, the question arises whether ascending/descending collations need to have different names. Or should there be two collations with the same name with ASC/DESC as a modifier? Do collations have to be unique across different types; for example, can varchar and text both have a collation "ignorecase"? Another issue is that a column could be declared with a descending collation by default. Say it was an integer column, then (a < 5) would return FALSE for a =3D 1. While technically correct, I'm thinking of ruling it out on the basis of being too confusing, and only allow descending collations at query time or in index specifications. Another strange point at the moment is how to determine the default collation of a type. At the moment it is done by finding the default operator class and looking up the ascending version of that. However, we may want to provide the user a way of specifying it directly, perhaps by: ALTER TYPE text SET DEFAULT COLLATION ignorecase; PATHKEYS Currently during planning, pathkeys are indicated by an operator of the operator class. Here we would simply replace that with the oid of the collation, which can be matched directly with the collation defined by the index. USER DEFINED TYPES None of this is interesting unless it can be applied to user-defined types also. Fortunatly this is easy, when the user declares a b-tree operator class, we can generate the collations automatically. We may even allow the user to specify the name of the collation. However, if the user wanted to create multiple collations based on the same operator class (by using the PG_GETCOLLATE() macro above, we may want to provide them a way of creating them directly. COLLATIONS USING LOCALES For strings, collation can be done in many different ways defined by what is referred to as a locale. As indicated above in the definition of pg_collations, there is a colltype field. For most collations this will be 0 (simple collation). However, for strings the intention is to use a type 1 (using locales). In this case the last column refers to the OID of the locale, so you can many collations using the same operator class, but different locale oids. On a system level it changes nothing, but inside the functions implementing it, they should use PG_GETLOCALE(). This will return an opaque pg_locale_t (see below) handle which can then be used to implement the specifics. In principle, user-defined types need to be able to use this also, perhaps by using the clause COLLATE USING LOCALE in the operator class. In theory there should a collation for each combination of locale-dependant datatype, locale and order ascending/descending. How/when these are created has not yet be determined. MORE TYPES OF COLLATION Another collation type I've speculated about but not thought about implementing is a "mapping collation", in which you map the values through a function and then collate that. The obvious example would be a case-insensetive mapping where lower is applied before collation. Implementation could be pretty much done by simply substituting the functions into the parse tree. For example, if you defined something like: CREATE COLLATION ignorecase ON text USING lower($1) COLLATE defaulttext; Then anytime you did a comparison with that collation, you would simply insert those function calls into the parse-tree and then collate with "defaulttext". When declaring an index you would just make it a functional index. The rules for functional indexes should make it work out-of-the-box. OTHER TECHNICAL ISSUES - Applying a COLLATE clause to an unknown literal causes it to be coerced to the type that collation is based on. But what about if we have something like COLLATE DESC? - This requires some changes in the bootstrap procedures given that we need to be able to do lookups on the operator class for each type fairly early on. Some are predefined but it does require moving the opclass setup further up the list. However, if we store a default collation in pg_type, we wouldn't need to do that. - Sorting arrays. Should they get their own collations, or should they use the collations of their base types. LOCALES I've left this to the end because I don't want people distracted by what is essentially a side-issue. While this would be needed to implement COLLATE the way the SQL spec intended, it can actually be implemented and dealt with as a seperate patch. The main reason a basic implementation exists is that it provides a great way of finding places that didn't define a collation, since any comparison involving "text" requires one. To deal with locales I created another table in the catalog, pg_locales. This provides an OID which can be referenced from elsewhere, such as the pg_collations table. The design is intended to provide some pluggability, so locale information can come from multiple sources. Also, each locale will be referenced by an identifier which is unrelated to any external identifier, so we're not bound by them. The columns defined currently are: Name locname - Identifier used by postgresql Name locsysname - String identifying the locale for the locale provider int4 locsource - System providing this locale int4 locencoding - Encoding expected by provider It is expected that the list of sources for locale data will be short, probably hard-coded into the backend (currenty internal/system/icu). The only locale defined at startup is POSIX, which is implemented internally. The intention is for any other locales to be defined at the end of initdb. The expected syntax is something like: CREATE LOCALE hungarian AS 'hu_HU' USING glibc; This should use the provider to check the locale exists and has a conpatible encoding. If so it is entered into the table ready for use. In the backend, there will be implementations of functions like pg_strcoll_l, pg_localeconv_l, which work like the C system library versions only they take an extra pg_locale_t argument. This is used to dispatch the call to the right place. There will be a function to quickly determine if a locale is C to shortcircuit complexity where it is not needed. STATUS Implementation so far is available here: http://svana.org/kleptog/temp/collate-current.patch.gz This patch isn't "clean" and changes a few things that are not strictly necessary. It won't finish initdb right now because it gets an error in ANALYSE (the array issue above). Feedback, help, comments: please reply. Have a nice day, --=20 Martijn van Oosterhout http://svana.org/kleptog/ > Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a > tool for doing 5% of the work and then sitting around waiting for someone > else to do the other 95% so you can sue them. --W/nzBZO5zC0uMSeA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.6 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQFDsUHDIB7bNG8LQkwRAnh0AJ0YUNLkVaSY3u0jWBPdlaq+9dujZACfTmis JR1mF60lKx14Ih850p3lpVk= =/Ghs -----END PGP SIGNATURE----- --W/nzBZO5zC0uMSeA-- From pgsql-hackers-owner+M78869@postgresql.org Tue Jan 24 10:59:03 2006 Return-path: Received: from ams.hub.org (ams.hub.org [200.46.204.13]) by candle.pha.pa.us (8.11.6/8.11.6) with ESMTP id k0OGx0324883 for ; Tue, 24 Jan 2006 11:59:00 -0500 (EST) Received: from postgresql.org (postgresql.org [200.46.204.71]) by ams.hub.org (Postfix) with ESMTP id 6E4BE67B09D; Tue, 24 Jan 2006 12:58:56 -0400 (AST) X-Original-To: pgsql-hackers-postgresql.org@localhost.postgresql.org Received: from localhost (av.hub.org [200.46.204.144]) by postgresql.org (Postfix) with ESMTP id 1C9E59DC9BE for ; Tue, 24 Jan 2006 12:58:31 -0400 (AST) Received: from postgresql.org ([200.46.204.71]) by localhost (av.hub.org [200.46.204.144]) (amavisd-new, port 10024) with ESMTP id 43415-09 for ; Tue, 24 Jan 2006 12:58:29 -0400 (AST) Received: from svr4.postgresql.org (svr4.postgresql.org [66.98.251.159]) by postgresql.org (Postfix) with ESMTP id F05519DC86E for ; Tue, 24 Jan 2006 12:58:27 -0400 (AST) Received: from alife.ru (alife.ru [82.146.44.110]) by svr4.postgresql.org (Postfix) with ESMTP id 42D155AF193 for ; Tue, 24 Jan 2006 16:58:27 +0000 (GMT) Received: from [212.192.243.99] (elizabet.sai.msu.ru [212.192.243.99]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by alife.ru (Postfix) with ESMTP id A54D554219D for ; Tue, 24 Jan 2006 19:58:20 +0300 (MSK) Message-ID: <43D65CA3.4000402@tronet.ru> Date: Tue, 24 Jan 2006 19:58:11 +0300 From: Alexey Slynko User-Agent: Mozilla Thunderbird 1.0.7 (X11/20051005) X-Accept-Language: ru-ru, ru MIME-Version: 1.0 To: pgsql-hackers@postgresql.org Subject: [HACKERS] TODO item: locale per database patch (new iteration) Content-Type: multipart/mixed; boundary="------------060707070409000500060805" X-Virus-Scanned: by amavisd-new at hub.org X-Spam-Status: No, score=0 required=5 tests=[none] X-Spam-Score: 0 X-Mailing-List: pgsql-hackers List-Archive: List-Help: List-Id: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Precedence: bulk Sender: pgsql-hackers-owner@postgresql.org Status: OR This is a multi-part message in MIME format. --------------060707070409000500060805 Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Hi, it's a renewed locale per database patch. Unfortunately, i've not found clean way to rebuild database indexes automatically, if locale settings of two databases (created and template) are differs. Now it's only raises a NOTICE. So, if anyone has a right notion about it - let will express. Comment and suggestions are highly appreciated --------------060707070409000500060805 Content-Type: text/plain; name="locale_per_database.patch" Content-Transfer-Encoding: 7bit Content-Disposition: inline; filename="locale_per_database.patch" Index: src/backend/access/transam/xlog.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/backend/access/transam/xlog.c,v retrieving revision 1.226 diff -u -r1.226 xlog.c --- src/backend/access/transam/xlog.c 11 Jan 2006 08:43:12 -0000 1.226 +++ src/backend/access/transam/xlog.c 22 Jan 2006 16:41:02 -0000 @@ -3394,7 +3394,6 @@ { int fd; char buffer[BLCKSZ]; /* need not be aligned */ - char *localeptr; /* * Initialize version and compatibility-check fields @@ -3418,18 +3417,6 @@ ControlFile->enableIntTimes = FALSE; #endif - ControlFile->localeBuflen = LOCALE_NAME_BUFLEN; - localeptr = setlocale(LC_COLLATE, NULL); - if (!localeptr) - ereport(PANIC, - (errmsg("invalid LC_COLLATE setting"))); - StrNCpy(ControlFile->lc_collate, localeptr, LOCALE_NAME_BUFLEN); - localeptr = setlocale(LC_CTYPE, NULL); - if (!localeptr) - ereport(PANIC, - (errmsg("invalid LC_CTYPE setting"))); - StrNCpy(ControlFile->lc_ctype, localeptr, LOCALE_NAME_BUFLEN); - /* Contents are protected with a CRC */ INIT_CRC32(ControlFile->crc); COMP_CRC32(ControlFile->crc, @@ -3612,34 +3599,6 @@ " but the server was compiled without HAVE_INT64_TIMESTAMP."), errhint("It looks like you need to recompile or initdb."))); #endif - - if (ControlFile->localeBuflen != LOCALE_NAME_BUFLEN) - ereport(FATAL, - (errmsg("database files are incompatible with server"), - errdetail("The database cluster was initialized with LOCALE_NAME_BUFLEN %d," - " but the server was compiled with LOCALE_NAME_BUFLEN %d.", - ControlFile->localeBuflen, LOCALE_NAME_BUFLEN), - errhint("It looks like you need to recompile or initdb."))); - if (pg_perm_setlocale(LC_COLLATE, ControlFile->lc_collate) == NULL) - ereport(FATAL, - (errmsg("database files are incompatible with operating system"), - errdetail("The database cluster was initialized with LC_COLLATE \"%s\"," - " which is not recognized by setlocale().", - ControlFile->lc_collate), - errhint("It looks like you need to initdb or install locale support."))); - if (pg_perm_setlocale(LC_CTYPE, ControlFile->lc_ctype) == NULL) - ereport(FATAL, - (errmsg("database files are incompatible with operating system"), - errdetail("The database cluster was initialized with LC_CTYPE \"%s\"," - " which is not recognized by setlocale().", - ControlFile->lc_ctype), - errhint("It looks like you need to initdb or install locale support."))); - - /* Make the fixed locale settings visible as GUC variables, too */ - SetConfigOption("lc_collate", ControlFile->lc_collate, - PGC_INTERNAL, PGC_S_OVERRIDE); - SetConfigOption("lc_ctype", ControlFile->lc_ctype, - PGC_INTERNAL, PGC_S_OVERRIDE); } void Index: src/backend/commands/dbcommands.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/backend/commands/dbcommands.c,v retrieving revision 1.175 diff -u -r1.175 dbcommands.c --- src/backend/commands/dbcommands.c 22 Nov 2005 18:17:08 -0000 1.175 +++ src/backend/commands/dbcommands.c 22 Jan 2006 16:41:03 -0000 @@ -25,6 +25,10 @@ #include #include +#ifdef HAVE_LANGINFO_H +#include +#endif + #include "access/genam.h" #include "access/heapam.h" #include "catalog/catalog.h" @@ -49,6 +53,7 @@ #include "utils/fmgroids.h" #include "utils/guc.h" #include "utils/lsyscache.h" +#include "utils/pg_locale.h" #include "utils/syscache.h" @@ -57,9 +62,11 @@ int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, Oid *dbLastSysOidP, TransactionId *dbVacuumXidP, TransactionId *dbFrozenXidP, - Oid *dbTablespace); + Oid *dbTablespace, char **dbCollate, char **dbCtype); static bool have_createdb_privilege(void); static void remove_dbtablespaces(Oid db_id); +static char * get_locale_encoding(const char *ctype); +static int check_locale_encoding(int encid, const char *ctype); /* @@ -73,6 +80,8 @@ Oid src_dboid; Oid src_owner; int src_encoding; + char *src_collate; + char *src_ctype; bool src_istemplate; bool src_allowconn; Oid src_lastsysoid; @@ -92,10 +101,14 @@ DefElem *downer = NULL; DefElem *dtemplate = NULL; DefElem *dencoding = NULL; + DefElem *dlc_collate = NULL; + DefElem *dlc_ctype = NULL; DefElem *dconnlimit = NULL; char *dbname = stmt->dbname; char *dbowner = NULL; const char *dbtemplate = NULL; + char *lc_collate = NULL; + char *lc_ctype = NULL; volatile int encoding = -1; volatile int dbconnlimit = -1; @@ -139,6 +152,22 @@ errmsg("conflicting or redundant options"))); dencoding = defel; } + else if (strcmp(defel->defname, "lccollate") == 0) + { + if (dlc_collate) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("conflicting or redundant options"))); + dlc_collate = defel; + } + else if (strcmp(defel->defname, "lcctype") == 0) + { + if (dlc_ctype) + ereport(ERROR, + (errcode(ERRCODE_SYNTAX_ERROR), + errmsg("conflicting or redundant options"))); + dlc_ctype = defel; + } else if (strcmp(defel->defname, "connectionlimit") == 0) { if (dconnlimit) @@ -192,6 +221,22 @@ elog(ERROR, "unrecognized node type: %d", nodeTag(dencoding->arg)); } + if (dlc_collate && dlc_collate->arg) { + lc_collate = strVal(dlc_collate->arg); + if ((locale_collate_assign(lc_collate, false, (GucSource)NULL)) == NULL) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("%s is not a valid LC_COLLATE name", + lc_collate))); + } + if (dlc_ctype && dlc_ctype->arg) { + lc_ctype = strVal(dlc_ctype->arg); + if ((locale_collate_assign(lc_ctype, false, (GucSource)NULL)) == NULL) + ereport(ERROR, + (errcode(ERRCODE_UNDEFINED_OBJECT), + errmsg("%s is not a valid LC_CTYPE name", + lc_ctype))); + } if (dconnlimit && dconnlimit->arg) dbconnlimit = intVal(dconnlimit->arg); @@ -224,7 +269,7 @@ * grab the exclusive lock. */ if (get_db_info(dbname, NULL, NULL, NULL, - NULL, NULL, NULL, NULL, NULL, NULL)) + NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL)) ereport(ERROR, (errcode(ERRCODE_DUPLICATE_DATABASE), errmsg("database \"%s\" already exists", dbname))); @@ -237,7 +282,8 @@ if (!get_db_info(dbtemplate, &src_dboid, &src_owner, &src_encoding, &src_istemplate, &src_allowconn, &src_lastsysoid, - &src_vacuumxid, &src_frozenxid, &src_deftablespace)) + &src_vacuumxid, &src_frozenxid, &src_deftablespace, + &src_collate, &src_ctype)) ereport(ERROR, (errcode(ERRCODE_UNDEFINED_DATABASE), errmsg("template database \"%s\" does not exist", dbtemplate))); @@ -277,6 +323,21 @@ (errcode(ERRCODE_WRONG_OBJECT_TYPE), errmsg("invalid server encoding %d", encoding))); + /* Set database lc_collate and lc_ctype */ + if (!lc_collate) + lc_collate = src_collate; + if (!lc_ctype) + lc_ctype = src_ctype; + +#if defined(HAVE_LANGINFO_H) && defined(CODESET) + if (encoding > 0 && check_locale_encoding(encoding, lc_ctype) == -1) + ereport(ERROR, + (errcode(ERRCODE_WRONG_OBJECT_TYPE), + errmsg("encoding %s is not suitable for locale %s", + pg_encoding_to_char(encoding), + lc_ctype))); +#endif + /* Resolve default tablespace for new database */ if (dtablespacename && dtablespacename->arg) { @@ -441,7 +502,7 @@ /* Check to see if someone else created same DB name meanwhile. */ if (get_db_info(dbname, NULL, NULL, NULL, - NULL, NULL, NULL, NULL, NULL, NULL)) + NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL)) ereport(ERROR, (errcode(ERRCODE_DUPLICATE_DATABASE), errmsg("database \"%s\" already exists", dbname))); @@ -459,6 +520,11 @@ DirectFunctionCall1(namein, CStringGetDatum(dbname)); new_record[Anum_pg_database_datdba - 1] = ObjectIdGetDatum(datdba); new_record[Anum_pg_database_encoding - 1] = Int32GetDatum(encoding); + new_record[Anum_pg_database_datcollate - 1] = + DirectFunctionCall1(namein, CStringGetDatum(lc_collate)); + new_record[Anum_pg_database_datctype - 1] = + DirectFunctionCall1(namein, CStringGetDatum(lc_ctype)); + new_record[Anum_pg_database_datistemplate - 1] = BoolGetDatum(false); new_record[Anum_pg_database_datallowconn - 1] = BoolGetDatum(true); new_record[Anum_pg_database_datconnlimit - 1] = Int32GetDatum(dbconnlimit); @@ -527,6 +593,15 @@ * Set flag to update flat database file at commit. */ database_file_update_needed(); + + /* + * Message about reindexing new database + */ + if (lc_collate != src_collate || lc_ctype != src_ctype) + ereport(NOTICE, + (errmsg("database \"%s\" need to be reindexed manually (REINDEX DATABASE)", + dbname))); + } PG_CATCH(); { @@ -584,7 +659,7 @@ pgdbrel = heap_open(DatabaseRelationId, ExclusiveLock); if (!get_db_info(dbname, &db_id, NULL, NULL, - &db_istemplate, NULL, NULL, NULL, NULL, NULL)) + &db_istemplate, NULL, NULL, NULL, NULL, NULL, NULL, NULL)) { if (!missing_ok) { @@ -1100,7 +1175,7 @@ int *encodingP, bool *dbIsTemplateP, bool *dbAllowConnP, Oid *dbLastSysOidP, TransactionId *dbVacuumXidP, TransactionId *dbFrozenXidP, - Oid *dbTablespace) + Oid *dbTablespace, char **dbCollate, char **dbCtype) { Relation relation; ScanKeyData scanKey; @@ -1155,6 +1230,11 @@ /* default tablespace for this database */ if (dbTablespace) *dbTablespace = dbform->dattablespace; + /* default locale settings for this database */ + if (dbCollate) + *dbCollate = NameStr(dbform->datcollate); + if (dbCtype) + *dbCtype = NameStr(dbform->datctype); } systable_endscan(scan); @@ -1416,3 +1496,45 @@ else strcat(buf, "UNKNOWN"); } + +#if defined(HAVE_LANGINFO_H) && defined(CODESET) + +static char * +get_locale_encoding(const char *ctype) +{ + char *save; + char *sys; + + save = setlocale(LC_CTYPE, NULL); + if (!save) + return NULL; + save = pstrdup(save); + + setlocale(LC_CTYPE, ctype); + sys = nl_langinfo(CODESET); + sys = pstrdup(sys); + + setlocale(LC_CTYPE, save); + pfree(save); + + return sys; +} + +static int +check_locale_encoding(int encid, const char *ctype) +{ + char *sys; + + sys = get_locale_encoding(ctype); + if (encid == pg_char_to_encoding(sys)) + { + pfree(sys); + return 0; + } + + pfree(sys); + return -1; +} + +#endif + Index: src/backend/parser/gram.y =================================================================== RCS file: /projects/cvsroot/pgsql/src/backend/parser/gram.y,v retrieving revision 2.522 diff -u -r2.522 gram.y --- src/backend/parser/gram.y 21 Jan 2006 02:16:19 -0000 2.522 +++ src/backend/parser/gram.y 22 Jan 2006 16:41:09 -0000 @@ -372,7 +372,7 @@ KEY - LANCOMPILER LANGUAGE LARGE_P LAST_P LEADING LEAST LEFT LEVEL + LANCOMPILER LANGUAGE LARGE_P LAST_P LCCOLLATE LCCTYPE LEADING LEAST LEFT LEVEL LIKE LIMIT LISTEN LOAD LOCAL LOCALTIME LOCALTIMESTAMP LOCATION LOCK_P LOGIN_P @@ -4635,6 +4635,22 @@ { $$ = makeDefElem("encoding", NULL); } + | LCCOLLATE opt_equal name + { + $$ = makeDefElem("lccollate", (Node *)makeString($3)); + } + | LCCOLLATE opt_equal DEFAULT + { + $$ = makeDefElem("lccollate", NULL); + } + | LCCTYPE opt_equal name + { + $$ = makeDefElem("lcctype", (Node *)makeString($3)); + } + | LCCTYPE opt_equal DEFAULT + { + $$ = makeDefElem("lcctype", NULL); + } | CONNECTION LIMIT opt_equal SignedIconst { $$ = makeDefElem("connectionlimit", (Node *)makeInteger($4)); @@ -8225,6 +8241,8 @@ | LANGUAGE | LARGE_P | LAST_P + | LCCOLLATE + | LCCTYPE | LEVEL | LISTEN | LOAD Index: src/backend/parser/keywords.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/backend/parser/keywords.c,v retrieving revision 1.170 diff -u -r1.170 keywords.c --- src/backend/parser/keywords.c 27 Dec 2005 04:00:07 -0000 1.170 +++ src/backend/parser/keywords.c 22 Jan 2006 16:41:09 -0000 @@ -193,6 +193,8 @@ {"language", LANGUAGE}, {"large", LARGE_P}, {"last", LAST_P}, + {"lccollate", LCCOLLATE}, + {"lcctype", LCCTYPE}, {"leading", LEADING}, {"least", LEAST}, {"left", LEFT}, Index: src/backend/utils/adt/pg_locale.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/backend/utils/adt/pg_locale.c,v retrieving revision 1.34 diff -u -r1.34 pg_locale.c --- src/backend/utils/adt/pg_locale.c 2 Jan 2006 20:25:45 -0000 1.34 +++ src/backend/utils/adt/pg_locale.c 22 Jan 2006 16:41:10 -0000 @@ -10,10 +10,8 @@ */ /*---------- - * Here is how the locale stuff is handled: LC_COLLATE and LC_CTYPE - * are fixed by initdb, stored in pg_control, and cannot be changed. - * Thus, the effects of strcoll(), strxfrm(), isupper(), toupper(), - * etc. are always in the same fixed locale. + * Here is how the locale stuff is handled: + * LC_COLLATE and LC_CTYPE are defined by createdb and stored in pg_database. * * LC_MESSAGES is settable at run time and will take effect * immediately. @@ -208,6 +206,17 @@ return value; } +const char * +locale_collate_assign(const char *value, bool doit, GucSource source) +{ + return locale_xxx_assign(LC_COLLATE, value, doit, source); +} + +const char * +locale_ctype_assign(const char *value, bool doit, GucSource source) +{ + return locale_xxx_assign(LC_CTYPE, value, doit, source); +} const char * locale_monetary_assign(const char *value, bool doit, GucSource source) Index: src/backend/utils/init/postinit.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/backend/utils/init/postinit.c,v retrieving revision 1.160 diff -u -r1.160 postinit.c --- src/backend/utils/init/postinit.c 4 Jan 2006 21:06:32 -0000 1.160 +++ src/backend/utils/init/postinit.c 22 Jan 2006 16:41:10 -0000 @@ -138,6 +138,8 @@ ScanKeyData key; HeapTuple tup; Form_pg_database dbform; + char *lc_ctype; + char *lc_collate; /* * Because we grab RowShareLock here, we can be sure that dropdb() is not @@ -225,6 +227,32 @@ SetConfigOption("client_encoding", GetDatabaseEncodingName(), PGC_BACKEND, PGC_S_DEFAULT); + /* Set up database locale */ + lc_collate = NameStr(dbform->datcollate); + lc_ctype = NameStr(dbform->datctype); + + if (setlocale(LC_COLLATE, lc_collate) == NULL) + ereport(FATAL, + (errmsg("database locale is incompatible with operating system"), + errdetail("The database was initialized with LC_COLLATE \"%s\"," + " which is not recognized by setlocale().", + lc_collate), + errhint("It looks like you need to recreate database or install locale support."))); + if (setlocale(LC_CTYPE, lc_ctype) == NULL) + ereport(FATAL, + (errmsg("database locale are incompatible with operating system"), + errdetail("The database was initialized with LC_CTYPE \"%s\"," + " which is not recognized by setlocale().", + lc_ctype), + errhint("It looks like you need to recreate database or install locale support."))); + + /* Record it as a GUC internal option, too */ + SetConfigOption("lc_collate", lc_collate, + PGC_INTERNAL, PGC_S_DATABASE); + SetConfigOption("lc_ctype", lc_ctype, + PGC_INTERNAL, PGC_S_DATABASE); + + /* * Lastly, set up any database-specific configuration variables. */ Index: src/bin/initdb/initdb.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/bin/initdb/initdb.c,v retrieving revision 1.106 diff -u -r1.106 initdb.c --- src/bin/initdb/initdb.c 5 Jan 2006 10:07:46 -0000 1.106 +++ src/bin/initdb/initdb.c 22 Jan 2006 16:41:13 -0000 @@ -1377,6 +1377,10 @@ bki_lines = replace_token(bki_lines, "ENCODING", encodingid); + bki_lines = replace_token(bki_lines, "LC_COLLATE", lc_collate); + + bki_lines = replace_token(bki_lines, "LC_CTYPE", lc_ctype); + /* * Pass correct LC_xxx environment to bootstrap. * @@ -2617,7 +2621,7 @@ printf(_("The database cluster will be initialized with locale %s.\n"), lc_ctype); else { - printf(_("The database cluster will be initialized with locales\n" + printf(_("The database template1 will be initialized with locales\n" " COLLATE: %s\n" " CTYPE: %s\n" " MESSAGES: %s\n" Index: src/bin/pg_controldata/pg_controldata.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/bin/pg_controldata/pg_controldata.c,v retrieving revision 1.27 diff -u -r1.27 pg_controldata.c --- src/bin/pg_controldata/pg_controldata.c 15 Oct 2005 02:49:37 -0000 1.27 +++ src/bin/pg_controldata/pg_controldata.c 22 Jan 2006 16:41:13 -0000 @@ -177,9 +177,5 @@ printf(_("Maximum columns in an index: %u\n"), ControlFile.indexMaxKeys); printf(_("Date/time type storage: %s\n"), (ControlFile.enableIntTimes ? _("64-bit integers") : _("floating-point numbers"))); - printf(_("Maximum length of locale name: %u\n"), ControlFile.localeBuflen); - printf(_("LC_COLLATE: %s\n"), ControlFile.lc_collate); - printf(_("LC_CTYPE: %s\n"), ControlFile.lc_ctype); - return 0; } Index: src/bin/pg_resetxlog/pg_resetxlog.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/bin/pg_resetxlog/pg_resetxlog.c,v retrieving revision 1.39 diff -u -r1.39 pg_resetxlog.c --- src/bin/pg_resetxlog/pg_resetxlog.c 5 Jan 2006 03:01:37 -0000 1.39 +++ src/bin/pg_resetxlog/pg_resetxlog.c 22 Jan 2006 16:41:13 -0000 @@ -465,22 +465,6 @@ #else ControlFile.enableIntTimes = FALSE; #endif - ControlFile.localeBuflen = LOCALE_NAME_BUFLEN; - - localeptr = setlocale(LC_COLLATE, ""); - if (!localeptr) - { - fprintf(stderr, _("%s: invalid LC_COLLATE setting\n"), progname); - exit(1); - } - StrNCpy(ControlFile.lc_collate, localeptr, LOCALE_NAME_BUFLEN); - localeptr = setlocale(LC_CTYPE, ""); - if (!localeptr) - { - fprintf(stderr, _("%s: invalid LC_CTYPE setting\n"), progname); - exit(1); - } - StrNCpy(ControlFile.lc_ctype, localeptr, LOCALE_NAME_BUFLEN); /* * XXX eventually, should try to grovel through old XLOG to develop more @@ -530,9 +514,6 @@ printf(_("Maximum columns in an index: %u\n"), ControlFile.indexMaxKeys); printf(_("Date/time type storage: %s\n"), (ControlFile.enableIntTimes ? _("64-bit integers") : _("floating-point numbers"))); - printf(_("Maximum length of locale name: %u\n"), ControlFile.localeBuflen); - printf(_("LC_COLLATE: %s\n"), ControlFile.lc_collate); - printf(_("LC_CTYPE: %s\n"), ControlFile.lc_ctype); } Index: src/bin/psql/describe.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/bin/psql/describe.c,v retrieving revision 1.130 diff -u -r1.130 describe.c --- src/bin/psql/describe.c 22 Nov 2005 18:17:29 -0000 1.130 +++ src/bin/psql/describe.c 22 Jan 2006 16:41:15 -0000 @@ -360,6 +360,12 @@ appendPQExpBuffer(&buf, ",\n pg_catalog.pg_encoding_to_char(d.encoding) as \"%s\"", _("Encoding")); + appendPQExpBuffer(&buf, + ",\n d.datcollate as \"%s\"", + _("LC_COLLATE")); + appendPQExpBuffer(&buf, + ",\n d.datctype as \"%s\"", + _("LC_CTYPE")); if (verbose) appendPQExpBuffer(&buf, ",\n pg_catalog.obj_description(d.oid, 'pg_database') as \"%s\"", Index: src/bin/scripts/createdb.c =================================================================== RCS file: /projects/cvsroot/pgsql/src/bin/scripts/createdb.c,v retrieving revision 1.15 diff -u -r1.15 createdb.c --- src/bin/scripts/createdb.c 21 Jun 2005 04:02:33 -0000 1.15 +++ src/bin/scripts/createdb.c 22 Jan 2006 16:41:15 -0000 @@ -34,6 +34,8 @@ {"tablespace", required_argument, NULL, 'D'}, {"template", required_argument, NULL, 'T'}, {"encoding", required_argument, NULL, 'E'}, + {"lc-collate", required_argument, NULL, 1}, + {"lc-ctype", required_argument, NULL, 2}, {NULL, 0, NULL, 0} }; @@ -53,6 +55,8 @@ char *tablespace = NULL; char *template = NULL; char *encoding = NULL; + char *lc_collate = NULL; + char *lc_ctype = NULL; PQExpBufferData sql; @@ -98,6 +102,12 @@ case 'E': encoding = optarg; break; + case 1: + lc_collate = optarg; + break; + case 2: + lc_ctype = optarg; + break; default: fprintf(stderr, _("Try \"%s --help\" for more information.\n"), progname); exit(1); @@ -155,7 +165,12 @@ appendPQExpBuffer(&sql, " ENCODING '%s'", encoding); if (template) appendPQExpBuffer(&sql, " TEMPLATE %s", fmtId(template)); + if (lc_collate) + appendPQExpBuffer(&sql, " LCCOLLATE %s", fmtId(lc_collate)); + if (lc_ctype) + appendPQExpBuffer(&sql, " LCCTYPE %s", fmtId(lc_ctype)); appendPQExpBuffer(&sql, ";\n"); + conn = connectDatabase(strcmp(dbname, "postgres") == 0 ? "template1" : "postgres", host, port, username, password, progname); @@ -219,19 +234,20 @@ printf(_("Usage:\n")); printf(_(" %s [OPTION]... [DBNAME] [DESCRIPTION]\n"), progname); printf(_("\nOptions:\n")); - printf(_(" -D, --tablespace=TABLESPACE default tablespace for the database\n")); - printf(_(" -E, --encoding=ENCODING encoding for the database\n")); - printf(_(" -O, --owner=OWNER database user to own the new database\n")); - printf(_(" -T, --template=TEMPLATE template database to copy\n")); - printf(_(" -e, --echo show the commands being sent to the server\n")); - printf(_(" -q, --quiet don't write any messages\n")); - printf(_(" --help show this help, then exit\n")); - printf(_(" --version output version information, then exit\n")); + printf(_(" -D, --tablespace=TABLESPACE default tablespace for the database\n")); + printf(_(" -E, --encoding=ENCODING encoding for the database\n")); + printf(_(" --lc-collate, --lc-ctype=LOCALE initialize database with given locale\n")); + printf(_(" -O, --owner=OWNER database user to own the new database\n")); + printf(_(" -T, --template=TEMPLATE template database to copy\n")); + printf(_(" -e, --echo show the commands being sent to the server\n")); + printf(_(" -q, --quiet don't write any messages\n")); + printf(_(" --help show this help, then exit\n")); + printf(_(" --version output version information, then exit\n")); printf(_("\nConnection options:\n")); - printf(_(" -h, --host=HOSTNAME database server host or socket directory\n")); - printf(_(" -p, --port=PORT database server port\n")); - printf(_(" -U, --username=USERNAME user name to connect as\n")); - printf(_(" -W, --password prompt for password\n")); + printf(_(" -h, --host=HOSTNAME database server host or socket directory\n")); + printf(_(" -p, --port=PORT database server port\n")); + printf(_(" -U, --username=USERNAME user name to connect as\n")); + printf(_(" -W, --password prompt for password\n")); printf(_("\nBy default, a database with the same name as the current user is created.\n")); printf(_("\nReport bugs to .\n")); } Index: src/include/catalog/pg_control.h =================================================================== RCS file: /projects/cvsroot/pgsql/src/include/catalog/pg_control.h,v retrieving revision 1.26 diff -u -r1.26 pg_control.h --- src/include/catalog/pg_control.h 22 Nov 2005 18:17:30 -0000 1.26 +++ src/include/catalog/pg_control.h 22 Jan 2006 16:41:15 -0000 @@ -137,11 +137,6 @@ /* flag indicating internal format of timestamp, interval, time */ uint32 enableIntTimes; /* int64 storage enabled? */ - /* active locales */ - uint32 localeBuflen; - char lc_collate[LOCALE_NAME_BUFLEN]; - char lc_ctype[LOCALE_NAME_BUFLEN]; - /* CRC of all above ... MUST BE LAST! */ pg_crc32 crc; } ControlFileData; Index: src/include/catalog/pg_database.h =================================================================== RCS file: /projects/cvsroot/pgsql/src/include/catalog/pg_database.h,v retrieving revision 1.38 diff -u -r1.38 pg_database.h --- src/include/catalog/pg_database.h 15 Oct 2005 02:49:42 -0000 1.38 +++ src/include/catalog/pg_database.h 22 Jan 2006 16:41:16 -0000 @@ -38,6 +38,8 @@ NameData datname; /* database name */ Oid datdba; /* owner of database */ int4 encoding; /* character encoding */ + NameData datcollate; /* locale LC_COLLATE */ + NameData datctype; /* locale LC_CTYPE */ bool datistemplate; /* allowed as CREATE DATABASE template? */ bool datallowconn; /* new connections allowed? */ int4 datconnlimit; /* max connections allowed (-1=no limit) */ @@ -60,21 +62,23 @@ * compiler constants for pg_database * ---------------- */ -#define Natts_pg_database 12 +#define Natts_pg_database 14 #define Anum_pg_database_datname 1 #define Anum_pg_database_datdba 2 #define Anum_pg_database_encoding 3 -#define Anum_pg_database_datistemplate 4 -#define Anum_pg_database_datallowconn 5 -#define Anum_pg_database_datconnlimit 6 -#define Anum_pg_database_datlastsysoid 7 -#define Anum_pg_database_datvacuumxid 8 -#define Anum_pg_database_datfrozenxid 9 -#define Anum_pg_database_dattablespace 10 -#define Anum_pg_database_datconfig 11 -#define Anum_pg_database_datacl 12 +#define Anum_pg_database_datcollate 4 +#define Anum_pg_database_datctype 5 +#define Anum_pg_database_datistemplate 6 +#define Anum_pg_database_datallowconn 7 +#define Anum_pg_database_datconnlimit 8 +#define Anum_pg_database_datlastsysoid 9 +#define Anum_pg_database_datvacuumxid 10 +#define Anum_pg_database_datfrozenxid 11 +#define Anum_pg_database_dattablespace 12 +#define Anum_pg_database_datconfig 13 +#define Anum_pg_database_datacl 14 -DATA(insert OID = 1 ( template1 PGUID ENCODING t t -1 0 0 0 1663 _null_ _null_ )); +DATA(insert OID = 1 ( template1 PGUID ENCODING "LC_CTYPE" "LC_COLLATE" t t -1 0 0 0 1663 _null_ _null_ )); DESCR("Default template database"); #define TemplateDbOid 1 Index: src/include/utils/pg_locale.h =================================================================== RCS file: /projects/cvsroot/pgsql/src/include/utils/pg_locale.h,v retrieving revision 1.21 diff -u -r1.21 pg_locale.h --- src/include/utils/pg_locale.h 28 Dec 2005 23:22:51 -0000 1.21 +++ src/include/utils/pg_locale.h 22 Jan 2006 16:41:16 -0000 @@ -22,6 +22,10 @@ extern char *locale_numeric; extern char *locale_time; +extern const char *locale_collate_assign(const char *value, + bool doit, GucSource source); +extern const char *locale_ctype_assign(const char *value, + bool doit, GucSource source); extern const char *locale_messages_assign(const char *value, bool doit, GucSource source); extern const char *locale_monetary_assign(const char *value, --------------060707070409000500060805 Content-Type: text/plain Content-Disposition: inline Content-Transfer-Encoding: 8bit MIME-Version: 1.0 ---------------------------(end of broadcast)--------------------------- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to majordomo@postgresql.org so that your message can get through to the mailing list cleanly --------------060707070409000500060805--