1) Presentation

When making a SunOne DS to RedHat-DS migration with different charset: Redhat DS is using UTF-8 charset. It means ldif import files used by RH-DS have to be  UTF-8. This is even more true, when it comes to binary data, which ought to be encoded using using UTF-8.

2) How is it possible to get the value of the charset

The command locale gives you teh charset used on your platform. It is recorded in the LANG environment variable

Discrepancy is coming from the following

1) SunOne charset is iso_8859 (1byte)
2) RH-DS does only accept UTF-8 (2 bytes)

3) Binary values are encoded on SunOne using iso_8859 charset.
When reading a ldif import coming from sunone, RH-DS import is blowing up indicating a violation with a message such as « violates attribute syntax »

3) How to fix it

The fix is quite tricky as for binary value it consists of providing the following operations:
-a) read the encoded binary value (using iso8859 charset)
-b) decode the binary value, which has been read
-c) reencode the value in binary using UTF8
At upper level, it makes binary attributes values and binary acis values which were iso_8859 charset encoded to be replaced by their corresponding UTF8 in the ldif file.
d) With such a transformation ldif import will now behaves fine using rh-ds

You can write such a transformation using java for example.

4) Using a parser

We internally developed a parser to cope with these charset burden, if you face such problem and are interested  for assitance to deploy this parser don’t hesitate to contact us.

Olivier Rivat

Senior Software Engineer with over 25 years of experience doing Software Development, Support and Consulting in Identity and Access Management Solutions.
Specialised in IAM (security, access control, identity management) and Open Source integration, settled in 2004 by IAM industry veteran, JANUA offers high value-added products and services to businesses and governements with a concern for Identity Management and Open Source components.
JANUA provides better security, build relationships, and enable new cloud, mobile, and IoT offerings from any device or connected thing.
Olivier Rivat