[HELP] Change or define charset / encoding?

[jmbluethner]

Hey there!

I was wondering if there's a way to adjust the charset? I can't find anything like that in the docs.
I'm from Germany, so we have quite a lot of datasets which contain special Characters like ä,ö,ü, Unfortunately, those show up as � characters.

Any help would be appreciated :)

Thanks in advance and have an amazing day.

[jmbluethner]

Update: I tried to add CHARSET=UTF-8, CHARSET=ISO8859-1 and CCSID=1208 to the DSN string, but no success so far.

[Ted-CAcert]

Update: I tried to add CHARSET=UTF-8, CHARSET=ISO8859-1 and CCSID=1208 to the DSN string, but no success so far.

I have recently tried to use "CharSet=utf8" (without the dash), and had at least a partial success with SAP Anywhere database. Now writing to the database from node works at least for the characters supported by the locally configured codepage (in my case 1252), and maybe even some more. Sadly, at least in my setup, it did not help to write the more remote codepoints... :-\

[kadler]

@jmbluethner setting the charset is a driver-specific thing, so it would help to know what driver you are using.

[Ted-CAcert]

@jmbluethner setting the charset is a driver-specific thing, so it would help to know what driver you are using.

Using the ODBC driver for SAP Anywhere 17 on Windows 10. According to ODBC Admin the version number is 17.00.11.6933.

My guess is that this is not so much an issue of the database side of the driver, but the handling by the ODBC interface itself, and probably changing the local settings of Windows changes things. My regional settings are set to Germany, which (afaik) implies codepage 1252 "for non unicode programs".
The database codepage is set to 1252.

[kadler]

The current design of the node-odbc, even on Windows is that it binds all character data as SQL_C_CHAR and interprets it as UTF-8. If the driver does not support setting the local encoding, then this will fail for data which isn't UTF-8.

There's an existing issue to support binding as SQL_C_WCHAR instead: #292

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[jmbluethner]

I'm using Progress OpenEdge 11.7.

I did some packet capturing while doing test connects in Windows ODBC settings to see what those requests look like, and I found out that there are two arguments which are beeing sent:

I have than added these into my DSN connection string

const connectStr = "DSN=<NAME>;HOST=<ADDR>;DB=<DB>;UID=<USER>;PWD=<PWD>;PORT=<PORT>;codePage=UTF-8;clobCodePage=UTF-8;"

  try {
      return await odbc.connect(connectStr);
  } catch(e) {
      console.log(e);
      process.exit();
      return false;
  }

Unfortunately I still have wrong encodings when logging the results :/

let res = [];
await connectToDatabase().then(async (conn) => {
    res = await conn.query(<SQL_QUERY_STRING>);
    await conn.close();
});
console.log(res);
return res;

Result:

[
  {
    Bezeichnung: 'Ladeger´┐¢t'
  },
  {<...>}
]

[jmbluethner]

I now found out that there seems to be another argument called IANAAppCodePage.
See https://docs.progress.com/de-DE/bundle/datadirect-odbc-reference/page/IANAAppCodePage-values.html

However, adding IANAAppCodePage=5 for ISO 8859-2 still does not change anything ...
According to https://de.wikipedia.org/wiki/ISO_8859-2 ISO 8859-2 should contain Ä,Ö und Ü. In Theory ...

But as this seems to be a problem with my specific driver, and not a problem that's directly connected to node-odbc, I might close this issue.

[Relbot]

We had the same problem. I tried everything the only possible way for us was to host the nodejs server on a linux based system. Not all linux versions work. We use Fedora Linux 38 (Workstation Edition)all special characters (Umlaute) show perfectly normal ÜÖÄüöä.
But we came across a new problem with decimal numbers, they lose their decimal places when using node-odbc: #406

Hope this helps

Regards from Germany
Tell

[jmbluethner]

@Relbot Thanks for your reply fellow German! :)

I've now adjusted my entire repo to make the API run on linux as well, but unfortunately - No luck :(

I've opened a ticket with Progress, so, we'll see.
I'll keep this task updated.

Best regards

[Relbot]

@jmbluethner which linux are you using ?

[jmbluethner]

@Relbot Debian 12. But that shouldn't make a difference, right? The ODBC drivers will be the same, no matter which distro it is. Or am I missing something?

[Relbot]

@jmbluethner
But that shouldn't make a difference, right?
I initially thought the same, but when I tested it on other Linux versions like Ubuntu, it didn’t work for me. While the driver remains the same, Fedora translates certain data types differently through the ODBC interface. (I’m not entirely sure why; it took me weeks to figure out.)

Try Fedora Linux 38 (Workstation Edition). Installation is straightforward, but one issue remains: decimal numbers tend to lose their decimal places. #406