Skip to content

Commit

Permalink
✨ Add --with-arrays flag to enable support for arrays in source files
Browse files Browse the repository at this point in the history
Fixes #102
  • Loading branch information
leolabs committed Jan 10, 2024
1 parent 0425e22 commit 8cf65f5
Show file tree
Hide file tree
Showing 3 changed files with 36 additions and 15 deletions.
33 changes: 20 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,19 +181,25 @@ DeepL Free is limited to 500,000 characters translated per month.
After you have completed your sign-up, you can pass the API key to
json-autotranslate using the `-c` or `--config` option.

The value of the `--config` argument is a comma separated string with the following: `appKey,formality,batchSize`.

The `formality` argument currently only works for target languages "DE" (German), "FR" (French), "IT" (Italian),
"ES" (Spanish), "NL" (Dutch), "PL" (Polish), "PT-PT", "PT-BR" (Portuguese) and "RU" (Russian). Possible options are:

- "default" (default)
- "more" - for a more formal language
- "less" - for a more informal language

To improve performance and prevent DeepL rate-limiting json-autotranslate batches multiple tokens into a single translation request.
By default, the `batchSize` is set to `1000`, meaning that `1000` tokens are translated at once. This can be controlled by adjusting the value in the `--config` parameter.
This value was chosen because the DeepL prevents the body of a request to be larger than `128 KiB (128 · 1024 bytes)``. Based on experimentation, even with long tokens, this limit is not reached.

The value of the `--config` argument is a comma separated string with the
following: `appKey,formality,batchSize`.

The `formality` argument currently only works for target languages "DE"
(German), "FR" (French), "IT" (Italian), "ES" (Spanish), "NL" (Dutch), "PL"
(Polish), "PT-PT", "PT-BR" (Portuguese) and "RU" (Russian). Possible options
are:

- "default" (default)
- "more" - for a more formal language
- "less" - for a more informal language

To improve performance and prevent DeepL rate-limiting json-autotranslate
batches multiple tokens into a single translation request. By default, the
`batchSize` is set to `1000`, meaning that `1000` tokens are translated at once.
This can be controlled by adjusting the value in the `--config` parameter. This
value was chosen because the DeepL prevents the body of a request to be larger
than `128 KiB (128 · 1024 bytes)``. Based on experimentation, even with long
tokens, this limit is not reached.

<sup><a href="https://www.deepl.com/de/docs-api/translating-text/">Reference</a></sup>

Expand Down Expand Up @@ -297,6 +303,7 @@ Options:
-s, --service <service> selects the service to be used for translation (default: "google-translate")
--list-services outputs a list of available services
-m, --matcher <matcher> selects the matcher to be used for interpolations (default: "icu")
-a, --with-arrays enables support for arrays in files, but removes support for keys named 0, 1, 2, etc.
--list-matchers outputs a list of available matchers
-c, --config <value> supply a config parameter (e.g. path to key file) to the translation service
-f, --fix-inconsistencies automatically fixes inconsistent key-value pairs by setting the value to the key
Expand Down
13 changes: 12 additions & 1 deletion src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,10 @@ commander
/^(key-based|natural|auto)$/,
'auto',
)
.option(
'-a, --with-arrays',
`enables support for arrays in files, but removes support for keys named 0, 1, 2, etc.`,
)
.option(
'-s, --service <service>',
`selects the service to be used for translation`,
Expand Down Expand Up @@ -85,6 +89,7 @@ const translate = async (
sourceLang: string = 'en',
deleteUnusedStrings = false,
fileType: FileType = 'auto',
withArrays: boolean = false,
dirStructure: DirectoryStructure = 'default',
fixInconsistencies = false,
service: keyof typeof serviceMap = 'google-translate',
Expand Down Expand Up @@ -122,7 +127,11 @@ const translate = async (
sourceLang,
);

const templateFiles = loadTranslations(templateFilePath, fileType);
const templateFiles = loadTranslations(
templateFilePath,
fileType,
withArrays,
);

if (templateFiles.length === 0) {
throw new Error(
Expand Down Expand Up @@ -275,6 +284,7 @@ const translate = async (
const existingFiles = loadTranslations(
evaluateFilePath(workingDir, dirStructure, language),
fileType,
withArrays,
);

if (deleteUnusedStrings) {
Expand Down Expand Up @@ -386,6 +396,7 @@ translate(
commander.sourceLanguage,
commander.deleteUnusedStrings,
commander.type,
commander.withArrays,
commander.directoryStructure,
commander.fixInconsistencies,
commander.service,
Expand Down
5 changes: 4 additions & 1 deletion src/util/file-system.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ export const detectFileType = (json: any): FileType => {
export const loadTranslations = (
directory: string,
fileType: FileType = 'auto',
withArrays = false,
) =>
fs
.readdirSync(directory)
Expand All @@ -58,7 +59,9 @@ export const loadTranslations = (
type,
content:
type === 'key-based'
? flatten(require(path.resolve(directory, f)), { safe: true })
? flatten(require(path.resolve(directory, f)), {
safe: !withArrays,
})
: require(path.resolve(directory, f)),
} as TranslatableFile;
});
Expand Down

0 comments on commit 8cf65f5

Please sign in to comment.