daniel-sc / xliff-simple-merge Goto Github PK

Merges XLIFF 1.2/2.0 files. Usable for Angular i18n automation.

License: MIT License

TypeScript 98.36% JavaScript 1.64%

xliff-simple-merge's Introduction

XLIFF Simple Merge

This program automates the merging of XLIFF files (version 1.2 and 2.0). New translations from the input file (e.g. "messages.xlf") are merged into the destination file (e.g "messages.fr-FR.xlf"), while keeping exising translations intact. Removed translations will be removed in the input file.

This can be used as post-processing to angular i18n extraction, to update translations files.

Angular users: It is recommended to not use this program directly but the custom angular tooling: ng-extract-i18n-merge

Usage

Either install via npm i -g xliff-simple-merge or run directly with npx xliff-simple-merge.

Options:
  -i, --input-file <inputFiles...>          input file(s)/merge origin(s)
  -d, --destination-file <destinationFile>  merge destination
  -o, --output-file <outputFile>            output file, if not provided "merge destination" is overwritten
  -e, -exclude-file <excludeFiles...>       exclude all unit IDs of the provided file(s) 
  -w, --overwrite-with-translated           overwrite target of destination with target of source, if it's translated and destination target not
  --no-match-fuzzy                          prevent fuzzy matching of similar units with changed id
  --no-collapse-whitespace                  prevent collapsing of multiple whitespaces when comparing translations sources
  --no-reset-translation-state              prevent (re-)setting the translation state to new/initial for new/changed units
  --no-replace-apostrophe                   prevent replacing of apostrophes (') with "&apos;"
  --debug                                   enable debug output
  -h, --help                                display help for command

Notes

If different input files contains the same unit IDs, only the latter ones will be used.
The input units may contain target nodes. They will be used in the destination units only if they don't contain any net.

xliff-simple-merge's People

Contributors

Stargazers

Watchers

Forkers

bugatino asgerjensen gergely-gyorgy-both pearnaly mpumford medondo michaelkrog

xliff-simple-merge's Issues

Support note tags

[Bug] The --exclude-file value is interpreted as the file content instead of the file path

Hi @daniel-sc, thanks for this useful library!

Summary

It seems that the --exclude-file file path is passed directly to the new XmlDocument, but this class expects the file content instead of the file path.

Execution and error

Error command:

npx xliff-simple-merge \
    -i projects/user-console/src/locale/messages.xlf \
    -e projects/shared/src/locale/messages.xlf \
    -d output.xlf

Error output:

/home/earenas/projects/my_project/node_modules/xmldoc/lib/xmldoc.js:92
    throw err;
    ^

Error: Non-whitespace before first tag.
Line: 0
Column: 1
Char: p
    at error (/home/earenas/projects/my_project/node_modules/sax/lib/sax.js:652:10)
    at strictFail (/home/earenas/projects/my_project/node_modules/sax/lib/sax.js:678:7)
    at beginWhiteSpace (/home/earenas/projects/my_project/node_modules/sax/lib/sax.js:952:7)
    at SAXParser.write (/home/earenas/projects/my_project/node_modules/sax/lib/sax.js:1007:11)
    at new XmlDocument (/home/earenas/projects/my_project/node_modules/xmldoc/lib/xmldoc.js:281:17)
    at /home/earenas/projects/my_project/node_modules/xliff-simple-merge/dist/src/merge.js:184:160
    at Array.map (<anonymous>)
    at mergeWithMapping (/home/earenas/projects/my_project/node_modules/xliff-simple-merge/dist/src/merge.js:184:141)
    at merge (/home/earenas/projects/my_project/node_modules/xliff-simple-merge/dist/src/merge.js:116:37)
    at Object.<anonymous> (/home/earenas/projects/my_project/node_modules/xliff-simple-merge/dist/src/index.js:29:37)

Node.js v18.15.0

The sax library error indicates the exclude file starts with the invalid character p instead of starting by a proper XML tag. However, if I change the --exclude-file option to aaa.xlf it shows the same error but telling that a is not a valid start (the file aaa.xlf doesn't even exist).

Workaround

Passing the actual content works:

npx xliff-simple-merge \
    -i projects/user-console/src/locale/messages.xlf \
    -e "$(cat projects/shared/src/locale/messages.xlf)" \
    -d output.xlf

Possible solution

I debugged the execution and found that the mergeWithMapping function expects options.excludeFiles to be the XML content as it happens with the inFilesContent argument:

export function mergeWithMapping(inFilesContent: string | string[], destFileContent: string, options?: MergeOptions, destFilePath?: string): [mergedDestFileContent: string, idMappging: { [oldId: string]: string }] {
    inFilesContent = Array.isArray(inFilesContent) ? inFilesContent : [inFilesContent];
    const inDocs = inFilesContent.map(inFileContent => new XmlDocument(inFileContent)); // <-------------------- XmlDocument uses the inFileContent
    const xliffVersion = inDocs[0].attr.version as '1.2' | '2.0' ?? '1.2';

    const destDoc = new XmlDocument(destFileContent.match(/^[\n\r\s]*$/) ? createEmptyTarget(xliffVersion === '2.0', extractSourceLocale(inDocs[0], xliffVersion === '2.0'), extractTargetLocale(destFilePath)) : destFileContent);
    const excludeDocs = (options?.excludeFiles ?? []).map(excludeFile => new XmlDocument(excludeFile));  // <-------------------- BUT this excludeFile is not the content but the path!!

    const destUnitsParent = xliffVersion === '2.0' ? destDoc.childNamed('file')! : destDoc.childNamed('file')?.childNamed('body')!;
    const excludeUnits = excludeDocs.map(excludeDoc => getUnits(excludeDoc, xliffVersion) ?? []).flat(1);
    const excludeUnitsId = new Set<string>(excludeUnits.map(unit => unit.attr.id!));
    const inUnits = inDocs.map(inDoc => getUnits(inDoc, xliffVersion) ?? []).flat(1).filter(inUnit => !excludeUnitsId .has(inUnit.attr.id));
   ...

I suppose that the fix should be loading the file at the index.ts as you do with the inFilesContent variable.

Please, let me know if I should provide more information or any kind support.

Module not found

I tried to upgrade to latest cliff-simple-merge (from 0.12.4) but it seems like it is unable to finde merge.js.
I tried via npx xliff-simple-merge with the same result.

(I trimmed my folders away from the following example)

Error: Cannot find module './merge'
Require stack:
- /.../node_modules/xliff-simple-merge/dist/src/index.js
    at Module._resolveFilename (node:internal/modules/cjs/loader:1077:15)
    at Module._load (node:internal/modules/cjs/loader:922:27)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/.../node_modules/xliff-simple-merge/dist/src/index.js:10:17)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12) {
  code: 'MODULE_NOT_FOUND',
  requireStack: [
    '/.../node_modules/xliff-simple-merge/dist/src/index.js'
  ]
}

TypeError: Cannot read properties of undefined (reading 'attr')

xliff-simple-merge --input-file messages.xlf --destination-file messages.de.xlf -output-file output.xlf
...npm\node_modules\xliff-simple-merge\dist\src\merge.js:54
destUnit.childNamed('target').attr.state = 'new';
^

TypeError: Cannot read properties of undefined (reading 'attr')

Prevent replacement of apostrophes with html character entities

Currently, apostrophes ( ' ) in xliff-files in <source> or <target> tags will be replaced by ' when running xliff-simple-merge.
Is there a way to prevent this behaviour?
Thanks in advance

Merge changes to notes

It seems like no changes to <note> items are carried over from the source file to the destination file.

For example, if a trans-unit already exists in the target file and the <note from="description"> for the unit has been updated in the source file, the updated description will not carry over into the destination file.

It feels like the expected behaviour would be that everything in the <trans-unit> should be updated, except for the <target>.

Not copying source to target when "newTranslationTargetsBlank" is true and "sourceLanguage" is also true

Hi!

I would like to generate blank target nodes for every locale, but not the one which is the source language. Currently, using the mentioned options together, in the sourceLanguage locale, I get <target state="final"/>, which is, I think, not the desired behavior.
An expected result would be (giving that the source language is in English, other locale is German:

English:

<trans-unit id="test" datatype="html">
        <source>This is the source code text</source>
        <target state="final">This is the source code text</target>
        <note priority="1" from="description">test</note>
</trans-unit>

German:

<trans-unit id="test" datatype="html">
        <source>This is the source code text</source>
        <target state="new"/>
        <note priority="1" from="description">test</note>
</trans-unit>

If you agree, I could make a pull request with the fix.

No way to prevent newlines being added to the merged output

I have issues with unwanted newlines in the translated files.

Fx. an extracted messages.xlf may have a unit that looks like this:

      <trans-unit id="AccountingView.Connected" datatype="html">
        <source>Connected to <x id="INTERPOLATION" equiv-text="{{system}}"/></source>
        <context-group purpose="location">
          <context context-type="sourcefile">../../accounting/src/lib/settings/settings.component.html</context>
          <context context-type="linenumber">25</context>
        </context-group>
      </trans-unit>

When merging that into an existing translated file, using xliff-simpl-merge, I get something like this (newlines added by xliff-simple-merge):

      <trans-unit id="AccountingView.Connected" datatype="html">
        <source>
          Connected to 
          <x id="INTERPOLATION" equiv-text="{{system}}"/>
        </source>
        <target state="new">
           Connected to 
          <x id="INTERPOLATION" equiv-text="{{system}}"/>
        </target>
        <context-group purpose="location">
          <context context-type="sourcefile">../../accounting/src/lib/settings/settings.component.html</context>
          <context context-type="linenumber">25</context>
        </context-group>
      </trans-unit>

In the end that comes out as new lines in my Angular app as well.

Failure on 1.1.0

Running merge on .xlf files generated by Angular gives error now. It works properly on 1.0.2.

npx --yes [email protected] -i projects/frontend/src/locale/messages.xlf -d projects/frontend/src/locale/
messages.es.xlf

C:\Users\myusername\AppData\Local\npm-cache\_npx\50a0e47d63d38cbd\node_modules\xliff-simple-merge\dist\src\index.js:26
const inFilesContent = options.inputFiles.map(inputFile => fs_1.default.readFileSync(inputFile, { encoding: 'utf8' }));
                                          ^

TypeError: Cannot read properties of undefined (reading 'map')
    at Object.<anonymous> (C:\Users\myusername\AppData\Local\npm-cache\_npx\50a0e47d63d38cbd\node_modules\xliff-simple-merge\dist\src\index.js:26:43)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:77:12)
    at node:internal/main/run_main_module:17:47

messages.zip

Running under Git Bash (probably was installed with TortoiseGit) on Windows Version 22H2 (OS Build 22621.1413) (64-bit).

xliff-simple-merge fails on unix host

How to reproduce:
$ yarn add -D xliff-simple-merge
$yarn run xliff-simple-merge --help

Result:
yarn run v1.22.19
$ /home/datav/dv-connect/itron/itron-common/node_modules/.bin/xliff-simple-merge --help
/usr/bin/env: ‘node\r’: No such file or directory
error Command failed with exit code 127.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

This is caused by the bin file having DOS/Windows line endings.

support translation state for xliff 1.2

In XLIFF 1.2 this is the attribute "state" on the element. It usually is "new" for newly added translations or "translated" for actually translated values.