globalizejs / globalize-compiler Goto Github PK

Hello,

I use the following function to get translations:

import Globalize from 'globalize';
export function t(category, key, ...params) {
    return Globalize.formatMessage(category + '/' + key, ...params);
}

Here I have the following issues:

• I can't use import, because babel renames variables
• I can't use spread operator, because the function will be expanded to e.g. Globalize.formatMessage.apply(Globalize, [category + '/' + key].concat(Array.from(arguments)))
• and finally I can't use wrapper because ast won't be used to parse for different calls

Is there any workaround or it is impossible to get compiler working?

I failed to run basic-globalize-compiler example (it seems that some points are missed from readme), but nevertheless I have a question: what about compiling data for all locales?

Example shows a situation when we know the locale (en), but what if locale is defined dynamically in runtime and we do not know it before?

Here is a PR to fix: #22

From the docs it's not clear what exactly does this compiler do.

Suppose my app formats currencies for two languages.
Raw CLDR data may be bundled and loaded at runtime, but as I assume the purpose of this compiler is to compile that CLDR data into a smaller subset (or isn't it) leaving just the fields required.
What will be the pseudocode command for doing that?
And where to should the output files be fed.

Currently, Globalize static methods are extracted but not instance usage. Consider supporting instance usage as well.

Instead of having to hack the output of the compiler like this or using the SkipAMDPlugin, you could just add an option to the compiler:

globalizeCompiler.compile(formattersAndParsers, {
    template: fs.readFileSync( __dirname + "/my_globalize.template" ).toString( "utf-8" );
});

For globalizejs/globalize#563 to work with the compiler, messageFormatters must be last, since they can now depend on anything.

This shouldn't cause any issues, since nothing else depends on messageFormatters.

Creating a new issue that is related to some discussion I saw in #10 to talk about having the compiler support automatically "pre-compiling" each currencyFormatter with a pre-set list of currencies provided separately. In cases where the currency may be dynamically chosen, but from a set of known values, it would be much simpler to be able to declare the formatter once, and have the compiler automatically expand this.

A similar behavior could be added for units, which I believe has a similar pattern.

I'm not sure how to implement this, but as far as API goes, having another property on options passed to compile(), such as supportedCurrencies/supportedUnits seems like the right approach. Without this present, the compiler would take the formatters literally, but with them present (or, perhaps with them present with a special sentinel value for currency/units), it would auto-expand and generate individual formatters for each of the passed in currencies/units.

The examples that I have seen, here and in the globalization repo, leave me wondering how to load CLDR data. This data, as you know, comes from multiple JSON files yet the examples show loading a single, manually created file, like this example.

Are there examples other than the extremely simple use case that doesn't really articulate how someone would use this in the real world? Why not have an example that illustrates how one would accomplish compiling actual data from the CLDR source? What good is manually creating a single file? Who would do that and why?

Updated 28 Sep
Just through a lot of trial and error I have determined that globalize-compiler just utilizes the CLDR data installed as a dependency, via npm install, to ensure the necessary information is added to the "compiled" js file. Hopefully, some of this information is useful for others attempting to use this library.

I don't see a lot of activity in this repo and I can see hours of frustration headed my way if there isn't any support for this. Will someone be able to assist?

Thanks in advance.

Hello @rxaviers,

What's your tip to create a compiled globalize version which includes multiple locales ?

May I have stumbled on a compiler bug?

I'm creating an HTML <select> control for timezones in their i18n names. Compilation of the CLDR logic is essential in this case for performance reasons.

The formatter works wonderfully (albeit slowly) in production but fails to compile:

# npm run build
...
undefined:4
    'timeZone': tz.name,
...                ^

The contentious line in app.js is:

s = Globalize.formatDateToParts( dummy_time, {"timeZone": tz.name, "datetime": "full"} )[14].value;

Indeed, replacing tz.name with a literal such as "America/Los_Angeles" eliminates the error. Of course, iterating over all IANA timezones manually in this fashion is not option :) Instead, I suckle all IANA names from moment.tz.names() a few lines earlier.

Happy to provide the complete code (converted into a minimal working example for the purposes of this discussion).

To repeat, this code works fine in development, producing no errors in the browser inspector.

Thanks in advance for your help.

Maybe if I submit some additional issues, someone will respond...

Issue

If I have the following code in my project:

    var curr = 'USD';
    var currentcyFormatter = Globalize.currencyFormatter( curr );
    var cur = currentcyFormatter(123.123123);

I get the following error message curr is not defined
I think the compiler is trying to be a little bit too smart here and is trying to figure out which cldr data to include. But when I load stuff like this e.g. from a API, it can't know this just from the code.

So is there a option to disable this and to just tell the compiler maybe statically which data to include instead of trying to deduce it from the source?

Hello,

We are using Globalize for some time now, but only for messages without plurals, so we only needed supplemental/likelySubtags in our application.

Now that we want to use all modules, we also need to add the CLDR data.
And we discovered that loading all required cldr datasets ( supplemental/likelySubtags, supplemental/numberingSystems, supplemental/plurals, supplemental/ordinals, supplemental/currencyData, supplemental/timeData, supplemental/weekData, main/{locale}/numbers, main/{locale}/currencies, main/{locale}/ca-gregorian, main/{locale}/timeZoneNames, main/{locale}/dateFields, main/{locale}/units )

We end up with 150K for the main data and 150K per locale.
This is quite big and we feel it's not optimal.

Is there a way to reduce this ? strip some parts of these files that are never used by globalize anyway ?

I was experimenting with globalizejs/globalize#720 when I found this.
If runtimeArgs contains undefined, the compiler will generate invalid code. E.g.:

var Globalize = require( "globalize" );
var globalizeCompiler = require('globalize-compiler');

Globalize.load( require( "cldr-data" ).entireSupplemental() );
Globalize.load( require( "cldr-data" ).entireMainFor( "en" ) );

var GyMMMd = Globalize( "en" ).dateFormatter({ skeleton: "GyMMMd" });
GyMMMd.runtimeArgs.push(undefined);
GyMMMd.runtimeArgs.push(undefined);

var out = globalizeCompiler.compile([
    GyMMMd,
], {});

The generated code will look like this:

Globalize.b938155015 = dateFormatterFn({"1":Globalize("en").numberFormatter({"raw":"0"})}, {"pattern":"MMM d, y G","timeSeparator":":","months":{"M":{"3":{"1":"Jan","2":"Feb","3":"Mar","4":"Apr","5":"May","6":"Jun","7":"Jul","8":"Aug","9":"Sep","10":"Oct","11":"Nov","12":"Dec"}}},"eras":{"0":"BC","1":"AD","0-alt-variant":"BCE","1-alt-variant":"CE"}}, , );

The , , ); is a syntax error.

The replacement happens here: https://github.com/globalizejs/globalize-compiler/blob/v1.0.0/lib/compile.js#L142

I suspect this isn't likely to come up in real-world usage, but shouldn't the compiler generate undefined? i.e.:

Globalize.b938155015 = dateFormatterFn({"1":Globalize("en").numberFormatter({"raw":"0"})}, {"pattern":"MMM d, y G","timeSeparator":":","months":{"M":{"3":{"1":"Jan","2":"Feb","3":"Mar","4":"Apr","5":"May","6":"Jun","7":"Jul","8":"Aug","9":"Sep","10":"Oct","11":"Nov","12":"Dec"}}},"eras":{"0":"BC","1":"AD","0-alt-variant":"BCE","1-alt-variant":"CE"}}, undefined, undefined);

As per http://www.unicode.org/reports/tr35/#Locale_Extension_Key_and_Type_Data a locale should be able to handle unicode extensions such as

• ca (Calendar algorithm)
• cf (Currency format style)
• co (Collation style)
• cu (Currency type)
• em (Emoji presentation style)
• fw (First day of week)
• hc (Hour cycle)
• lb (line break style)
• lw (line break word handling)
• ms (measurement system)
• nu (numbering system)

Now, I know many of these options are not supported by Globalize, but I know we can atleast handle -u-nu (numbering system), -u-cu (currency type) and -u-cf- (currency format style)

When running this in the Globalize compiler, there is an assumption that the passed in locale would match the CLDR directory structure.

➜  node-sandbox ./node_modules/.bin/globalize-compiler -l ar-EG-u-nu-latn -o dest.js app.js        
fs.js:808
  return binding.readdir(pathModule._makeLong(path));
                 ^

Error: ENOENT: no such file or directory, scandir '/home/site/ExternalRepos/node-sandbox/node_modules/cldr-data/main/ar-EG-u-nu-latn'
    at Error (native)
    at Object.fs.readdirSync (fs.js:808:18)
    at jsonFiles (/home/site/ExternalRepos/node-sandbox/node_modules/cldr-data/index.js:36:22)
    at /home/site/ExternalRepos/node-sandbox/node_modules/cldr-data/index.js:60:21
    at Array.reduce (native)
    at mainPathsFor (/home/site/ExternalRepos/node-sandbox/node_modules/cldr-data/index.js:59:18)
    at Function.cldrData.entireMainFor (/home/site/ExternalRepos/node-sandbox/node_modules/cldr-data/index.js:90:29)
    at cldr (/home/site/ExternalRepos/node-sandbox/node_modules/globalize-compiler/lib/compile-extracts.js:30:57)
    at Object.compileExtracts (/home/site/ExternalRepos/node-sandbox/node_modules/globalize-compiler/lib/compile-extracts.js:55:18)
    at Object.<anonymous> (/home/site/ExternalRepos/node-sandbox/node_modules/globalize-compiler/bin/globalize-compiler.js:82:28)

How should I resolve this? The problem is that when using Globalize with the globalize-compiler and the webpack plugin, we are assuming that the locale passed to Globalize matches the CLDR data directory.

Any guidance would be helpful.

Globalize has updated parseNumber and its runtimeArgs have regexps, which aren't supported by the compiler.

Say I have this code:

let pgencard = Globalize.pluralGenerator({type: 'cardinal'});
let pgenord = Globalize.pluralGenerator({type: 'ordinal'});
let pgendefault = Globalize.pluralGenerator();

Compiled, I get something like this (excerpt, transpiled with babel):

_globalizeRuntime2.default.a1662346136 = pluralGeneratorFn(function (n) {
  var s = String(n).split('.'),
      v0 = !s[1];
  return n == 1 && v0 ? 'one' : 'other';
});
_globalizeRuntime2.default.b996364137 = pluralGeneratorFn(function (n) {
  var s = String(n).split('.'),
      t0 = Number(s[0]) == n,
      n10 = t0 && s[0].slice(-1),
      n100 = t0 && s[0].slice(-2);
  return n10 == 1 && n100 != 11 ? 'one' : n10 == 2 && n100 != 12 ? 'two' : n10 == 3 && n100 != 13 ? 'few' : 'other';
  ;
});
_globalizeRuntime2.default.b1125263556 = pluralGeneratorFn(function (n) {
  var s = String(n).split('.'),
      v0 = !s[1];
  return n == 1 && v0 ? 'one' : 'other';
});

Notice that the cardinal plural generator and the default plural generator have the same function body. Calling without args should reference the same function as calling with default args, else there is duplication in the compiled code.

I'm trying to use globalize parser but I get an error in production, in development example is working.

Globalize.dateParser({`` date: "medium" })( "Nov 1, 2010" );
Uncaught TypeError: Globalize.dateParser(...) is not a function

Globalize.parseNumber("1000.10");
Uncaught TypeError: this.numberParser(...) is not a function

Include tests to check babel transforms.

Ideally, we should be able to test it for various babel versions (for instance, 5.x and 6.x). Challenge: have npm to install multiple babel version. jQuery UI uses bower. TodoMVC repo addresses that with a subdirectory with its own package.json file. See comments at #12 (diff)

Following globalizejs/globalize#645 suggestion, we are considering to move this project into @globalizejs for improved organization.

Version 1.1.1 of globalize-compiler generates code that produces the following error:

compiled-formatters.js:53 Uncaught ReferenceError: messageFormat is not defined
    at compiled-formatters.js:53
    at compiled-formatters.js:15
    at compiled-formatters.js:17

This does not occur with version 1.0.0.

The easiest way to reproduce this is to set up and run globalize's example code: https://github.com/globalizejs/globalize/tree/master/examples/globalize-compiler

Follow the instructions for production.html, start the server, and load in the browser (I'm using Chrome). Check the console in the browser's dev tools, and you'll see the error.

The errors generated by globalize-compiler are pretty cryptic and not useful when it fails to parse a file.

time.js

import Globalize from 'globalize';

/**
 * Intl Format a short time string e.g. "9:30 AM" in en-US
 */
export const formatTime = Globalize.dateFormatter({ date: 'short' });

globalize-compiler -l en -o en.js time.js

Gives the following result:

{...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:5702
            throw e;
            ^
Error: Line 1: Unexpected token
    at constructError ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:2407:21)
    at createError ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:2426:17)
    at unexpectedTokenError ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:2500:13)
    at tolerateUnexpectedToken ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:2509:21)
    at parseStatementListItem ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:3973:21)
    at parseScriptBody ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:5490:25)
    at parseProgram ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:5506:16)
    at Object.parse ({...}/node_modules/globalize-compiler/node_modules/esprima/esprima.js:5690:23)
    at Object.extractor [as extract] ({...}/node_modules/globalize-compiler/lib/extract.js:39:17)
    at {...}/node_modules/globalize-compiler/bin/globalize-compiler.js:84:27 {
  lineNumber: 1,
  description: 'Unexpected token',
  index: 0
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

In this example it's clear that globalize-compiler doesn't support ES20153. But It doesn't tell you what file the failure is. And if the error was somewhere in the middle of the file it wouldn't necessarily be easy to discover what the issue is.

Ref https://github.com/rxaviers/react-globalize-compiler

I'm trying to display a currency other than USD, such as EUR. To troubleshoot, I started with a simpler example:

https://github.com/jquery/globalize/tree/master/examples/globalize-compiler

In package.json, I changed the locale from en to de:

"build": "globalize-compiler -l de -c cldr.json -m messages.json -o compiled-formatters.js app.js"

In app.js, I changed the locale from en to de:

Globalize.locale( "de" );

And, also in app.js, I changed USD to EUR:

document.getElementById( "currency" ).textContent = Globalize.formatCurrency( 69900, "EUR" );

For good measure, I also added a de entry for the like message in messages.json.

Here is the error I receive, below. Any pointers?

dev@dev:~/globalize/examples/globalize-compiler$ npm run build

> basic-globalize-compiler@ build ./globalize/examples/globalize-compiler
> globalize-compiler -l de -c cldr.json -m messages.json -o compiled-formatters.js app.js


./globalize/examples/globalize-compiler/node_modules/globalize/node_modules/cldrjs/dist/cldr.js:164
            return cldr.get( "supplemental/likelySubtags/und" ).split( sep );
                                                                ^
TypeError: Cannot call method 'split' of undefined
    at coreLikelySubtags (./globalize/examples/globalize-compiler/node_modules/globalize/node_modules/cldrjs/dist/cldr.js:164:56)
    at Cldr.init (./globalize/examples/globalize-compiler/node_modules/globalize/node_modules/cldrjs/dist/cldr.js:605:19)
    at Cldr.init (./globalize/examples/globalize-compiler/node_modules/globalize/node_modules/cldrjs/dist/cldr/event.js:554:13)
    at Cldr.init (./globalize/examples/globalize-compiler/node_modules/globalize/node_modules/cldrjs/dist/cldr/supplemental.js:92:13)
    at new Cldr (./globalize/examples/globalize-compiler/node_modules/globalize/node_modules/cldrjs/dist/cldr.js:547:8)
    at alwaysCldr (./globalize/examples/globalize-compiler/node_modules/globalize/dist/globalize.js:286:55)
    at Function.Globalize.locale (./globalize/examples/globalize-compiler/node_modules/globalize/dist/globalize.js:370:15)
    at Object.compileExtracts (./globalize/examples/globalize-compiler/node_modules/globalize-compiler/lib/compile-extracts.js:53:12)
    at Object.<anonymous> (./globalize/examples/globalize-compiler/node_modules/globalize-compiler/bin/globalize-compiler.js:82:28)
    at Module._compile (module.js:456:26)

npm ERR! basic-globalize-compiler@ build: `globalize-compiler -l de -c cldr.json -m messages.json -o compiled-formatters.js app.js`
npm ERR! Exit status 8
npm ERR! 
npm ERR! Failed at the basic-globalize-compiler@ build script.
npm ERR! This is most likely a problem with the basic-globalize-compiler package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     globalize-compiler -l de -c cldr.json -m messages.json -o compiled-formatters.js app.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls basic-globalize-compiler
npm ERR! There is likely additional logging output above.
npm ERR! System Linux 4.2.0-23-generic
npm ERR! command "/usr/bin/nodejs" "/usr/bin/npm" "run" "build"
npm ERR! cwd ./globalize/examples/globalize-compiler
npm ERR! node -v v0.10.25
npm ERR! npm -v 1.4.21
npm ERR! code ELIFECYCLE
npm WARN This failure might be due to the use of legacy binary "node"
npm WARN For further explanations, please read
/usr/share/doc/nodejs/README.Debian

npm ERR! 
npm ERR! Additional logging details can be found in:
npm ERR!     ./globalize/examples/globalize-compiler/npm-debug.log
npm ERR! not ok code 0

Hi,

For a date input component, I'd like to provide some help text to the user indicating the date format the input component is expecting. I've compiled my formatters and parsers, thus I no longer seem to have access to the underlying CLDR data to get the date format. Or am I mistaken -- is there some way to access the underlying CLDR data associated with a compiled formatter/parser? I'd prefer not to have to load the CLDR data by itself to get this information.

I could conceive a similar problem with a currency input component as it would be useful to know the currency symbol and which side to place the currency symbol.

Thanks for all your hard work!

Here I would like to address the following problem:
if one wants to use the globalize-compiler during the build process of a web project there are no the configuration files for that.
Specifically I am building in .Net in Azure. So I would expect to be able to download the latest package.json, containing only the dependencies needed the globalize-compiler to run - currently these are:
cldr-data
globalize
iana-tz-data
and globalize-compiler itself.
Of course this could change in the future. Thus I expect to get the package.json automatically from the actual repository instead of doing it by myself every time (or at least check if it is actual or leave it as is until the build will break missing some new dependency).

Further one would like to get all of the needed files (globalize-runtime.xxx) ready for copy in the output directory. It is just not nice to go and search in node_modules/globalize/dist.... and look for them ... one can also automatically detect which one are needed. So if for example we do not need globalize-runtime/currency.js it would be automatically not present in the output dir.

And last but not the least: why does the output from the compiler not working with the globalize.js files but needs another globalize-runtime.js files?

Best regards,
Alex