Skip to content

Latest commit

 

History

History
130 lines (85 loc) · 4.39 KB

DEVELOPMENT.md

File metadata and controls

130 lines (85 loc) · 4.39 KB
Raw

Developing and maintaining ex_cldr

The ex_cldr library and its derivative packages are driven by data provided by the amazing CLDR project. Updates to CLDR are released twice a year which triggers the need to provision that data for ex_cldr.

ex_cldr operates on the json content generated from the CLDR xml data files. The remainder of this document describes the process by which CLDR data is generated and provisioned for ex_cldr.

Initial setup

Installation instructions are platform dependent. For a typical MacOS environment the following using Homebrew will suffice.

  1. Install git-lfs

git-lfs MUST be installed prior to cloning the repo since there are several large objects references from the repo.

brew install git-lfs
git lfs install
  1. Clone the CLDR repo

Prior to any development work the CLDR repository should be cloned to the same development machine upon which ex_cldr development is to take place.

NOTE that git-lfs MUST be installed prior to cloning the repo.

# Set to whatever directory is appropriate
export $CLDR_REPO="$HOME/development/cldr_repo"
git clone https://github.com/unicode-org/cldr $CLDR_REPO

Of course the locale name of the CLDR repo can be any valid repo name however for the remainder of this document it is assumed it is referenced by the shell variable $CLDR_REPO.

Java compiler and ANT

A java compiler and the ANT tool are requirements for generating the CLDR toolchain.

Java is pre-installed on MacOS so installation should not be required. ant is not installed by default so the following is required:

brew install ant

Clone the ex_cldr repository

Clone the ex_cldr repository into a suitable directory.

export EX_CLDR="$HOME/Development/ex_cldr"
git clone https://github.com/elixir-cldr/cldr $EX_CLDR

Updating CLDR content

When a new version of CLDR is released, notification is typically given on the Unicode blog. For example, this is the introduction of CLDR 37.

Whenever a new release is created, the local repo needs to be updated. A simple git pull is all that is required.

cd cldr_repo
git pull

Generating the CLDR utilities

CLDR data is primarily stored in a series of XML files. However ex_cldr operates on json data generated from these files. This section describes how to generate that json data.

  1. Generate the updated toolchain
cd $CLDR_REPO/tools/java
ant clean all jar
cd $OLDPWD
  1. Create the staging and production directories

Data generation is done via the file ldml2json which makes some assumptions about shell variable and directory locations. The staging directory is used used to store data after the first phase on expansion of CLDR and the production directory is the final location of the generated data.

mkdir $HOME/development/cldr_staging_data
mkdir $HOME/development/cldr_production_data
  1. Review and update the contents of ldml2json

Review the settings of the first 20 lines of ldml2json to ensure the environment variables are set to match your development environment. The file is also defined to use /bin/zsh. Change this to the appropriate shell for your environment.

  1. Copy ldml2json into a directory in your path

To make it easy to execute, copy the ldml2json file into any directory that is in your $PATH. Make sure the execution flag is set.

chmod u+x $EX_CLDR/ldml2json
cp $EX_CLDR/ldml2json /some/directory/in/path
  1. Download the up-to-date ISO currency database
mix cldr.download.iso_currency
  1. Execute ldml2json
ldml2json

Execution will take quite a few minutes.

Update ex_cldr content

With the CLDR content now generated an in place in the $CLDR_PRODUCTION directory we can now generate the consolidated content used in ex_cldr.

  1. Update the repo
cd $EX_CLDR
git pull
  1. After updating the respository, the locales need to be consolidated into the format used by ex_cldr. This is done by:
mix cldr.consolidate
  1. Then regenerate the language_tags.ebin file by executing the following. This task will run with MIX_ENV=test to ensure all available locales are generated.
mix cldr.generate_language_tags