Localization and the DITA Open Toolkit

Localization in the DITA Open Toolkit begins with the DITA XML attribute xml:lang. This attribute specifies the language and country of the contents of a particular element. In order for DITA OT localization to work, you must use the xml:lang attribute on the root element of your map (or bookmap) and on the root elements of any topics referenced by the map. If the xml:lang attribute is not specified, the DITA OT defaults to US English.

For instance, this xml:lang attribute indicates that a Concept topic contains Canadian French content:

<concept id="c_202934802" xml:lang="fr-ca">

When DITA topic and maps are localized, translators should add or modify the xml:lang attribute, specifying the target language and locale.

Tip: Even if you’re creating content for US English, it’s a good idea to add the xml:lang attribute when creating topics and maps. That way, the attribute is present in the files when they are received by the translator; this helps remind the translator that the xml:lang attribute must be updated.

When DITA Open Toolkit plugins transform DITA topics into output, they must occasionally insert additional text. For example, chapter titles (in English) often contain the word “Chapter”, admonitions usually need the labels “Note” or “Warning”, and page footers might contain a statement forbidding copying. These text strings are usually different in each target language.

These inserted text strings are not stored in the DITA topics; some come from the DITA Open Toolkit itself, other strings come from DITA OT plugins. Most plugins that create output define text strings. Usually they define multiple sets of strings, one for each language that the plugin supports.

<!-- Nav bar buttons -->  

<str name="contents_button">Contents</str>

<str name="index_button">Index</str>

<str name="search_button">Search</str>

<str name="print_button">Print</str>

It’s good programming practice to store strings in a separate file from code; the code then references each string when it is needed. This eliminates duplication and makes maintenance and translation much easier. Additionally, when a different language is required, a file (or set of files) with strings for the target language can be programmatically selected.

The DITA Open Toolkit uses this principle. Plugins can organize strings into parallel files, one file per language. The DITA OT selects the string file based on the xml:lang setting.

String storage and retrieval has been present in the DITA OT since the first versions. This original file organization is still used by most HTML plugins; this paper refers to these as “HTML strings files.” Strings are retrieved from these files with the getString template.

Idiom Technologies, the creators of the default PDF plugin in the DITA Open Toolkit, was aware of many issues associated with internationalization and localization. In creating the PDF plugin, Idiom realized that returning a single string was not flexible enough for some situations. Instead, Idiom ignored the HTML strings files (and the getString template) and implemented a much more sophisticated series of XSL templates. This paper refers to these as “PDF strings files.” Strings are retrieved from these fies with the insertVariable template.

The DITA Open Toolkit 2.1 introduced a more generalized way of accessing strings (the getVariable template), which works equally well with both HTML string files and PDF string files.

All three of these mechanisms are present in the modern DITA Open Toolkit (currently v2.5). Newer XSL stylesheets should use the getVariable template and the PDF string files. But it’s quite common to find legacy plugins that use the older templates and the HTML strings files.

Computers use numbers (as codes) to represent everything they have to process, from ID codes to machine instructions to characters (or “glyphs”) in human languages. Thus, in a computer, a lower-case “a” can be represented by the value (or code) 97. Each character in a given character set has a different code.

For many years, character sets were limited by the number of values (or characters) that could be represented by a single byte (usually 255 characters). Character sets were also constrained by the fact that each byte represented exactly one character. Conflicts could arise, because one character set might represent characters differently from another; additionally, the same codes might represent different characters in different character sets. You can still see this when using older versions of fonts, such as Zapf Dingbats, where the same code you use for a lowercase “s” instead represents a filled triangle (▲). If you have dealt with these kind of character sets and fonts, you know you need to use the right font for each character, otherwise your output will be garbled.

With the advent of Unicode (a variable, multi-byte encoding scheme), almost every character known in the world has a specific code or “code point.” Unicode can contain hundreds of thousands of code points. In Unicode if you need to write a lower case “s”, you use one code point (U+0073); if you need to write a filled triangle, you use a different code point (U+25B2). This frees you from worrying about using the right character set and the right font, unless you have a need to use one of these older fonts.

Although Unicode provides internal representation for almost every known character, your computer still needs be able to display that character on screen or print it on a piece of paper. This is the role of computer fonts².

Crafting a font that can represent a full set of Unicode code points is a massive undertaking. Instead, most fonts implement a more focused subset of code points (such as those required by a regional set of languages). A font usually contains characters and punctuation for a given language or region, plus a number of additional symbol characters. A typical font will implement up to 40,000 code points.

When localizing DITA content, it is crucial that you find and use fonts that are appropriate for your content. The fonts you choose should contain glyphs for all the code points you use.

If you’re looking for full Unicode code point coverage, consider:

• The Google Noto font family probably contains the most complete set of code points that is currently available. It is so large, it is divided into subsets so that the size of font files remains reasonable.
• For Western languages, the Arial Unicode MS font is one of the more densely populated fonts, containing more than 50,000 code points. It is distributed in Microsoft Office, but also bundled in Mac OS X v10.5 and later.

Index generation is a crucial aspect of localization because indexes must be sorted differently in each target language. Out-of-the-box, only four DITA Open Toolkit transformation types provide index generation (JavaHelp, Eclipse Help, HTML Help, and PDF); the others do not.

For the HTML-based help plugins (JavaHelp, Eclipse Help, HTML Help), the process of extracting, sorting, and generating indexes is not implemented in XSL. Instead, these steps are performed by Java classes in lib/dost.jar. These classes use the language-specific sorting sequences that are a part of Java, thus the results are appropriate for your target language.

PDF index processing is performed by both XSL templates and by additional Java classes.

The next two sections describe the HTML and PDF index processing in greater detail.

When you localize your DITA content, remember that DITA OT plugins also contain language-specific information.

The strings, images, icons, and fonts used by your plugins must be translated or localized with the same care and cultural sensitivity as your DITA content.

Many parts of the DITA Open Toolkit were created with localization in mind. It’s much easier (and more consistent) to use those mechanisms than to try to make language-specific changes in your transforms.