GNU LilyPond Contributor's Guide: 3.7 Translating the documentation

## 3.7 Translating the documentation

### 3.7.1 Getting started with documentation translation

First, get the sources from the Git repository, see Documentation translations source code.

#### Translation requirements

Working on LilyPond documentation translations requires the following pieces of software, in order to make use of dedicated helper tools:

- Python 2.4 or higher,
- GNU Make,
- Gettext.

It is not required to build LilyPond and the documentation to translate the documentation. However, if you have enough time and motivation and a suitable system, it can be very useful to build at least the documentation so that you can check the output yourself and more quickly; if you are interested, see Compiling from source.

#### Which documentation can be translated

The makefiles and scripts infrastructure currently supports translation of the following documentation:

- documentation index (HTML);
- user manual and program usage – Texinfo source, PDF and HTML output; Info output might be added if there is enough demand for it;
- the News document.

The following pieces of documentation should be added soon, by descending order of priority:

- automatically generated documentation: markup commands, predefined music functions;
- the Snippets List;
- the examples page;
- the Internals Reference.

#### Starting translation in a new language

At top of the source directory, do

>     ./autogen.sh

or (if you want to install your self-compiled LilyPond locally)

>     ./autogen.sh --prefix=$HOME

If you want to compile LilyPond – which is almost required to build the documentation, but is not required to do translation only – fix all dependencies and rerun `./configure` (with the same options as for `autogen.sh`).

Then `cd` into ‘`Documentation`’ and run

>     make ISOLANG=MY-LANGUAGE new-lang

where MY-LANGUAGE is the ISO 639 language code.

Finally, add a language definition for your language in ‘`python/langdefs.py`’.

Before starting the real translation work, it is recommended to commit changes you made so far to Git, so e.g. you are able to get back to this state of the sources easily if needed; see Sharing your changes.

### 3.7.2 Documentation translation details

Please follow all the instructions with care to ensure quality work.

All files should be encoded in UTF-8.

#### Files to be translated

Translation of ‘`Documentation/foo/bar`’ should be ‘`Documentation/LANG/foo/bar`’. Unmentioned files should not be translated.

Priorities:

- 1. delivery,
- 2. 3. 4. 5. later,
- 6. optional.

Files marked with priority 3, 4 or 5 may be submitted individually. Word counts (excluding LilyPond snippets) are given for each file.

>     -1- Documentation index and Tutorial
>     429   user/lilypond-learning.tely
>     6343  user/tutorial.itely
>     23    user/dedication.itely
>     432   user/macros.itexi
>     171   index.html.in
>     161   translations.template.html.in
>     6438  po/lilypond-doc.pot (translate to po/MY_LANGUAGE.po)
>     ---   ../lilypond-texi2html.init (section TRANSLATIONS)
>     13997 total

>     -2- Introduction and beginning of Application Usage
>     411   user/preface.itely
>     3855  user/introduction.itely
>     407   user/lilypond-program.tely
>     193   user/install.itely (partial translation)
>     1149  user/setup.itely
>     2827  user/running.itely
>     8842  total

>     -3- Learning manual
>     10318 user/fundamental.itely -- Fundamental concepts
>     14816 user/tweaks.itely -- Tweaking output
>     4550  user/working.itely -- Working on LilyPond files
>     498   user/templates.itely -- Templates
>     30182 total

>     -4- Notation reference
>     724   user/lilypond.tely
>     91    user/notation.itely -- Musical notation
>     3145  user/pitches.itely
>     4772  user/rhythms.itely
>     1393  user/expressive.itely
>     555   user/repeats.itely
>     1456  user/simultaneous.itely
>     1708  user/staff.itely
>     905   user/editorial.itely
>     2376  user/text.itely
>     76    user/specialist.itely -- Specialist notation
>     2725  user/vocal.itely
>     1516  user/chords.itely
>     702   user/piano.itely
>     810   user/percussion.itely
>     826   user/guitar.itely
>     66    user/strings.itely
>     242   user/bagpipes.itely
>     4487  user/ancient.itely
>     6024  user/input.itely -- Input syntax
>     2164  user/non-music.itely -- Non-musical notation
>     8663  user/spacing.itely -- Spacing issues
>     11747 user/changing-defaults.itely -- Changing defaults
>     5187  user/programming-interface.itely -- Interfaces for programmers
>     1182  user/notation-appendices.itely -- Notation manual tables
>     252   user/cheatsheet.itely -- Cheat sheet
>     63794 total

>     -5- Application usage
>     3248  user/lilypond-book.itely -- LilyPond-book
>     1171  user/converters.itely -- Converting from other formats
>     4419  total

>     -6- Appendices whose translation is optional
>     310   user/literature.itely
>     960   user/scheme-tutorial.itely (needs to be revised first)
>     1270  total

#### Translating the Learning Manual and other Texinfo documentation

Any title which comes with one of the following commands must not be translated directly in the Texinfo source

>     @node                                                   @majorheading
>     @chapter       @unnumbered          @appendix           @chapheading
>     @section       @unnumberedsec       @appendixsec        @heading
>     @subsection    @unnumberedsubsec    @appendixsubsec     @subheading
>     @subsubsection @unnumberedsubsubsec @appendixsubsubsec  @subsubheading
>     @ref  @rglos  @ruser  @rlearning  @rprogram  @rlsr

The same applies to first argument of `@rmanualnamed` commands; however, the second argument Bar baz of `@ref{Foo,Bar baz,,info-file}` and `@rmanualnamed{Foo,Bar baz}` should be translated.

`@uref`’s names are to be translated.

In any section which looks like

>     @menu
>     * node1:: thing1
>     * node2:: thing2
>     ...
>     @end menu

the node names nodeN are not to be translated, whereas extra title information thingN is.

Every node name or section title must from now on be translated separately in a ‘`.po`’ file (just as well as LilyPond output messages) in ‘`Documentation/po`’. The Gettext domain is named `lilypond-doc`, and unlike `lilypond` domain it is not managed through the Free Translation Project.

Take care of using typographic rules for your language, especially in ‘`user/macros.itexi`’.

Please keep verbatim copies of music snippets (in `@lilypond` blocs). However, some music snippets containing text that shows in the rendered music, and sometimes translating this text really helps the user to understand the documentation; in this case, and only in this case, you may as an exception translate text in the music snippet, and then you must add a line immediately before the `@lilypond` block, starting with

>     @c KEEP LY

Otherwise the music snippet would be reset to the same content as the English version at next `make snippet-update` run – see Updating documentation translation.

When you encounter

>     @lilypondfile[<number of fragment options>,texidoc]{filename.ly}

in the source, open ‘`input/lsr/filename.ly`’, translate the `texidoc` header field it contains, enclose it with `texidocMY-LANGUAGE = "` and `"`, and write it into ‘`input/texidocs/filename.texidoc`’ – please keep possibly existing translations in other languages! Additionnally, you may translate the snippet’s title in `doctitle` header field, in case `doctitle` is a fragment option used in `@lilypondfile`; you can do this exactly the same way as `texidoc`. For instance, ‘`input/texidocs/filename.texidoc`’ may contain

>     doctitlees = "Spanish title baz"
>     texidoces = "
>     Spanish translation blah
>     "
>     doctitlede = "German title bar"
>     texidocde = "German translation foo
>     "

`@example` blocs need not be verbatim copies, e.g. variable names, file names and comments should be translated.

Index entries (`@cindex` and so on) should be translated.

Finally, please carefully apply every rule exposed in Texinfo introduction and usage policy, and Documentation policy. If one of these rules conflicts with a rule specific to your language, please ask the Translation meister and/or the Documentation Editors on lilypond-devel@gnu.org.

#### Translating the Notation Reference and Application Usage

Copy ‘`user/lilypond.tely`’ (or ‘`user/lilypond-program.tely`’, respectively) into ‘`MY-LANGUAGE/user`’, then translate this file and run `skeleton-update` – see Updating documentation translation. Your are now ready to translate the Notation Reference (Application Usage, respectively) exactly like the Learning Manual.

#### Translating the Documentation index ‘`index.html.in`’

Unlike almost all HTML pages in this documentation, links in this page are not tweaked by ‘`postprocess_html.py`’, so links should be manually edited to link to existing translations.

### 3.7.3 Documentation translation maintenance

Several tools have been developed to make translations maintenance easier. These helper scripts make use of the power of Git, the version control system used for LilyPond development.

#### Check state of translation

First pull from Git, then cd into ‘`Documentation/`’ (or at top of the source tree, replace `make` with `make -C Documentation`) and run

>     make ISOLANG=MY_LANGUAGE check-translation

This presents a diff of the original files since the most recent revision of the translation. To check a single file, cd into ‘`Documentation/`’ and run

>     make CHECKED_FILES=MY_LANGUAGE/user/foo.itely check-translation

To see only which files need to be updated, do

>     make ISOLANG=MY_LANGUAGE check-translation | grep 'diff --git'

To avoid printing terminal colors control characters, which is often desirable when you redirect output to a file, run

>     make ISOLANG=MY_LANGUAGE NO_COLOR=1 check-translation

Global state of the translation is recorded in ‘`Documentation/translations.html.in`’, which is used to generate Translations status page. To update that page, do from ‘`Documentation/`’

>     make translation-status

This will also leave ‘`out/translations-status.txt`’, which contains up-to-dateness percentages for each translated file, and update word counts of documentation files in this Guide.

#### See also

Maintaining without updating translations.

#### Updating documentation translation

Instead of running `check-translation`, you may want to run `update-translation`, which will run your favorite text editor to update files. First, make sure environment variable `EDITOR` is set to a text editor command, then run from ‘`Documentation/`’

>     make ISOLANG=MY_LANGUAGE update-translation

or to update a single file

>     make CHECKED_FILES=MY_LANGUAGE/user/foo.itely update-translation

For each file to be udpated, update-translation will open your text editor with this file and a diff of the file in English; if the diff cannot be generated or is bigger than the file in English itself, the full file in English will be opened instead.

Texinfo skeleton files, i.e. ‘`.itely`’ files not yet translated, containing only the Texinfo structure can be updated automatically: whenever `make check-translation` shows that such files should be updated, run from ‘`Documentation/`’

>     make ISOLANG=MY_LANGUAGE skeleton-update

‘`.po`’ message catalogs in ‘`Documentation/po/`’ may be updated by issuing from ‘`Documentation/`’ or ‘`Documentation/po/`’

>     make po-update

Note: if you run po-update and somebody else does the same and pushes before you push or send a patch to be applied, there will be a conflict when you pull. Therefore, it is better that only the Translation meister runs this command.

Updating music snippets can quickly become cumbersome, as most snippets should be identical in all languages. Fortunately, there is a script that can do this odd job for you (run from ‘`Documentation/`’):

>     make ISOLANG=MY_LANGUAGE snippet-update

This script overwrites music snippets in ‘`MY_LANGUAGE/user/every.itely`’ with music snippets from ‘`user/every.itely`’. It ignores skeleton files, and keeps intact music snippets preceded with a line starting with `@c KEEP LY`; it reports an error for each ‘`.itely`’ that has not the same music snippet count in both languages. Always use this script with a lot of care, i.e. run it on a clean Git working tree, and check the changes it made with `git diff` before committing; if you don’t do so, some `@lilypond` snippets might be broken or make no sense in their context.

Finally, a command runs the three update processes above for all enabled languages (from ‘`Documentation/`’):

>     make all-translations-update

Use this command with caution, and keep in mind it will not be really useful until translations are stabilized after the end of GDP.

#### See also

Maintaining without updating translations.

### 3.7.4 Translations management policies

These policies show the general intent of how the translations should be managed, they aim at helping translators, developers and coordinators work efficiently.

#### Maintaining without updating translations

Keeping translations up to date under heavy changes in the documentation in English may be almost impossible, especially as during the former Grand Documentation Project (GDP) or the Grand Organization Project (GOP) when a lot of contributors brings changes. In addition, transloators may be (and that) involved in these porjects too.

it is possible – and even recommended – to perform some maintaining that keeps translated documentation usable and eases future translation updating. The rationale below the tasks list motivates this plan. The rationale below the tasks list motivates this plan.

The following tasks are listed in decreasing priority order.

1. Update macros.itexi. For each obsolete macro definition, if it is possible to update macro usage in documentation with an automatic text or regexp substitution, do it and delete the macro definition from macros.itexi; otherwise, mark this macro definition as obsolete with a comment, and keep it in macros.itexi until the documentation translation has been updated and no longer uses this macro.
2. Update ‘`*.tely`’ files completely with `make check-translation` – you may want to redirect ouptput to a file because of overwhelming output, or call check-translation.py on individual files, see Check state of translation.
3. In ‘`.itelys`’, match sections and .itely file names with those from English docs, which possibly involves moving nodes contents in block between files, without updating contents itself. In other words, the game is catching where has gone each section. In Learning manual, and in Notation Reference sections which have been revised in GDP, there may be completely new sections: in this case, copy `@node` and `@section`-command from English docs, and add the marker for untranslated status `@untranslated` on a single line. Note that it is not possible to exactly match subsections or subsubsections of documentation in English, when contents has been deeply revised; in this case, keep obsolete (sub)subsections in the translation, marking them with a line `@c obsolete` just before the node.

   Emacs with Texinfo mode makes this step easier:

   - without Emacs AucTeX installed, <C-c C-s> shows structure of current Texinfo file in a new buffer *Occur*; to show structure of two files simultaneously, first split Emacs window in 4 tiles (with <C-x 1> and <C-x 2>), press <C-c C-s> to show structure of one file (e.g. the translated file), copy *Occur* contents into *Scratch*, then press <C-c C-s> for the other file.

     If you happen to have installed AucTeX, you can either call the macro by doing <M-x texinfo-show-structure> or create a key binding in your ‘`~/.emacs`’, by adding the four following lines:

     >     (add-hook 'Texinfo-mode-hook
     >               '(lambda ()
     >                  (define-key Texinfo-mode-map "\C-cs"
     >                   'texinfo-show-structure)))

     and then obtain the structure in the *Occur* buffer with <C-c s>.

   - Do not bother updating `@menu`s when all menu entries are in the same file, just do <C-c C-u C-a> ("update all menus") when you have updated all the rest of the file.
   - Moving to next or previous node using incremental search: press <C-s> and type `node` (or <C-s @node> if the text contains the word ‘node’) then press <C-s> to move to next node or <C-r> to move to previous node. Similar operation can be used to move to the next/previous section. Note that every cursor move exits incremental search, and hitting <C-s> twice starts incremental search with the text entered in previous incremental search.
   - Moving a whole node (or even a sequence of nodes): jump to beginning of the node (quit incremental search by pressing an arrow), press <C-SPACE>, press <C-s node> and repeat <C-s> until you have selected enough text, cut it with <C-w> or <C-x>, jump to the right place (moving between nodes with the previous hint is often useful) and paste with <C-y> or <C-v>.
4. Update sections finished in the English documentation; check sections status at http://lilypondwiki.tuxfamily.org/index.php?title=Documentation_coordination.
5. Update documentation PO. It is recommended not to update strings which come from documentation that is currently deeply revised in English, to avoid doing the work more than once.
6. Fix broken cross-references by running (from ‘`Documentation/`’)

   >     make ISOLANG=YOUR-LANGUAGE fix-xrefs

   This step requires a sucessful documentation build (with `make doc`). Some cross-references are broken because they point to a node that exists in the documentation in English, which has not been added to the translation; in this case, do not fix the cross-reference but keep it "broken", so that the resulting HTML link will point to an existing page of documentation in English.

#### Rationale

You may wonder if it would not be better to leave translations as-is until you can really start updating translations. There are several reasons to do these maintenance tasks right now.

- This will have to be done sooner or later anyway, before updating translation of documentation contents, and this can already be done without needing to be redone later, as sections of documentation in English are mostly revised once. However, note that not all documentation sectioning has been revised in one go, so all this maintenance plan has to be repeated whenever a big reorganization is made.
- This just makes translated documentation take advantage of the new organization, which is better than the old one.
- Moving and renaming sections to match sectioning of documentation in English simplify future updating work: it allows updating the translation by side-by-side comparison, without bothering whether cross-reference names already exist in the translation.
- Each maintenance task except ‘Updating PO files’ can be done by the same person for all languages, which saves overall time spent by translators to achieve this task: the node names and section titles are in English, so you can do. It is important to take advantage of this now, as it will be more complicated (but still possible) to do step 3 in all languages when documentation is compiled with `texi2html` and node names are directly translated in source files.

#### Managing documentation translation with Git

This policy explains how to manage Git branches and commit translations to Git.

- Translation changes matching master branch are preferably made on `lilypond/translation` branch; they may be pushed directly to `master` only if they do not break compilation of LilyPond and its documentation, and in this case they should be pushed to `lilypond/translation` too. Similarly, changes matching `stable/X.Y` are preferably made on `lilypond/X.Ytranslation`.
- `lilypond/translation` Git branch may be merged into master only if LilyPond (`make all`) and documentation (`make doc`) compile succesfully.
- `master` Git branch may be merged into `lilypond/translation` whenever `make` and `make doc` are succesful (in order to ease documentation compilation by translators), or when significant changes had been made in documentation in English in master branch.
- General maintenance may be done by anybody who knows what he does in documentation in all languages, without informing translators first. General maintenance include simple text substitutions (e.g. automated by sed), compilation fixes, updating Texinfo or lilypond-book commands, updating macros, updating ly code, fixing cross-references, and operations described in Maintaining without updating translations.

### 3.7.5 Technical background

A number of Python scripts handle a part of the documentation translation process. All scripts used to maintain the translations are located in ‘`scripts/auxiliar/`’.

- ‘`check_translation.py`’ – show diff to update a translation,
- ‘`texi-langutils.py`’ – quickly and dirtily parse Texinfo files to make message catalogs and Texinfo skeleton files,
- ‘`texi-skeleton-update.py`’ – update Texinfo skeleton files,
- ‘`update-snippets.py`’ – synchronize ly snippets with those from English docs,
- ‘`translations-status.py`’ – update translations status pages and word counts in the file you are reading,
- ‘`tely-gettext.py`’ – gettext node names, section titles and references in the sources; WARNING only use this script once for each file, when support for "makeinfo –html" has been dropped.

Other scripts are used in the build process, in ‘`scripts/build/`’:

- ‘`html-gettext.py`’ – translate node names, section titles and cross references in HTML files generated by `makeinfo`,
- ‘`texi-gettext.py`’ – gettext node names, section titles and references before calling `texi2pdf`,
- ‘`mass-link.py`’ – link or symlink files between English documentation and documentation in other languages.

Python modules used by scripts in ‘`scripts/auxiliar/`’ or ‘`scripts/build/`’ (but not by installed Python scripts) are located in ‘`python/auxiliar/`’:

- ‘`manuals_definitions.py`’ – define manual names and name of cross-reference Texinfo macros,
- ‘`buildlib.py`’ – common functions (read piped output of a shell command, use Git),
- ‘`postprocess_html.py`’ (module imported by ‘`www_post.py`’) – add footer and tweak links in HTML pages.

And finally

- ‘`python/langdefs.py`’ – language definitions module

### 3.7.6 Translation status

#### Contributor’s Guide