On this page

Guidedog / Documentation 0.2.0

gettext¶

Package gettext reads, writes, merges, and looks up GNU gettext message catalogs: PO and POT text files, and compiled MO files. Its lookups follow the C library’s gettext, ngettext, and pgettext; state is explicit, a Catalog or a Translator, instead of the C library’s process-wide domain table. It imports only Odin’s core packages.

Types

gettext.Bound_Catalog :: struct¶

Bound_Catalog is a catalog a translator serves for a domain.

domain: string¶

catalog: Catalog ¶

gettext.Catalog :: struct¶

Catalog is one message catalog. Everything it holds lives in its own arena, so destroy frees it at once. Lookups use an index built on first use.

name: string¶: file name, or the name given to parse_po.

header: Message ¶: the msgid “” entry; its strs[0] holds the header fields.

has_header: bool¶

messages: [dynamic]Message ¶

plural: Plural_Rule ¶

allow_fuzzy: bool¶: look fuzzy translations up (gettext_allow_fuzzy_translations).

index: Index ¶

arena: ^Catalog_Arena ¶: nil when even it could not be allocated.

allocator: mem.Allocator¶: the arena’s allocator; everything above lives in it.

owner: mem.Allocator¶: allocated the arena itself.

gettext.Catalog_Arena :: struct¶

Catalog_Arena is what a catalog owns: a growing arena on the allocator make_catalog was given. Once refused memory it refuses everything (refused), so a catalog is never built from a mix of what was and was not allocated; catalog_refused tells.

using arena: runtime.Arena¶

refused: bool¶

gettext.Error :: struct¶

Error explains a problem in the style of Elm’s compiler: where, what, and what to do. kind == .None means success. Strings are allocated with the allocator the failing procedure was given; destroy_error frees them.

kind: Error_Kind ¶

file: string¶

line: int¶: 1-based; 0 when the problem has no line.

column: int¶: 1-based byte column of the problem in excerpt; 0 for none.

excerpt: string¶: the offending line.

message: string¶

hint: string¶

gettext.Error_Kind :: enum u8¶

None¶

Syntax¶: the PO text breaks the format.

Duplicate¶: a message is defined twice.

Charset¶: the catalog is not UTF-8.

Encoding¶: bytes that are not valid UTF-8.

Plural¶: a Plural-Forms header that cannot be used.

Mo_Format¶: a truncated or inconsistent MO file.

Io¶: the file could not be read or written.

Memory¶: there was not enough memory to hold the catalog.

gettext.Header_Field :: struct¶

Header_Field is one “Name: value” line of the header entry.

name: string¶

value: string¶

gettext.Index :: struct¶

Index is a catalog’s lookup table, built once from its messages. The translations it serves are copied into one pool, each form followed by “\n”, so every string a lookup returns is a slice of pool: a reader can parse them as one source (see pool_of).

built: bool¶

fuzzy: bool¶: built with allow_fuzzy.

slots: []i32¶: open addressing over entries; -1 is empty.

entries: []Index_Entry ¶

forms: []string¶: slices of pool.

pool: string¶

gettext.Index_Entry :: struct¶

message: i32¶: in messages, or -1 for the header.

first: i32¶: first form in forms.

count: i32¶

gettext.Merge_Options :: struct¶

fuzzy_matching: bool¶: look for similar messages (msgmerge without -N).

keep_obsolete: bool¶: keep translated messages that left the template as “#~”.

stats: ^Merge_Stats ¶: when set, what the fuzzy search did (fuzzy.odin).

gettext.Merge_Stats :: struct¶

Merge_Stats counts what a merge’s fuzzy search did; set Merge_Options.stats to read it.

searches: int¶: msgids searched for (each distinct msgid once).

bounds: int¶: candidates whose byte counts were compared (similarity_bound).

comparisons: int¶: candidates compared in full (similarity).

work: int¶: bytes those comparisons read.

cut_short: int¶: searches the work bound stopped before they had looked everywhere.

gettext.Message :: struct¶

Message is one catalog entry. strs holds the msgstr, or msgstr[0..n) of a plural message; an empty string is an untranslated form.

ctx: string¶

has_ctx: bool¶: msgctxt “” is a context too, different from none.

id: string¶

id_plural: string¶

has_plural: bool¶

strs: [dynamic]string¶

comments: [dynamic]string¶: “# “ translator comments.

notes: [dynamic]string¶: “#.” comments for translators, from the extractor.

references: [dynamic]string¶: “#:” locations, file:line each.

flags: [dynamic]string¶: “#,” flags other than fuzzy, e.g. “c-format”.

fuzzy: bool¶

obsolete: bool¶: “#~”: kept for reuse, never looked up.

previous: Previous ¶

has_previous: bool¶

line: int¶: where the entry starts in its file; 0 when built in code.

gettext.Plural_Node :: struct¶

op: Plural_Op ¶

a: i16¶

b: i16¶

c: i16¶

value: u64¶

gettext.Plural_Op :: enum u8¶

Number¶

N¶

Not¶

Mul¶

Div¶

Mod¶

Add¶

Sub¶

Less¶

Greater¶

Less_Equal¶

Greater_Equal¶

Equal¶

Not_Equal¶

And¶

Or¶

Choose¶: a ? b : c

gettext.Plural_Problem :: struct¶

Plural_Problem is where and why an expression was refused; column is 1-based.

message: string¶: static text.

column: int¶

gettext.Plural_Rule :: struct¶

Plural_Rule is a compiled Plural-Forms header: nplurals and the plural expression.

nplurals: int¶

nodes: [MAX_PLURAL_NODES]Plural_Node ¶

count: int¶

root: int¶

gettext.Previous :: struct¶

Previous holds a fuzzy message’s earlier msgctxt, msgid, and msgid_plural (the “#|” comments msgmerge writes), so a translator sees what changed.

ctx: string¶

has_ctx: bool¶

id: string¶

id_plural: string¶

has_plural: bool¶

gettext.Reference_Style :: enum u8¶

Wrapped¶: “#: a.rst:1 b.rst:2”, wrapped at the width, as msgmerge writes.

One_Per_Line¶: one “#:” line per location, as Sphinx’s gettext builder writes.

None¶: no locations (gettext_location = False).

gettext.Stats :: struct¶

translated: int¶

fuzzy: int¶

untranslated: int¶

obsolete: int¶

gettext.Translator :: struct¶

Translator serves several text domains, each from a chain of catalogs: the first bound catalog that translates a message wins, as Sphinx searches locale_dirs in order before its own catalogs. It stands in for the C library’s global state (bindtextdomain, textdomain, setlocale).

language: string¶

domain: string¶: the current domain (textdomain); “messages” by default.

bound: [dynamic]Bound_Catalog ¶

allocator: mem.Allocator¶

gettext.Write_Options :: struct¶

width: int¶: lines are wrapped to this many characters; 0 or less: no wrapping.

references: Reference_Style ¶

previous: bool¶: write “#|” previous msgids of fuzzy messages.

obsolete: bool¶: write “#~” obsolete messages.

Procedures

gettext.add_catalog :: proc(tr: ^Translator, domain: string, cat: Catalog)¶: add_catalog binds a catalog to a domain after those already bound; the translator owns it from now on.

gettext.add_message :: proc(cat: ^Catalog, m: Message) -> ^Message ¶: add_message appends a copy of m, cloning its strings into the catalog’s arena, and returns the copy, which the catalog owns, or nil when the arena refused the memory. The pointer is valid until the next message is added (the list may then move). The index is rebuilt on the next lookup.

gettext.bind_text_domain :: proc(tr: ^Translator, domain, dir: string, allow_fuzzy := false) -> (found: bool, err: Error)¶: bind_text_domain looks for dir/<language>/LC_MESSAGES/<domain>.po, then .mo, trying the language, then its base language (pt_BR, then pt), and binds the first catalog found, which the translator then owns. found is false, with no error, when there is none; an error is allocated with the translator’s allocator (destroy_error).

gettext.build_index :: proc(cat: ^Catalog)¶: build_index (re)builds the lookup table, in the catalog’s arena. Lookups call it when the catalog changed or allow_fuzzy was toggled; call it yourself to pay the cost up front.

gettext.catalog_bytes :: proc(cat: ^Catalog) -> int¶: catalog_bytes is what cat’s arena holds, in bytes: what a host charges for keeping it.

gettext.catalog_refused :: proc(cat: ^Catalog) -> bool¶: catalog_refused tells whether cat’s arena was refused memory, so that what it holds may be incomplete. The readers then fail with an error of kind Memory.

gettext.charset :: proc(cat: ^Catalog) -> string¶: charset returns the charset the header’s Content-Type names, or “” without one.

gettext.clone_message :: proc(cat: ^Catalog, m: Message) -> Message ¶: clone_message copies m and its strings into the catalog’s arena.

gettext.compile_plural :: proc(expr: string, nplurals: int) -> (Plural_Rule, Plural_Problem)¶: compile_plural compiles an expression for nplurals forms.

gettext.default_merge_options :: proc() -> Merge_Options ¶: default_merge_options is msgmerge’s default: fuzzy matching on, obsolete messages kept.

gettext.default_write_options :: proc() -> Write_Options ¶: default_write_options wraps at 76 columns, as sphinx-intl (through Babel) does.

gettext.destroy :: proc(cat: ^Catalog)¶: destroy frees a catalog and everything it holds.

gettext.destroy_error :: proc(err: Error, allocator := context.allocator)¶: destroy_error frees the strings of an error; an error of kind Memory has static text, which it leaves alone.

gettext.destroy_translator :: proc(tr: ^Translator)¶: destroy_translator frees the translator and every catalog bound to it.

gettext.domain_plural :: proc(tr: ^Translator, domain, id, id_plural: string, n: int) -> string¶: domain_plural is dngettext.

gettext.domain_plural_with_context :: proc(tr: ^Translator, domain: string, ctx: Maybe(string), id, id_plural: string, n: int) -> string¶: domain_plural_with_context is dnpgettext.

gettext.domain_text :: proc(tr: ^Translator, domain, id: string) -> string¶: domain_text is dgettext.

gettext.domain_with_context :: proc(tr: ^Translator, domain, ctx, id: string) -> string¶: domain_with_context is dpgettext.

gettext.escape_string :: proc(s: string, allocator := context.allocator) -> string¶: escape_string escapes a value for a PO string: backslash, quote, and the control characters C names. The result is allocated with allocator; the caller owns it.

gettext.find :: proc(cat: ^Catalog, id: string, ctx: Maybe(string) = nil) -> ([]string, bool)¶: find returns the translated forms of a message, slices of the index pool. ctx nil means no context. A fuzzy message is found only when allow_fuzzy is set.

gettext.find_catalog_file :: proc(dir, language, domain: string, allocator := context.allocator) -> (string, bool)¶: find_catalog_file returns the path of a domain’s catalog for a language, if any, allocated with allocator; the caller owns it.

gettext.format_error :: proc(err: Error, allocator := context.allocator) -> string¶

format_error renders an error as a friendly report, in a string allocated with allocator that the caller owns (“” for no error):

-- PO SYNTAX ERROR ----------------------------------------- de/index.po:12

This string has no closing quote.

12| msgstr "Hallo
           ^
Hint: End the string with " on the same line; continue on the next line.

gettext.format_stats :: proc(s: Stats, allocator := context.allocator) -> string¶: format_stats writes sphinx-intl’s summary line, allocated with allocator; the caller owns it.

gettext.header_field :: proc(cat: ^Catalog, name: string) -> (string, bool)¶: header_field returns the value of a header field, compared case-insensitively.

gettext.language_candidates :: proc(language: string, allocator := context.allocator) -> []string¶: language_candidates lists the catalog directory names to try for a language, most specific first: “pt-BR” gives pt-BR, pt_BR, and pt; “sr@latin” gives it, then sr. The list and each of its strings are allocated with allocator, apart from language and from each other; the caller owns them. A candidate whose copy is refused is left out.

gettext.load_mo :: proc(path: string, allocator := context.allocator) -> (Catalog, Error)¶: load_mo reads a compiled catalog. Its messages have no comments or references. The catalog owns its arena, made with allocator (destroy); an error is allocated with allocator too (destroy_error).

gettext.load_po :: proc(path: string, allow_fuzzy := false, allocator := context.allocator) -> (Catalog, Error)¶: load_po reads a PO or POT file. Errors are allocated with allocator.

gettext.make_catalog :: proc(name := "", allocator := context.allocator) -> Catalog ¶: make_catalog returns an empty catalog with a germanic plural rule (n != 1). The catalog owns an arena, made with allocator, that holds everything added to it; destroy frees it whole.

gettext.make_translator :: proc(language: string, allocator := context.allocator) -> Translator ¶: make_translator returns a translator for language with no catalogs bound, in the “messages” domain. It allocates with allocator; destroy_translator frees it and the catalogs bound to it.

gettext.merge :: proc(def, ref: ^Catalog, options := Merge_Options{fuzzy_matching = true, keep_obsolete = true}, allocator := context.allocator) -> Catalog ¶

merge updates a translation (def) to a new template (ref), as msgmerge and sphinx-intl update do. The result has the template’s messages in the template’s order, with its locations, notes, and flags:

a message found in def keeps its translation, translator comments, and fuzzy flag;
a new message similar to an old translated one gets that translation, marked fuzzy, with the old msgid kept as its previous msgid (“#|”);
old translated messages the template lost become obsolete (“#~”).

The header is def’s, with the template’s POT-Creation-Date. The result is a new catalog that owns its arena, made with allocator (destroy); def and ref are borrowed.

gettext.new_message :: proc(cat: ^Catalog) -> Message ¶: new_message returns an empty message whose lists use the catalog’s arena.

gettext.parse_mo :: proc(data: []u8, name := "<mo>", allocator := context.allocator) -> (Catalog, Error)¶: parse_mo decodes MO bytes into a catalog that copies what it keeps into its own arena, made with allocator (destroy), so data may be freed afterwards. Every offset is checked, so a damaged file gives an error, allocated with allocator (destroy_error).

gettext.parse_plural_forms :: proc(value: string, allocator := context.allocator) -> (Plural_Rule, Error)¶: parse_plural_forms compiles a Plural-Forms header value such as “nplurals=2; plural=(n != 1);”. The rule is a value; an error, allocated with allocator (destroy_error), carries the value as its excerpt.

gettext.parse_po :: proc(text: string, name := "<string>", allow_fuzzy := false, allocator := context.allocator) -> (Catalog, Error)¶: parse_po parses PO or POT text. name labels errors and the catalog. The catalog copies what it keeps, so text may be freed afterwards.

gettext.plural_forms_for :: proc(language: string) -> string¶: plural_forms_for returns a Plural-Forms header value for a language, for a new catalog: the rules the GNU gettext manual lists for common languages, and the germanic rule (n != 1) for others. “pt_BR” and “pt-BR” find “pt_BR”, then “pt”. The value is static; the candidates are tried in the temp allocator.

gettext.plural_index :: proc(rule: ^Plural_Rule, n: u64) -> int¶: plural_index returns the msgstr index for n; out-of-range results select form 0, as the C library does.

gettext.pool_of :: proc(cat: ^Catalog) -> string¶: pool_of returns the text every translation of the catalog is a slice of, so a caller can register it once as a source and give parsed translations exact spans.

gettext.save_po :: proc(cat: ^Catalog, path: string, options := Write_Options{width = 76, previous = true, obsolete = true}, allocator := context.allocator) -> Error ¶: save_po writes a catalog to a file, formatting it in the temp allocator; an error is allocated with allocator (destroy_error).

gettext.set_header_field :: proc(cat: ^Catalog, name, value: string)¶: set_header_field replaces a header field, or appends it, keeping the others in order; the catalog’s arena holds the copy.

gettext.similarity :: proc(a, b: string) -> f64¶: similarity is 2·M / (|a| + |b|), where M is the length of the longest common subsequence of bytes: 1 for equal strings, 0 for strings with nothing in common. It is the measure msgmerge’s fstrcmp approximates. Long strings are compared by their first FUZZY_PREFIX bytes.

gettext.sort_by_reference :: proc(cat: ^Catalog)¶: sort_by_reference orders messages by their first location: file name, then line number as a number, then msgid; messages without locations go last. Obsolete messages keep their place after the live ones. Each message’s own locations are sorted the same way, as Sphinx sorts them.

gettext.stats :: proc(cat: ^Catalog) -> Stats ¶: stats counts messages as sphinx-intl stat does; the header does not count.

gettext.text_domain :: proc(tr: ^Translator, domain: string)¶: text_domain sets the current domain, as textdomain does; the translator owns its copy.

gettext.translated :: proc(m: Message) -> bool¶: translated reports whether every form of m has a translation.

gettext.translation :: proc(cat: ^Catalog, id: string, ctx: Maybe(string) = nil) -> (string, bool)¶: translation returns a message’s singular translation when there is one.

gettext.write_mo :: proc(cat: ^Catalog, include_fuzzy := false, allocator := context.allocator) -> []u8¶: write_mo compiles a catalog as msgfmt does: the header and every translated message, sorted by key, with a hash table. Fuzzy messages are left out unless include_fuzzy. The bytes are allocated with allocator; the caller owns them. They are nil when the memory was refused: a compiled catalog is never shorter than its 28-byte header.

gettext.write_po :: proc(cat: ^Catalog, options := Write_Options{width = 76, previous = true, obsolete = true}, allocator := context.allocator) -> string¶: write_po writes a catalog as PO text: the header, then the messages in order, then the obsolete ones, each entry laid out as msgmerge lays it out. The output depends on nothing but the catalog and the options, so files diff cleanly. The text is allocated with allocator; the caller owns it.

Procedure groups

gettext.plural :: proc{catalog_plural, translator_plural}¶: plural is ngettext.

gettext.plural_with_context :: proc{catalog_plural_with_context, translator_plural_with_context}¶: plural_with_context is npgettext.

gettext.text :: proc{catalog_text, translator_text}¶: text is gettext, on a catalog or on a translator’s current domain.

gettext.with_context :: proc{catalog_with_context, translator_with_context}¶: with_context is pgettext.

Constants

gettext.FUZZY_MIN_WORK :: 1 << 26¶

gettext.FUZZY_THRESHOLD :: 0.6¶: FUZZY_THRESHOLD is msgmerge’s: a changed message inherits the translation of the most similar old message when their similarity is at least this.

gettext.FUZZY_WORK_PER_BYTE :: 256¶

Variables

@(rodata) gettext.ERROR_TITLES¶