このページの内容

Guidedog / ドキュメント 0.2.0

gettext

Package gettext reads, writes, merges, and looks up GNU gettext message catalogs: PO and POT text files, and compiled MO files. Its lookups follow the C library’s gettext, ngettext, and pgettext; state is explicit, a Catalog or a Translator, instead of the C library’s process-wide domain table. It imports only Odin’s core packages.

Types

gettext.Bound_Catalog :: struct

Bound_Catalog is a catalog a translator serves for a domain.

domain: string

catalog: Catalog

gettext.Catalog :: struct

Catalog is one message catalog. Everything it holds lives in its own arena, so destroy frees it at once. Lookups use an index built on first use.

name: string

file name, or the name given to parse_po.

header: Message

the msgid 「」 entry; its strs[0] holds the header fields.

has_header: bool

messages: [dynamic]Message

plural: Plural_Rule

allow_fuzzy: bool

look fuzzy translations up (gettext_allow_fuzzy_translations).

index: Index

arena: ^Catalog_Arena

nil when even it could not be allocated.

allocator: mem.Allocator

the arena’s allocator; everything above lives in it.

owner: mem.Allocator

allocated the arena itself.

gettext.Catalog_Arena :: struct

Catalog_Arena is what a catalog owns: a growing arena on the allocator make_catalog was given. Once refused memory it refuses everything (refused), so a catalog is never built from a mix of what was and was not allocated; catalog_refused tells.

using arena: runtime.Arena

refused: bool

gettext.Error :: struct

Error explains a problem in the style of Elm’s compiler: where, what, and what to do. kind == .None means success. Strings are allocated with the allocator the failing procedure was given; destroy_error frees them.

kind: Error_Kind

file: string

line: int

1-based; 0 when the problem has no line.

column: int

1-based byte column of the problem in excerpt; 0 for none.

excerpt: string

the offending line.

message: string

hint: string

gettext.Error_Kind :: enum u8

None

Syntax

the PO text breaks the format.

Duplicate

a message is defined twice.

Charset

the catalog is not UTF-8.

Encoding

bytes that are not valid UTF-8.

Plural

a Plural-Forms header that cannot be used.

Mo_Format

a truncated or inconsistent MO file.

Io

the file could not be read or written.

Memory

there was not enough memory to hold the catalog.

gettext.Header_Field :: struct

Header_Field is one 「Name: value」 line of the header entry.

name: string

value: string

gettext.Index :: struct

Index is a catalog’s lookup table, built once from its messages. The translations it serves are copied into one pool, each form followed by 「\n」, so every string a lookup returns is a slice of pool: a reader can parse them as one source (see pool_of).

built: bool

fuzzy: bool

built with allow_fuzzy.

slots: []i32

open addressing over entries; -1 is empty.

entries: []Index_Entry

forms: []string

slices of pool.

pool: string

gettext.Index_Entry :: struct

message: i32

in messages, or -1 for the header.

first: i32

first form in forms.

count: i32

gettext.Merge_Options :: struct

fuzzy_matching: bool

look for similar messages (msgmerge without -N).

keep_obsolete: bool

keep translated messages that left the template as 「#~」.

stats: ^Merge_Stats

when set, what the fuzzy search did (fuzzy.odin).

gettext.Merge_Stats :: struct

Merge_Stats counts what a merge’s fuzzy search did; set Merge_Options.stats to read it.

searches: int

msgids searched for (each distinct msgid once).

bounds: int

candidates whose byte counts were compared (similarity_bound).

comparisons: int

candidates compared in full (similarity).

work: int

bytes those comparisons read.

cut_short: int

searches the work bound stopped before they had looked everywhere.

gettext.Message :: struct

Message is one catalog entry. strs holds the msgstr, or msgstr[0..n) of a plural message; an empty string is an untranslated form.

ctx: string

has_ctx: bool

msgctxt 「」 is a context too, different from none.

id: string

id_plural: string

has_plural: bool

strs: [dynamic]string

comments: [dynamic]string

「# 「 translator comments.

notes: [dynamic]string

「#.」 comments for translators, from the extractor.

references: [dynamic]string

「#:」 locations, file:line each.

flags: [dynamic]string

「#,」 flags other than fuzzy, e.g. 「c-format」.

fuzzy: bool

obsolete: bool

「#~」: kept for reuse, never looked up.

previous: Previous

has_previous: bool

line: int

where the entry starts in its file; 0 when built in code.

gettext.Plural_Node :: struct

op: Plural_Op

a: i16

b: i16

c: i16

value: u64

gettext.Plural_Op :: enum u8

Number

N

Not

Mul

Div

Mod

Add

Sub

Less

Greater

Less_Equal

Greater_Equal

Equal

Not_Equal

And

Or

Choose

a ? b : c

gettext.Plural_Problem :: struct

Plural_Problem is where and why an expression was refused; column is 1-based.

message: string

static text.

column: int

gettext.Plural_Rule :: struct

Plural_Rule is a compiled Plural-Forms header: nplurals and the plural expression.

nplurals: int

nodes: [MAX_PLURAL_NODES]Plural_Node

count: int

root: int

gettext.Previous :: struct

Previous holds a fuzzy message’s earlier msgctxt, msgid, and msgid_plural (the 「#|」 comments msgmerge writes), so a translator sees what changed.

ctx: string

has_ctx: bool

id: string

id_plural: string

has_plural: bool

gettext.Reference_Style :: enum u8

Wrapped

「#: a.rst:1 b.rst:2」, wrapped at the width, as msgmerge writes.

One_Per_Line

one 「#:」 line per location, as Sphinx’s gettext builder writes.

None

no locations (gettext_location = False).

gettext.Stats :: struct

translated: int

fuzzy: int

untranslated: int

obsolete: int

gettext.Translator :: struct

Translator serves several text domains, each from a chain of catalogs: the first bound catalog that translates a message wins, as Sphinx searches locale_dirs in order before its own catalogs. It stands in for the C library’s global state (bindtextdomain, textdomain, setlocale).

language: string

domain: string

the current domain (textdomain); 「messages」 by default.

bound: [dynamic]Bound_Catalog

allocator: mem.Allocator

gettext.Write_Options :: struct

width: int

lines are wrapped to this many characters; 0 or less: no wrapping.

references: Reference_Style

previous: bool

write 「#|」 previous msgids of fuzzy messages.

obsolete: bool

write 「#~」 obsolete messages.

Procedures

gettext.add_catalog :: proc(tr: ^Translator, domain: string, cat: Catalog)

add_catalog binds a catalog to a domain after those already bound; the translator owns it from now on.

gettext.add_message :: proc(cat: ^Catalog, m: Message) -> ^Message

add_message appends a copy of m, cloning its strings into the catalog’s arena, and returns the copy, which the catalog owns, or nil when the arena refused the memory. The pointer is valid until the next message is added (the list may then move). The index is rebuilt on the next lookup.

gettext.bind_text_domain :: proc(tr: ^Translator, domain, dir: string, allow_fuzzy := false) -> (found: bool, err: Error)

bind_text_domain looks for dir/<language>/LC_MESSAGES/<domain>.po, then .mo, trying the language, then its base language (pt_BR, then pt), and binds the first catalog found, which the translator then owns. found is false, with no error, when there is none; an error is allocated with the translator’s allocator (destroy_error).

gettext.build_index :: proc(cat: ^Catalog)

build_index (re)builds the lookup table, in the catalog’s arena. Lookups call it when the catalog changed or allow_fuzzy was toggled; call it yourself to pay the cost up front.

gettext.catalog_bytes :: proc(cat: ^Catalog) -> int

catalog_bytes is what cat’s arena holds, in bytes: what a host charges for keeping it.

gettext.catalog_refused :: proc(cat: ^Catalog) -> bool

catalog_refused tells whether cat’s arena was refused memory, so that what it holds may be incomplete. The readers then fail with an error of kind Memory.

gettext.charset :: proc(cat: ^Catalog) -> string

charset returns the charset the header’s Content-Type names, or 「」 without one.

gettext.clone_message :: proc(cat: ^Catalog, m: Message) -> Message

clone_message copies m and its strings into the catalog’s arena.

gettext.compile_plural :: proc(expr: string, nplurals: int) -> (Plural_Rule, Plural_Problem)

compile_plural compiles an expression for nplurals forms.

gettext.default_merge_options :: proc() -> Merge_Options

default_merge_options is msgmerge’s default: fuzzy matching on, obsolete messages kept.

gettext.default_write_options :: proc() -> Write_Options

default_write_options wraps at 76 columns, as sphinx-intl (through Babel) does.

gettext.destroy :: proc(cat: ^Catalog)

destroy frees a catalog and everything it holds.

gettext.destroy_error :: proc(err: Error, allocator := context.allocator)

destroy_error frees the strings of an error; an error of kind Memory has static text, which it leaves alone.

gettext.destroy_translator :: proc(tr: ^Translator)

destroy_translator frees the translator and every catalog bound to it.

gettext.domain_plural :: proc(tr: ^Translator, domain, id, id_plural: string, n: int) -> string

domain_plural is dngettext.

gettext.domain_plural_with_context :: proc(tr: ^Translator, domain: string, ctx: Maybe(string), id, id_plural: string, n: int) -> string

domain_plural_with_context is dnpgettext.

gettext.domain_text :: proc(tr: ^Translator, domain, id: string) -> string

domain_text is dgettext.

gettext.domain_with_context :: proc(tr: ^Translator, domain, ctx, id: string) -> string

domain_with_context is dpgettext.

gettext.escape_string :: proc(s: string, allocator := context.allocator) -> string

escape_string escapes a value for a PO string: backslash, quote, and the control characters C names. The result is allocated with allocator; the caller owns it.

gettext.find :: proc(cat: ^Catalog, id: string, ctx: Maybe(string) = nil) -> ([]string, bool)

find returns the translated forms of a message, slices of the index pool. ctx nil means no context. A fuzzy message is found only when allow_fuzzy is set.

gettext.find_catalog_file :: proc(dir, language, domain: string, allocator := context.allocator) -> (string, bool)

find_catalog_file returns the path of a domain’s catalog for a language, if any, allocated with allocator; the caller owns it.

gettext.format_error :: proc(err: Error, allocator := context.allocator) -> string

format_error renders an error as a friendly report, in a string allocated with allocator that the caller owns (「」 for no error):

-- PO SYNTAX ERROR ----------------------------------------- de/index.po:12

This string has no closing quote.

12| msgstr "Hallo
           ^
Hint: End the string with " on the same line; continue on the next line.

gettext.format_stats :: proc(s: Stats, allocator := context.allocator) -> string

format_stats writes sphinx-intl’s summary line, allocated with allocator; the caller owns it.

gettext.header_field :: proc(cat: ^Catalog, name: string) -> (string, bool)

header_field returns the value of a header field, compared case-insensitively.

gettext.language_candidates :: proc(language: string, allocator := context.allocator) -> []string

language_candidates lists the catalog directory names to try for a language, most specific first: 「pt-BR」 gives pt-BR, pt_BR, and pt; 「sr@latin」 gives it, then sr. The list and each of its strings are allocated with allocator, apart from language and from each other; the caller owns them. A candidate whose copy is refused is left out.

gettext.load_mo :: proc(path: string, allocator := context.allocator) -> (Catalog, Error)

load_mo reads a compiled catalog. Its messages have no comments or references. The catalog owns its arena, made with allocator (destroy); an error is allocated with allocator too (destroy_error).

gettext.load_po :: proc(path: string, allow_fuzzy := false, allocator := context.allocator) -> (Catalog, Error)

load_po reads a PO or POT file. Errors are allocated with allocator.

gettext.make_catalog :: proc(name := "", allocator := context.allocator) -> Catalog

make_catalog returns an empty catalog with a germanic plural rule (n != 1). The catalog owns an arena, made with allocator, that holds everything added to it; destroy frees it whole.

gettext.make_translator :: proc(language: string, allocator := context.allocator) -> Translator

make_translator returns a translator for language with no catalogs bound, in the 「messages」 domain. It allocates with allocator; destroy_translator frees it and the catalogs bound to it.

gettext.merge :: proc(def, ref: ^Catalog, options := Merge_Options{fuzzy_matching = true, keep_obsolete = true}, allocator := context.allocator) -> Catalog

merge updates a translation (def) to a new template (ref), as msgmerge and sphinx-intl update do. The result has the template’s messages in the template’s order, with its locations, notes, and flags:

• a message found in def keeps its translation, translator comments, and fuzzy flag;
• a new message similar to an old translated one gets that translation, marked fuzzy, with the old msgid kept as its previous msgid (「#|」);
• old translated messages the template lost become obsolete (「#~」).

The header is def’s, with the template’s POT-Creation-Date. The result is a new catalog that owns its arena, made with allocator (destroy); def and ref are borrowed.

gettext.new_message :: proc(cat: ^Catalog) -> Message

new_message returns an empty message whose lists use the catalog’s arena.

gettext.parse_mo :: proc(data: []u8, name := "<mo>", allocator := context.allocator) -> (Catalog, Error)

parse_mo decodes MO bytes into a catalog that copies what it keeps into its own arena, made with allocator (destroy), so data may be freed afterwards. Every offset is checked, so a damaged file gives an error, allocated with allocator (destroy_error).

gettext.parse_plural_forms :: proc(value: string, allocator := context.allocator) -> (Plural_Rule, Error)

parse_plural_forms compiles a Plural-Forms header value such as 「nplurals=2; plural=(n != 1);」. The rule is a value; an error, allocated with allocator (destroy_error), carries the value as its excerpt.

gettext.parse_po :: proc(text: string, name := "<string>", allow_fuzzy := false, allocator := context.allocator) -> (Catalog, Error)

parse_po parses PO or POT text. name labels errors and the catalog. The catalog copies what it keeps, so text may be freed afterwards.

gettext.plural_forms_for :: proc(language: string) -> string

plural_forms_for returns a Plural-Forms header value for a language, for a new catalog: the rules the GNU gettext manual lists for common languages, and the germanic rule (n != 1) for others. 「pt_BR」 and 「pt-BR」 find 「pt_BR」, then 「pt」. The value is static; the candidates are tried in the temp allocator.

gettext.plural_index :: proc(rule: ^Plural_Rule, n: u64) -> int

plural_index returns the msgstr index for n; out-of-range results select form 0, as the C library does.

gettext.pool_of :: proc(cat: ^Catalog) -> string

pool_of returns the text every translation of the catalog is a slice of, so a caller can register it once as a source and give parsed translations exact spans.

gettext.save_po :: proc(cat: ^Catalog, path: string, options := Write_Options{width = 76, previous = true, obsolete = true}, allocator := context.allocator) -> Error

save_po writes a catalog to a file, formatting it in the temp allocator; an error is allocated with allocator (destroy_error).

gettext.set_header_field :: proc(cat: ^Catalog, name, value: string)

set_header_field replaces a header field, or appends it, keeping the others in order; the catalog’s arena holds the copy.

gettext.similarity :: proc(a, b: string) -> f64

similarity is 2·M / (|a| + |b|), where M is the length of the longest common subsequence of bytes: 1 for equal strings, 0 for strings with nothing in common. It is the measure msgmerge’s fstrcmp approximates. Long strings are compared by their first FUZZY_PREFIX bytes.

gettext.sort_by_reference :: proc(cat: ^Catalog)

sort_by_reference orders messages by their first location: file name, then line number as a number, then msgid; messages without locations go last. Obsolete messages keep their place after the live ones. Each message’s own locations are sorted the same way, as Sphinx sorts them.

gettext.stats :: proc(cat: ^Catalog) -> Stats

stats counts messages as sphinx-intl stat does; the header does not count.

gettext.text_domain :: proc(tr: ^Translator, domain: string)

text_domain sets the current domain, as textdomain does; the translator owns its copy.

gettext.translated :: proc(m: Message) -> bool

translated reports whether every form of m has a translation.

gettext.translation :: proc(cat: ^Catalog, id: string, ctx: Maybe(string) = nil) -> (string, bool)

translation returns a message’s singular translation when there is one.

gettext.write_mo :: proc(cat: ^Catalog, include_fuzzy := false, allocator := context.allocator) -> []u8

write_mo compiles a catalog as msgfmt does: the header and every translated message, sorted by key, with a hash table. Fuzzy messages are left out unless include_fuzzy. The bytes are allocated with allocator; the caller owns them. They are nil when the memory was refused: a compiled catalog is never shorter than its 28-byte header.

gettext.write_po :: proc(cat: ^Catalog, options := Write_Options{width = 76, previous = true, obsolete = true}, allocator := context.allocator) -> string

write_po writes a catalog as PO text: the header, then the messages in order, then the obsolete ones, each entry laid out as msgmerge lays it out. The output depends on nothing but the catalog and the options, so files diff cleanly. The text is allocated with allocator; the caller owns it.

Procedure groups

gettext.plural :: proc{catalog_plural, translator_plural}

plural is ngettext.

gettext.plural_with_context :: proc{catalog_plural_with_context, translator_plural_with_context}

plural_with_context is npgettext.

gettext.text :: proc{catalog_text, translator_text}

text is gettext, on a catalog or on a translator’s current domain.

gettext.with_context :: proc{catalog_with_context, translator_with_context}

with_context is pgettext.

Constants

gettext.FUZZY_MIN_WORK :: 1 << 26

gettext.FUZZY_THRESHOLD :: 0.6

FUZZY_THRESHOLD is msgmerge’s: a changed message inherits the translation of the most similar old message when their similarity is at least this.

gettext.FUZZY_WORK_PER_BYTE :: 256

Variables

@(rodata) gettext.ERROR_TITLES