On this page

Guidedog / Documentation 0.2.0

host

Package host composes the allocation-free core with memory, files, and retries. Everything here may allocate; the core’s zero-allocation contract ends at gd calls.

Ownership, unless a procedure says otherwise: what a procedure returns (a string, a slice, a report, a problem’s text) is allocated in context.allocator and belongs to the caller, which frees it with that allocator’s arena rather than piece by piece; its arguments are borrowed for the call only. A failure is a Problem (exit != 0) or a false result, never a panic. The surface is listed in lib/host/README.md.

Types

host.Arena :: struct

Arena hands out memory from blocks it takes from a backing allocator (a Meter, usually) and frees them all at once. It is core:mem’s Dynamic_Arena made safe for a backing that refuses: a block the backing refuses leaves the arena as it was, and the request fails with .Out_Of_Memory, where Dynamic_Arena would go on handing out memory at address 0. The blocks are linked through a header at their start, so the arena allocates nothing besides them; each block is used through bounds-checked slices of itself. It is not safe for use by several threads at once.

backing: mem.Allocator

block: int

bytes of a block; 0: ARENA_BLOCK_BYTES.

blocks: ^Arena_Block

the newest first; the first is the one being filled.

last: []byte

the latest allocation from the first block, which can grow.

bytes: int

the bytes of every block held.

host.Artifact_Request :: struct

Artifact_Request is one conversion of guidedog convert. Its strings and slices are borrowed for the call.

input: string

path, or “-“ for standard input.

from: Format

to: Target

output: string

path, or “-“ for standard output.

emit_typst: string

optional path for the intermediate Typst source.

force: bool

policy: gd.Policy

limits: gd.Limits

read: gd.Read_Options

render: gd.Render_Options

typst_program: string

Typst program name or path.

typst_options: typst.Options

packages and fonts.

roots: []string

approved resource roots; the input’s directory by default.

budget: int

allocator: mem.Allocator

session regions; unset: context.allocator. Must reclaim frees.

host.Artifact_Result :: struct

Artifact_Result is what convert_artifact did: exit 0 and the published path (or the bytes for standard output), or exit != 0 and the problem; reports either way. All of it is allocated in the caller’s context.allocator.

exit: int

reports: []Report

problem: Problem

written: string

the published path, or “” for standard output.

stdout: []u8

bytes to print when output is “-“.

host.Budget :: struct

Budget bounds the memory sessions hold at once: one session’s alone, or, shared, the total of a parallel build’s threads. A session reserves what it will allocate before allocating it, and releases it when it frees it.

Units of work that run at once (the threads of a parallel read) join the budget while they run. A reservation the budget cannot hold now waits while another joined unit is still running, since that one will release what it holds when it finishes; when every other unit is waiting too, nothing would be released, and it is refused. So a reservation never waits for itself, and waiting units never wait for each other: a build whose units together need more than the budget refuses some of them.

A reservation the budget cannot hold, and no unit will make room for, goes to the budget’s policy (Budget_Policy): it is refused, unless the policy raises the limit or spills working memory to disk.

limit: int

used: int

active: int

units joined: see budget_join.

waiting: int

joined units waiting in reserve.

peak: int

the most used at once.

mutex: sync.Mutex

freed: sync.Cond

policy: Budget_Policy

stopped: bool

the policy chose to stop: later refusals do not ask again.

refused_used: int

What the budget held, and was asked for, when it last refused (budget_hint).

refused_need: int

host.Budget_Policy :: struct

Budget_Policy is what a budget does when it cannot hold a reservation (Budget). With no on_refusal, and spill not set, the reservation is refused: the step that asked fails with host.budget. on_refusal can raise the limit, or turn on spilling; spill set from the start spills without asking. Spilling sends working memory past the limit (sessions’ storage, and what Meters allocate) to disk: slower, but bounded by the Disk’s own limit and free space. Other reservations, such as a file a build keeps in memory, are still refused.

on_refusal: On_Refusal

user: rawptr

disk: ^Disk

borrowed; the owner destroys it after the build is released.

spill: bool

locate: proc(user: rawptr) -> Report

locate says where the work that asked for memory is, for Refusal.place; it is called, with locate_user, from inside the allocator that asked.

locate_user: rawptr

host.Confinement :: enum u8

Confinement is where a path lies with respect to the roots a project reads from.

Inside

confined: below a root once every link is followed.

Outside

the path as written is below no root.

Through_Link

written below a root, but a link on the way leads out of the roots.

host.Disk :: struct

Disk is working memory on disk: memory-mapped temporary files, used when a build’s budget is exhausted and the build may go on slowly rather than stop (Budget_Policy). Each file is written out in full before it is mapped, so the disk space is taken before the memory is handed out and writing to it never meets a full disk. On Linux, macOS, and the BSDs a file is removed as soon as it is mapped, so the system reclaims it however the process ends, a crash or Ctrl+C included; on Windows it is created to be deleted when closed.

A Disk hands out memory from its files, as an arena does from its blocks, and counts the allocations each file holds: a file is unmapped when its last allocation is freed, or by disk_destroy, and nothing reads it after. It is safe for use by several threads at once. Its zero value, with dir set, is ready to use; the owner calls disk_destroy once nothing allocated from it is used any more.

dir: string

the folder of the temporary files; “” for the system’s.

limit: int

the most bytes mapped at once; 0: the free space, less DISK_MARGIN.

used: int

bytes mapped now.

peak: int

the most bytes mapped at once.

refused: int

bytes a refused request asked for, or 0.

failure: Disk_Failure

why the latest request was refused.

chunks: ^Disk_Chunk

the newest first; the first is the one being filled.

mutex: sync.Mutex

host.Disk_Failure :: enum u8

Disk_Failure is why a Disk refused a request.

None

Limit

the disk budget (Disk.limit) cannot hold it.

Space

the disk lacks the free space, less DISK_MARGIN.

System

the system could not create, write, or map the file.

host.Files :: struct

Files provides included resources from approved roots. It refuses Docutils’ “<standard>” includes, names that leave the roots, and names that pass through a symbolic link. Links are checked on the path as written, component by component with lstat, before it is resolved; then the resolved path must still lie in a root. The rule is therefore the same on Windows, macOS, and Linux.

roots: []string

max_bytes: int

project: string

When set, a name starting with / is relative to this directory, as Sphinx reads /path in include and literalinclude; otherwise such a name is absolute.

canonical: []string

the roots, resolved; filled by files_provider.

ledger: ^Ledger

When set, each file’s size is charged here before it is read (see read_file); a file the budget cannot hold is refused. The caller settles it when the files go.

host.Folder_Entry :: struct

Folder_Entry is a name in a folder, what it names (a symbolic link is .Symlink, whatever it leads to), and its inode, as the listing gives it: two names with the same inode on the same file system are the same file. inode is 0 where the system gives none.

name: string

type: os.File_Type

inode: u64

host.Format :: enum u8

Format is the markup convert_artifact reads.

Rst

Commonmark

Typst

host.Guard :: struct

Guard is an allocator that watches another for refusals. Guidedog’s libraries do not all stop at a failed allocation: they go on with what they have, never touching memory they were refused (tests/allocation_failure_test.odin holds them to that), so their result may be incomplete. A caller that must know its work is whole allocates through a Guard and asks guard_problem when the work ends.

The first refusal trips the guard, and from then on it refuses every allocation itself: memory that ran out once is not relied on again, so the work that follows only shrinks, and nothing is built from a mix of what was and was not allocated. Frees still reach the backing allocator. A guard may be shared by threads.

backing: mem.Allocator

tripped: i32

atomic: 1 once an allocation was refused.

refused: int

atomic: bytes the first refused allocation asked for.

host.Identity :: struct

Identity is a file’s or a folder’s device and inode, which a rename keeps. The zero value is none: the path does not exist, or the system gives no inode (Windows may not).

device: u64

inode: u64

host.Ledger :: struct

Ledger counts what one unit of work (reading a document, rendering a page, composing a book) holds of a budget besides its sessions: the files it reads, the graphs it draws. Each charge is made before the memory is allocated, and settle releases them all when the unit frees its memory. A ledger without a budget counts nothing.

budget: ^Budget

held: int

refused: int

bytes a refused charge asked for, or 0.

host.Meter :: struct

Meter is an allocator that charges a ledger for each allocation before passing it to its backing allocator. It is for work whose memory is not known in advance, such as rendering a page’s layout or parsing a saved environment. A charge the budget refuses is refused before anything is allocated: the allocation fails with .Out_Of_Memory, the ledger records the refused bytes (ledger.refused), and the step that asked stops with metered_refusal’s problem. Code behind a meter must therefore handle a failed allocation; see “Memory” in lib/host/README.md for what the build’s steps do.

When the budget spills (Budget_Policy), an allocation it cannot hold comes from the policy’s Disk instead, and is freed there. Frees of memory from the backing allocator are not discharged one by one (an arena’s allocator is told no sizes); settle returns the step’s charges when it frees its memory.

prepaid holds bytes charged ahead (meter_prepay) that allocations use before charging anew.

backing: mem.Allocator

ledger: ^Ledger

prepaid: int

host.On_Refusal :: proc(user: rawptr, r: Refusal) -> Refusal_Decision

On_Refusal decides what a budget does with a reservation it cannot hold. It is called at most once at a time, while the budget’s other reservations wait; it must not reserve from the budget itself.

host.Problem :: struct

Problem is a host failure: files, the Typst backend, or command usage.

code: string

title: string

path: string

message: string

hint: string

exit: int

host.Refusal :: struct

Refusal is what a budget tells its policy when it cannot hold a reservation: the bytes asked for, what the budget holds, and its limit; and where the work that asked is, as the policy’s locate says (a report’s path, line, and excerpt; zero when unknown), so that a question names the place a report would.

need: int

used: int

limit: int

disk: bool

the policy has a Disk to go on with.

place: Report

host.Refusal_Choice :: enum u8

Refusal_Choice is what a build does when its budget cannot hold a reservation: stop with the budget’s problem, raise the budget, or go on with working memory on disk.

Stop

Raise

to Refusal_Decision.limit.

Disk

host.Refusal_Decision :: struct

choice: Refusal_Choice

limit: int

Raise: the new limit in bytes, more than Refusal.limit.

host.Report :: struct

Report is a diagnostic with its template filled in and its position resolved.

schema: int

code: string

severity: string

title: string

message: string

hint: string

path: string

line: int

column: int

end_line: int

end_column: int

excerpt: string `json:"-"`

marker: string `json:"-"`

source: string

the complete source line, for JSON consumers; columns count runes.

category: string

The warning’s category as “type.subtype”, such as “ref.term”, which a host may let users silence (Sphinx’s suppress_warnings); “” for none.

host.Session :: struct

Session owns the storage of one conversion so reports stay valid while it lives. Its storage, output, and source map are reserved from shared when that is set (the threads of a parallel build share one), and otherwise from a budget of its own of limit bytes. allocator supplies its regions (context.allocator when unset); it must outlive the session and reclaim freed regions to bound actual memory across retries. A shared budget’s policy may raise its limit; all reservations use that same limit.

allocator: mem.Allocator

ws: gd.Workspace

storage: gd.Storage

sizes: Sizes

output: []u8

result: gd.Result

source_map: []gd.Map_Entry

map_size: int

source map entries; 0 for none.

retries: int

ready: bool

limit: int

0: DEFAULT_BUDGET.

shared: ^Budget

nil: own.

own: Budget

held: int

bytes reserved for the storage, output, and source map.

inputs: ^Kept_File

owned input buffers; freed before releasing kept.

kept: int

bytes reserved for owned input buffers and their records.

refused: int

bytes a refused reservation asked for, or 0; max(int): too many.

denied: bool

the system refused an allocation of refused bytes the budget allowed.

disk: ^Disk

where the regions are when the budget spilled them; nil: memory.

allocated: mem.Allocator

the allocator that owns the current regions.

host.Sizes :: struct

Sizes are region capacities in elements (bytes for text and scratch).

nodes: int

attributes: int

lists: int

links: int

cells: int

text: int

scratch: int

sources: int

reports: int

output: int

second_bank: bool

needed only for passes.

host.Target :: enum u8

Target is the artifact convert_artifact publishes.

Html

Pdf

Procedures

host.arena_allocator :: proc(a: ^Arena) -> mem.Allocator

arena_allocator returns an allocator that allocates from a; it borrows a. Free does nothing; Free_All frees every block.

host.arena_destroy :: proc(a: ^Arena)

arena_destroy frees every block of a, which is empty after and may be used again. Nothing allocated from a may be used after it.

host.arena_init :: proc(a: ^Arena, backing: mem.Allocator, block := 0)

arena_init starts an empty arena over backing, which must outlive it, taking blocks of block bytes (0: ARENA_BLOCK_BYTES; tests use small ones to reach every allocation).

host.available_memory :: proc() -> (bytes: int, ok: bool)

available_memory is the memory the system could give the process now without swapping, as it reports it: MemAvailable on Linux, lowered to what the process’s cgroup still allows, the available physical memory on Windows, and the free and inactive pages on macOS; or GUIDEDOG_AVAILABLE_MIB when it is set to a number. ok is false when it cannot say. It allocates nothing.

host.backend_problem :: proc(program: string) -> Problem

backend_problem is the problem of a Typst program that cannot be found; program is borrowed into it.

host.budget_advice :: proc(used, need, limit, available: int, disk: bool, other: string, available_known := false) -> string

budget_advice is the hint budget_hint gives, for a step that asked for need bytes while the budget held used of its limit, on a machine with available bytes free (0 when unknown unless available_known is true). It names the budget to build with (suggest_budget); when that is more than the free memory, it says so, and names keeping working memory on disk first, when disk is there to offer. It is allocated.

host.budget_hint :: proc(b: ^Budget, need: int, other: string) -> string

budget_hint is the hint of a budget’s problem (budget_advice): the budget the refused step fits in, from what b held and was asked for when it last refused (need, the bytes the step needed, when b refused nothing itself), capped by the memory the machine has free; keeping working memory on disk, when the caller gave b a Disk; and the other way out the caller names. It is allocated.

host.budget_join :: proc(b: ^Budget)

budget_join counts a unit of work that runs at once with others until budget_leave.

host.budget_leave :: proc(b: ^Budget)

budget_leave ends a unit budget_join counted, waking reservations that waited for it.

host.budget_limit :: proc(b: ^Budget) -> int

budget_limit is b’s limit now: on_refusal may have raised it since it was set.

host.budget_problem :: proc(what, path: string, bytes: int, b: ^Budget) -> Problem

budget_problem explains a charge the budget refused: what needed the bytes, and where. what and path are borrowed into it; the message is allocated.

host.budget_report :: proc(s: ^Session) -> (Report, bool)

budget_report explains a session that stopped at its budget, or at an allocation the system refused, or returns false. The message is allocated.

host.cancel_probe :: proc() -> gd.Cancel_Probe

cancel_probe returns a probe with which the core polls the Ctrl+C flag (catch_interrupts) during long conversions; it holds no state of its own.

host.canonical_roots :: proc(roots: []string) -> []string

canonical_roots resolves roots to absolute paths, as confined compares paths with them; a root that cannot be resolved is kept as written. The slice is allocated.

host.catch_interrupts :: proc()

catch_interrupts makes Ctrl+C (SIGINT) set a flag instead of ending the process, for the whole process: was_interrupted reads it, and cancel_probe gives it to the core. A program calls it once, at the start.

host.charge :: proc(l: ^Ledger, bytes: int) -> bool

charge reserves bytes for the unit, waiting as Budget describes, or returns false, reserving nothing and recording the refused bytes (l.refused). A nil ledger, or one without a budget, accepts every charge.

host.confined :: proc(roots: []string, path: string) -> bool

confined reports whether a file a project reads (a document, an image, a template, a static file) lies in one of the canonical roots once every link on its way is followed. Where the system resolves paths (POSIX) the resolved path decides; on Windows, where it does not, a path through a link is refused.

host.confinement :: proc(roots: []string, path: string) -> Confinement

confinement tells whether confined accepts a path, and why not when it does not, so a refusal can say whether the path itself or a link on its way leads outside. A refused path has a link leading out when one of its folders is itself confined: the path is written below a readable folder, and something below that folder leads away.

host.containing_root :: proc(roots: []string, path: string) -> (string, bool)

containing_root returns the root of roots that path is written below, or false. It compares the paths as written (case-insensitively on Windows) and follows no link; confined is the check that does.

host.convert_artifact :: proc(registry: gd.Registry, req: Artifact_Request) -> Artifact_Result

convert_artifact is the host route from a file to a published HTML or PDF artifact: read the input, convert it in a session of req.budget bytes, run Typst for PDF, and publish the output atomically (publish), or return it in stdout for “-“. Guidedoc conversion inside it is allocation-free; reading, Typst, and publishing are not. The result, its reports, and stdout are allocated in context.allocator and owned by the caller; the session is destroyed before it returns. A failure sets exit and problem and leaves any existing output unchanged.

host.crash_on_assertion :: proc(prefix, message: string, loc: runtime.Source_Code_Location) -> !

crash_on_assertion reports a panic or a failed assertion in Odin code, with its message and where it was raised, and exits with CRASH_EXIT.

host.default_stack_budget :: proc(stack: int) -> int

default_stack_budget is the budget to use when none is asked for: the default, or less on a thread too small for it.

host.describe :: proc(d: gd.Diagnostic, sources: []gd.Source_Entry) -> Report

describe fills a diagnostic’s templates and resolves its span against sources into a Report: path, line, column, excerpt and marker. It borrows d and sources for the call; the report’s strings are allocated and do not refer to the sources afterwards.

host.describe_all :: proc(s: ^Session) -> []Report

describe_all describes what the session’s conversion reported, and the budget it reached if any (budget_report). The reports are allocated and do not borrow the session, so they outlive session_destroy.

host.destroy_storage :: proc(st: gd.Storage, allocator := context.allocator)

destroy_storage frees every region of storage make_storage allocated with allocator; a region left nil is skipped, so it also frees a partly made storage.

host.discharge :: proc(l: ^Ledger, bytes: int)

discharge returns bytes the unit has freed before it ends.

host.disk_allocator :: proc(d: ^Disk) -> mem.Allocator

disk_allocator returns an allocator that allocates from d’s files; it borrows d. Free returns an allocation, and unmaps its file once the file holds none; Free_All is not supported (disk_destroy unmaps everything).

host.disk_destroy :: proc(d: ^Disk)

disk_destroy unmaps every file of d, freed or not; nothing allocated from d may be used after it. d may be used again.

host.disk_owns :: proc(d: ^Disk, p: rawptr) -> bool

disk_owns reports whether p lies in one of d’s files.

host.display_path :: proc(name: string) -> string

display_path returns name relative to the working directory when that is shorter to read (not above it by three folders or more), and name itself otherwise or for “” and “-“. The result may borrow name; a relative one is allocated.

host.dump :: proc(s: ^Session, doc: gd.Document) -> string

dump returns the tree outline of a successfully read document, allocated, or “” when the document is no longer valid; tests use it.

host.each_chunk :: proc(path: string, user: rawptr, each: proc(user: rawptr, chunk: []u8) -> bool) -> bool

each_chunk reads a file through a CHUNK_BYTES buffer, giving each chunk to each until it returns false. It returns whether the whole file was read and accepted.

host.excerpt_at :: proc(r: ^Report, text: string, line, column: int)

excerpt_at fills a report’s excerpt and marker at a 1-based line and Unicode scalar column, for positions in sources Guidedoc generated, such as a composed book.

host.exchange :: proc(a, b: string) -> (swapped: bool, err: os.Error)

Availability: Windows, Darwin, Linux, FreeBSD, OpenBSD, NetBSD.

exchange: see switch.odin. macOS swaps with renamex_np and RENAME_SWAP. It allocates nothing (fixed.odin).

host.exit_code :: proc(status: gd.Status) -> int

exit_code maps a conversion status to the process exit status Guidedog’s commands use: 0 ok, 1 invalid input or policy, 3 capacity or limits, 5 internal, 130 cancelled.

host.files_provider :: proc(f: ^Files) -> gd.Resource_Provider

files_provider resolves f.roots and returns a gd.Resource_Provider that serves includes through f, which it borrows: f and its roots must outlive every conversion the provider is given to. Each file it serves is allocated in the context.allocator of the conversion’s caller and charged to f.ledger when that is set.

host.fitting_budget :: proc(available: int) -> int

fitting_budget is a budget that fits in the free memory: nine tenths of it, in steps of 16 MiB; 0 when there is no room for even that minimum. Callers must not offer a zero budget to the CLI, where it means the default budget.

host.folders_of :: proc(paths: []string, top: string) -> []string

folders_of lists, once each and in order, the folders that hold paths and the folders between them and top (top included): the folders whose names a publication adds or changes. A path not below top gives only its own folder.

host.grow_session :: proc(s: ^Session, need: gd.Capacity_Need) -> bool

grow_session enlarges the region need names, for the next prepare_session; false when the enlarged sizes cannot be represented, which counts as the budget reached.

host.guard_allocator :: proc(g: ^Guard) -> mem.Allocator

guard_allocator returns an allocator that allocates from g.backing until it refuses once, and then refuses everything (see Guard); it borrows g, which must outlive every allocation made through it.

host.guard_problem :: proc(g: ^Guard, what: string) -> (Problem, bool)

guard_problem is host.memory (exit 3) when an allocation through g was refused, with what, a static sentence naming the work, as its message; or false. Its text is static, since the memory to write more has run out.

host.guard_tripped :: proc(g: ^Guard) -> bool

guard_tripped tells whether an allocation through g was refused. A caller may poll it to stop long work early; guard_problem explains it.

host.identity :: proc(path: string) -> (id: Identity, err: os.Error)

identity is the identity of what is at path; {} with no error when nothing is. It allocates nothing.

host.install_crash_reporter :: proc(args: []string, version: string)

install_crash_reporter prepares the crash report for the command line args (the program name first) and version, what guidedog --version prints (one or more lines), and installs the handlers that print it: for faults and traps: signals on POSIX, an unhandled-exception filter on Windows. It allocates nothing. The caller also sets context.assertion_failure_proc to crash_on_assertion, so a panic or failed assertion is reported with its message.

host.interrupted_problem :: proc() -> Problem

interrupted_problem is the problem, with exit status 130, of a command stopped by Ctrl+C.

host.io_problem :: proc(path: string, err: os.Error) -> Problem

io_problem explains an operating-system error on path; path is borrowed into it.

host.is_stage_name :: proc(name: string) -> bool

is_stage_name is whether name is one stage_file gives a temporary file: STAGE_PREFIX, STAGE_DIGITS digits, then nothing or one extension. Any other name, however close, is not Guidedog’s.

host.link_file :: proc(from, to: string, type: os.File_Type) -> (copied: bool, err: os.Error)

link_file makes to name the same file as from: a hard link, or, where the file system cannot link it (FAT, some network shares), a copy, which copied reports and the caller must flush. A symbolic link is made again, to the same target. An existing to is an error, never replaced. A regular file is linked or copied without allocating (fixed.odin).

host.list_folder :: proc(dir: string, allocator := context.allocator) -> (entries: []Folder_Entry, err: os.Error)

Availability: Windows, Darwin, Linux, FreeBSD, OpenBSD, NetBSD.

list_folder lists a folder’s entries, “.” and “..” left out, in the order the system gives them, with each entry’s type from the listing itself (d_type): unlike os.read_all_directory_by_path it opens no entry, which matters for a folder of thousands of files. An entry whose type the file system does not give is read with lstat. The names are allocated in allocator; an allocation that fails is an error.

host.main_stack_bytes :: proc() -> int

Availability: Windows, Darwin, Linux, FreeBSD, OpenBSD, NetBSD.

main_stack_bytes is the stack of the main thread.

host.make_storage :: proc(s: Sizes, allocator := context.allocator) -> (st: gd.Storage, ok: bool) #optional_ok

make_storage allocates storage of the sizes with allocator; the caller owns it and frees it with destroy_storage. ok is false, and nothing is kept allocated, when any region cannot be allocated: a region is never left nil. It reserves nothing from a Budget; a Session does that before calling it.

host.meter_allocator :: proc(m: ^Meter) -> mem.Allocator

meter_allocator returns an allocator that charges m.ledger and allocates from m.backing; it borrows m, which must outlive every allocation made through it.

host.meter_prepay :: proc(m: ^Meter, bytes: int) -> bool

meter_prepay charges bytes ahead, for work whose memory can be bounded before it starts: the work is refused at once when the budget cannot hold the bound, before any of it is done. Allocations then use the prepaid bytes before charging more; meter_refund returns what they did not use.

host.meter_refund :: proc(m: ^Meter)

meter_refund returns the prepaid bytes no allocation used to the budget.

host.metered_refusal :: proc(l: ^Ledger, what, path: string) -> (Problem, bool)

metered_refusal reports a charge a Meter’s ledger could not make: what the step needed, as budget_problem explains it.

host.out_of_memory :: proc(what: string) -> Problem

out_of_memory is host.memory with what, a static sentence naming the work that could not get memory, as its message; nothing in it is allocated.

host.output_text :: proc(s: ^Session) -> string

output_text is what the session’s last successful run rendered. It borrows the session’s output buffer: it is valid until the next run, prepare_session, resize_output, or session_destroy.

host.path_type :: proc(path: string) -> (type: os.File_Type, found: bool, err: os.Error)

path_type is what is at path, not following a symbolic link; found is false, with no error, when nothing is. It allocates nothing.

host.plan_session :: proc(s: ^Session, input_bytes: int, second_bank: bool) -> bool

plan_session sizes the session for an input of input_bytes; false when the sizes cannot be represented, which counts as the budget reached.

host.prepare_session :: proc(s: ^Session) -> bool

prepare_session allocates storage of the session’s current sizes, releasing any older storage first. It reserves the bytes from the session’s budget before allocating, and returns false, allocating nothing, when the budget cannot hold them. run and read call it; callers that load objects call it directly.

host.publish :: proc(path: string, data: []u8, force: bool) -> Problem

publish writes bytes through a sibling temporary file, flushes it, and renames it over the destination, making the folders on its way: path holds its old bytes or the new ones, never a part. An existing destination is refused (host.output.exists) unless force is set; any failure leaves the destination unchanged and returns its Problem.

host.read :: proc(s: ^Session, reader: gd.Reader, request: gd.Request, budget := DEFAULT_BUDGET) -> gd.Read_Result

read parses without rendering, then runs request.passes (a translation, for example) over the document; storage grows and the whole read retries when a region fills. The document and reports borrow the session, as run’s do, and the source text.

host.read_file :: proc(path: string, ledger: ^Ledger, limit := MAX_SOURCE_BYTES, allocator := context.allocator) -> ([]u8, Problem)

read_file reads a regular file of at most limit bytes whole, charging its size to ledger before allocating; the caller discharges the bytes when it frees them. A file that grows while it is read is refused. If it shrinks, its original allocation stays charged until freed: fewer bytes read do not mean fewer bytes held.

host.read_kept :: proc(s: ^Session, path: string, limit: int) -> ([]u8, Problem)

read_kept reads a file whose bytes the caller keeps while the session’s document refers to them. The bytes belong to the caller, allocated in context.allocator; the caller frees them no later than session_destroy, which returns their budget charge. Use read_owned when the session should own and free the bytes itself.

host.read_owned :: proc(s: ^Session, path: string, limit: int) -> ([]u8, Problem)

read_owned reads a file the session’s document refers to while it lives, such as its cached object. The session owns the returned bytes until session_destroy, which frees them before releasing their charge. Its allocator supplies both the bytes and their ownership record; each allocation is charged before it is made.

host.read_source :: proc(path: string, limit := MAX_SOURCE_BYTES) -> (text, canonical: string, p: Problem)

read_source reads a named input, or standard input for “-“, within limit bytes. It returns the text and the absolute path used as the source’s canonical name.

host.remove_path :: proc(path: string) -> os.Error

Availability: Windows, Darwin, Linux, FreeBSD, OpenBSD, NetBSD.

remove_path removes a file, a link, or an empty folder; see fixed.odin.

host.remove_stages :: proc(dir: string)

remove_stages removes, in dir and every folder below it, the temporary files that a process stopped (killed, or the machine lost power) left before renaming or removing them: files, never folders or links, whose names is_stage_name accepts. Every other name is left as it is. A folder it cannot list, or has no memory to name, is left as it is: a stage file is clutter that nothing reads. The caller makes sure no stage file below dir is still in use.

host.rename_path :: proc(from, to: string) -> os.Error

Availability: Windows, Darwin, Linux, FreeBSD, OpenBSD, NetBSD.

rename_path renames from to to, replacing a file at to; see fixed.odin.

host.render_json :: proc(r: Report) -> string

render_json is a report as one JSON object (schema DIAGNOSTICS_SCHEMA), without its terminal excerpt and marker; “{}” when it cannot be encoded. The text is allocated.

host.render_problem_json :: proc(p: Problem) -> string

render_problem_json is a host problem as a JSON report of severity “error”.

host.render_problem_text :: proc(p: Problem, color: bool) -> string

render_problem_text lays out a host problem as render_text lays out an error report.

host.render_text :: proc(report: Report, consequence: string, color: bool) -> string

render_text lays out a report the Elm way: what happened, where, what it means for the result, and what to do next. Color supplements words; it never replaces them.

host.resize_output :: proc(s: ^Session, bytes: int) -> bool

resize_output replaces the session’s output buffer with one of bytes, reserving the whole new buffer while the old one still lives. It returns false, keeping the old buffer, when the budget cannot hold the new one.

host.resize_source_map :: proc(s: ^Session, entries: int) -> bool

resize_source_map gives the session a source map of entries, reserved as resize_output reserves its buffer.

host.run :: proc(s: ^Session, registry: gd.Registry, request: gd.Request, budget := DEFAULT_BUDGET, map_source := false) -> gd.Result

run converts with storage sized for the input, growing the exhausted region and retrying until it fits or the budget (bytes; see Session) would be exceeded, which ends it with status Limit and a report that says so (budget_report). With map_source set, the session also collects a source map from the renderer. The session owns everything the result refers to: its reports and output_text are valid until the next run or read on s, or session_destroy. request is borrowed for the call, and its source text for as long as the reports are read.

host.run_workers :: proc(count: int, data: ^$T, work: proc(data: ^T))

run_workers runs work(data) on count threads at once and returns when every one has ended; work shares data between them, and takes its share of the work itself (an index taken atomically, for one). A thread that cannot start is left out; when none can, work runs once on the calling thread, so the work is always done. It allocates the threads with context.allocator, which the workers may share (read_pending gives them the build’s, through a mutex this procedure does not take): so every thread is made before any starts, and freed only once every one has ended, and nothing is allocated while a worker runs. A refusal starts fewer.

host.same_contents :: proc(path: string, data: []u8) -> bool

same_contents reports whether the file at path holds exactly data, reading it in chunks and stopping at the first difference.

host.session_destroy :: proc(s: ^Session)

session_destroy frees the session’s storage, output, source map, and kept inputs, and releases what it reserved and kept from its budget. Reports, output_text, and documents the session returned are invalid after it. The session may be run again.

host.settle :: proc(l: ^Ledger)

settle releases everything l holds of its budget; the unit calls it when it frees the memory it charged. A nil ledger or one without a budget is a no-op.

host.sizes_for :: proc(input_bytes: int) -> (s: Sizes, ok: bool) #optional_ok

sizes_for estimates capacities from the input length. The estimate need not be tight: a capacity result names the region, and the session grows it and retries. ok is false when a capacity cannot be represented, which a session reports as its budget reached.

host.spill_disk :: proc(b: ^Budget) -> ^Disk

spill_disk is the Disk that working memory b cannot hold goes to, or nil when b does not spill.

host.stack_budget_limit :: proc(stack: int) -> int

stack_budget_limit is the largest stack budget a thread with stack bytes can hold: all but STACK_HEADROOM, and never more than half the stack. An 8 MiB thread (the main thread’s and core:thread’s usual stack on Linux and macOS) holds 6 MiB; a 1 MiB one (Windows) holds core.DEFAULT_STACK_BYTES, which stack.odin sizes to fit it.

host.suggest_budget :: proc(used, need, limit, available: int, available_known := false) -> (budget: int, fits: bool)

suggest_budget is the budget, in bytes, to suggest for a step that asked for need bytes while the budget held used of its limit, so that one change is enough: what is held, and room for the request STEP_COPIES times over (a step that builds a value of that size also holds its output and the copy it writes), or twice the old limit, whichever is larger, in steps of 64 MiB. When the machine has less memory free (available; 0 when unknown), the budget is capped to what fits there (fitting_budget), or, when what is needed does not fit, is what is needed, in steps of 16 MiB: then fits is false. available_known distinguishes measured zero from the default, unknown zero.

host.sync_all :: proc(paths: []string) -> Problem

sync_all flushes files and folders to storage, SYNC_WORKERS at a time, and returns the first that failed. A folder is flushed so the names in it are durable. Windows keeps its folders’ names durable itself and flushes no folder, so they are skipped there.

host.sync_each :: proc(paths: []string) -> Problem

sync_each flushes each path in turn on the calling thread, as sync_all does with threads, and returns the first that failed; it allocates nothing.

host.thread_stack_bytes :: proc() -> int

Availability: Windows, Darwin, Linux, FreeBSD, OpenBSD, NetBSD.

thread_stack_bytes is the stack of a thread core:thread starts.

host.was_interrupted :: proc() -> bool

was_interrupted reports whether Ctrl+C was pressed since catch_interrupts.

host.write_unsynced :: proc(path: string, data: []u8) -> Problem

write_unsynced writes bytes as publish does, through a sibling temporary file renamed over path, so path holds either its old bytes or the new ones, but does not flush them: a caller that needs them durable syncs them later (sync_all).

Constants

host.ARENA_BLOCK_BYTES :: 64 << 10

ARENA_BLOCK_BYTES is the size of the blocks an Arena takes from its backing allocator; a request larger than a quarter of it gets a block of its own.

host.AVAILABLE_OVERRIDE :: "GUIDEDOG_AVAILABLE_MIB"

AVAILABLE_OVERRIDE names the environment variable that states the memory available, in MiB, in place of what the system reports: for machines whose report misleads, and for tests that need the same advice on every machine.

host.CRASH_EXIT :: 70

CRASH_EXIT is the exit status of a command that crashed: EX_SOFTWARE, an internal software error, which no other failure uses.

host.DEFAULT_BUDGET :: 256 << 20

DEFAULT_BUDGET is a session’s budget in bytes when none is given (256 MiB).

host.DIAGNOSTICS_SCHEMA :: 1

DIAGNOSTICS_SCHEMA is the version of the JSON form of Report (render_json), which a consumer checks before reading the fields; it changes only when a field does.

host.DISK_CHUNK_BYTES :: 64 << 20

DISK_CHUNK_BYTES is the size of the temporary files a Disk maps; a larger request gets a file of its own size.

host.DISK_MARGIN :: 256 << 20

DISK_MARGIN is the free space a Disk leaves on the disk it maps files from, so that spilling working memory never fills the disk the rest of the system writes to.

host.ISSUE_TRACKER :: #config(GUIDEDOG_ISSUE_TRACKER, "")

ISSUE_TRACKER is where a crash is to be reported: the address of the issue tracker of whoever distributes the build, set when it is built (-define:GUIDEDOG_ISSUE_TRACKER=https://…). Without it, the report asks the user to send the details to whoever provided their build, and names no address.

host.MAX_RETRIES :: 64

MAX_RETRIES only guards against a bug: the budget is what bounds a session. Each retry at least doubles the region that filled, so growth reaches the budget in a few dozen steps however the regions fill in turn, as a document made of includes fills them.

host.MAX_SOURCE_BYTES :: 32 << 20

MAX_SOURCE_BYTES is the default size limit of one input file (32 MiB).

host.MEMORY_HINT

MEMORY_HINT is what to do when the system refuses memory.

host.PATH_BYTES :: 4096

PATH_BYTES is the longest path, with its terminator, these operations take: the longest that Linux takes (PATH_MAX), and four times macOS’s.

host.STAGE_PREFIX :: ".guidedog-stage-"

STAGE_PREFIX begins the name of every temporary file stage_file makes. It is followed by the STAGE_DIGITS digits os.create_temp_file draws, then the destination’s extension.

Foreign imports

foreign import host.kernel32 "system:Kernel32.lib"

Availability: Windows.

foreign import host.system "system:System"

Availability: Darwin.