.sys.* — system info and control
Process-level introspection (build, memory, host info) and command-style operations (shell-out, profiler toggle, IPC listener bind, env dump, GC trigger). The dotted .sys.<name> builtins are the typed entry points; .sys.cmd parses a colon-style command string and dispatches to the same handlers — used by the REPL's :listen 5000 style commands.
Restricted under -U
.sys.exec, .sys.cmd, .sys.listen, and .sys.querylog.enable are RAY_FN_RESTRICTED — they can run arbitrary shell commands, change the process's network surface, or change server-wide logging behaviour. The introspection entries (.sys.args, .sys.build, .sys.info, .sys.mem, .sys.prof, .sys.querylog, .sys.gc, .sys.env, .sys.timeit) are unrestricted.
Reference
| Function | Arity | Flags | Description |
|---|---|---|---|
.sys.args |
variadic | — | Command-line arguments as a typed dict. |
.sys.build |
nullary | — | Version + build date as a dict. |
.sys.info |
nullary | — | Host and process facts: cores, page size, total memory, pool size, pid, hostname. |
.sys.mem |
nullary | — | Allocator statistics. |
.sys.prof |
nullary | — | Last profiled query's per-step statistics as a table. |
.sys.querylog |
variadic | — | Ambient per-query statistics ring as a table. |
.sys.querylog.enable |
variadic | restricted | Toggle query-statistics logging. |
.sys.gc |
variadic (0–1) | — | Run allocator maintenance and return 0; (.sys.gc 1) also releases free pages now, best effort. |
.sys.env |
variadic (0–1) | — | Count or list of globally bound names. |
.sys.exec |
variadic | restricted | Run a shell command; return its exit code, optionally with stdout. |
.sys.cmd |
unary | restricted | Dispatch a colon-command string. |
.sys.listen |
unary | restricted | Bind an IPC listener on a TCP port. |
.sys.timeit |
variadic (0–1) | — | Toggle / set the per-expression profiler. |
.sys.args
Signature: (.sys.args). Returns the process's command-line arguments as a dictionary. Because the launcher flags have a known format, each top-level value carries its natural type; everything after a -- separator is collected into a user subdict (string → string), so an application can read its own options without colliding with Rayforce's own flags.
| Key | Type | Source flag | Notes |
|---|---|---|---|
file |
str | -f / positional |
Script path; empty if none. |
port |
i64 | -p |
IPC listen port; 0 if unset. |
cores |
i64 | -c |
Total execution cores, including the main thread; 0 = auto. |
timeit |
bool | -t |
Profiler enabled at startup. |
querylog |
bool | -Q |
Query-statistics logging enabled at startup. |
interactive |
bool | -i |
Force the REPL after a script. |
log |
str | -l / -L |
Journal base path; empty if none. |
source |
str | — | The file currently being evaluated: the innermost load, or the command-line script, by the path that was actually opened. Empty at the REPL, under a pipe, or in a hook or timer that runs outside any file. file is the script named on the command line ($0); source follows nested loads ($BASH_SOURCE). |
user |
dict | after -- |
The application's own arguments. |
The top-level schema is stable — every launcher key is always present with its effective value (the default when the flag wasn't passed), so (get (.sys.args) 'port) never misses. Auth passwords (-u / -U) and the RAM budget (-m / --mem) are deliberately not exposed; the effective budget is anon-watermark in (.sys.mem).
;; load a sibling of the running file, whichever directory it was started from
(set here (str-join (drop -1 (split (at (.sys.args) 'source) "/")) "/"))
(load (concat here "/lib.rfl"))
user parsing. Tokens after -- are paired -key value / --key value: a token starting with - is a key (leading dashes stripped to a symbol), and the next token is its value — unless that token also starts with -, in which case the value is the empty string (a bare flag). Duplicate keys keep the last value.
;; launched as: ./rayforce app.rfl -- -opt 123 --verbose
(.sys.args)
;; => {file:"app.rfl" port:0 cores:0 timeit:false interactive:false log:0Nc
;; user:{opt:"123" verbose:0Nc}}
(at (.sys.args) 'user) ;; just the application's options
;; => {opt:"123" verbose:0Nc}
(get (.sys.args) 'port) ;; a typed launcher value
;; => 0
Empty strings render as 0Nc in the REPL (log and the bare verbose flag above). See passing arguments to a script for the launcher side.
.sys.build
Signature: (.sys.build). Returns a dict with version (string) and build-date (string). Both fall back to "unknown" when the build didn't define RAYFORCE_VERSION / RAYFORCE_BUILD_DATE.
.sys.info
Signature: (.sys.info). Returns {cores: i64, page-size: i64, total-mem: i64, workers: i64, pid: i64, hostname: str} on every platform (on Windows from GetSystemInfo / GlobalMemoryStatusEx).
cores: the CPUs this process may run on. On Linux that is its affinity mask, whichtaskset, a cpuset ordocker --cpuset-cpusmay narrow below the machine's CPU count.workers: the threads of the execution pool, main included; before the pool exists, the size it will get..sys.infodoes not create the pool (see Running Several Instances).
(.sys.info)
;; => {cores: 10, page-size: 16384, total-mem: 68719476736,
;; workers: 10, pid: 48213, hostname: "tp-prod-01"}
pid and hostname describe this process, not the machine, and exist so an
embedded process can name itself. The common use is a per-start epoch that a
peer can check a stale cursor against:
(format "%-%-%" (get (.sys.info) 'hostname)
(get (.sys.info) 'pid)
(timestamp 'global))
;; => "tp-prod-01-48213-2026.09.18D10:05:59.000000000"
That identity is worth preferring over a random token because it is
diagnosable: reading it back tells you which host and which process produced
it. Note (timestamp 'global) is second-resolution, so pid is what separates
two starts within the same second.
If hostname cannot be obtained, it reports "" rather than failing the whole
call — one unavailable field should not cost the caller the others.
.sys.mem
Signature: (.sys.mem). Returns the allocator's running counters, the spill
watermark, a few diagnostic counters and the symbol-table size, all i64:
| Key | Meaning |
|---|---|
alloc-count |
Cumulative ray_alloc calls. |
bytes-allocated |
Live bytes in buddy-pool blocks (sub-32 MB objects). |
direct-bytes |
Live bytes in direct mmaps — large objects (≥ 32 MB) mapped at their exact page-rounded size, bypassing the power-of-2 buddy pool. bytes-allocated + direct-bytes is the true live object footprint. |
peak-bytes |
High-watermark since process start. |
slab-hits |
Cumulative fast-path slab allocations. |
sys-current |
Committed RAM: every anonymous mapping — the buddy pools (where all vectors live), sys allocations, and the swap-fallback pool. The true live-memory figure. |
sys-mapped |
File-backed bytes currently mapped (splayed columns, the symbol file, CSV/script parse buffers). Page-cache, evictable, resident only on touch — kept separate from sys-current so a read-once column can't masquerade as live RAM. |
sys-mapped-peak |
High-watermark of sys-mapped. |
anon-committed |
Anonymous RAM the heap has committed: buddy pools, direct blocks and sys blocks (off-heap tables such as the symbol arenas). Spill-file mappings are not counted. The figure checked against anon-watermark. |
anon-peak |
High-watermark of anon-committed since process start — the peak RAM the workload demanded, spill excluded. |
anon-watermark |
Spill threshold in bytes: a new pool, direct or sys block that would push anon-committed past it is mapped over a disk spill file instead of anonymous RAM. Set by -m SIZE / --mem SIZE; by default the RAM limit (physical RAM, or the cgroup memory limit when smaller) less a quarter of it, at least 3 GiB and at most half, left to the page cache. |
locked-bytes |
Bytes of the heap's mappings locked against swap-out (Linux): the mapped size, which is what RLIMIT_MEMLOCK counts, not the resident pages — a page is locked when it is first touched. At most the smaller of anon-watermark and half the RAM limit. |
lock-failed |
Times the kernel refused a lock, or a page release could not lock its range again, since start. |
lock-skipped |
Bytes, summed since start, that the heap mapped under the watermark but left unlocked: past the bound, past RLIMIT_MEMLOCK, or refused. The heap works on, unlocked. |
group-perpart-runs |
Diagnostic: results served by the streaming parted by: (GROUP) kernel — one per query, not per partition; a run that falls back to the flatten path is not counted. |
asof-perpart-runs |
Diagnostic: days processed by the streaming parted asof join — one per day in the union of both sides' partitions. |
sort-perpart-runs |
Diagnostic: partitions processed by the streaming parted asc: / desc: paths (ordered concat of pre-sorted partitions, or per-partition top-k). |
window-perpart-runs |
Diagnostic: segments processed by the streaming parted window path (the window's partition key includes the table's partition key). |
join-perpart-runs |
Diagnostic: segments processed by the broadcast-streaming join (one side parted, each segment joined against the whole other side). |
sym-count |
Entries in the global symbol table. |
sym-bytes |
Bytes held by the symbol table's string arena (every interned name). |
Every mapping — buddy pool or file — is counted exactly once through the VM
wrapper layer, so sys-current reflects the real committed footprint (a large
til shows up here, not as a flat few MB). anon-watermark is a spill
threshold, not a hard cap: work past it is never rejected, it goes to a
file-backed spill mapping (as it does when anonymous mmap is refused)
instead of risking the OOM killer — see
out-of-core spill.
Compare anon-committed against anon-watermark to gauge how close a
workload is to spilling; total physical RAM is reported separately by
.sys.info as total-mem.
The *-perpart-runs keys are monotonic diagnostic counters: a delta across a
query shows whether a parted table was streamed partition by partition
instead of flattened. sym-count / sym-bytes show whether a query left the
symbol table where it found it.
(.sys.mem)
;; => {alloc-count: 786, bytes-allocated: 28672, direct-bytes: 0,
;; peak-bytes: 28672, slab-hits: 354, sys-current: 34836480,
;; sys-mapped: 0, sys-mapped-peak: 0, anon-committed: 34795520,
;; anon-peak: 34799616, anon-watermark: 50178081792,
;; group-perpart-runs: 0, asof-perpart-runs: 0, sort-perpart-runs: 0,
;; window-perpart-runs: 0, join-perpart-runs: 0,
;; sym-count: 424, sym-bytes: 30336}
.sys.prof
Signature: (.sys.prof). Returns the profile of the most recently
completed query as a table — one row per pipeline step (parse, the
optimizer passes, each heavy operator, and materialize). Because it is an
ordinary table it can be queried like any other, and an IPC client gets it
with no extra protocol.
Profiling is opt-in: toggle it with the REPL :t command, .sys.timeit,
or the -t startup flag. With profiling off the table is empty. Capturing
the payload adds no measurable cost to a non-profiled query.
| Column | Meaning |
|---|---|
operator |
Step / operator label (symbol). |
depth |
Nesting depth (0 = top level). |
cumulative-ms |
Wall-clock milliseconds, including child operators. |
exclusive-ms |
Self time — this step minus its children (where time actually went). |
percent |
Share of total query time, exclusive-based (columns sum to ~100). |
rows |
Result element / row count (operators). |
output-kib |
Result serialized footprint, KiB (bandwidth produced). |
allocated-kib |
Net process bytes allocated across the step, KiB. |
workers |
Worker threads that ran a task for this step. |
busy-ms |
Summed worker busy time, ms. |
parallelism |
Effective parallelism = busy-ms / cumulative-ms (worker time ÷ wall time). |
Timing follows the same convention as EXPLAIN ANALYZE-style profilers:
cumulative-ms is the wall time of a step and everything nested under it,
while exclusive-ms strips the children out so the true hot step stands out
(a count over a huge til shows a large cumulative but a tiny exclusive —
the til underneath is the real cost). percent ranks steps by exclusive
time, so scanning that column finds the bottleneck directly.
;; run a query with the profiler on (:t 1), then:
(.sys.prof)
;; a table: operator | depth | cumulative-ms | exclusive-ms | percent | rows |
;; output-kib | allocated-kib | workers | busy-ms | parallelism
;; e.g. the GROUP row → rows=5000, output-kib=78.2, workers=28, parallelism=7.2
;; it is a normal table, so query it — rank steps by self time:
(select {from: (.sys.prof) where: (> exclusive-ms 1.0)})
The :t command also prints this as an indented tree after each query,
with self time, percent, and the same payload appended to every operator line.
.sys.querylog
Signature: (.sys.querylog). Returns the query-statistics ring as a table —
one row per completed query, oldest first, capped at the ring capacity (4096).
This is the ambient, server-side statistics feed: when logging is enabled the
server records a summary of every query it handles, and you read it back with an
ordinary query (an IPC client needs no extra protocol). The model mirrors a
system.query_log table.
Because each call returns a fresh materialized snapshot, anything you do to the
returned table (filter, sort, upsert) touches the copy, never the ring; and
.sys.querylog is a reserved name, so user code cannot rebind it. Empty until
logging is enabled with the -Q startup flag or (.sys.querylog.enable 1).
| Column | Meaning |
|---|---|
time |
Wall-clock time the query finished (timestamp). |
duration-ms |
Total wall time to answer the query. |
result-rows |
Rows in the result (a scalar counts as 1). |
output-kib |
Serialized result footprint, KiB. |
memory-kib |
Net process allocation across the query, KiB. |
workers |
Worker threads that ran a task. |
parallelism |
Effective parallelism = worker busy time ÷ wall time. |
status |
ok, or the error kind (e.g. type, name) on failure. |
query |
Source text (truncated to 256 chars). |
;; start the server with -Q 1 (or toggle at runtime), then:
(.sys.querylog)
;; a table: time | duration-ms | result-rows | output-kib | memory-kib |
;; workers | parallelism | status | query
;; it is a normal table, so query it — slowest queries first:
(select {from: (.sys.querylog) where: (> duration-ms 100.0)})
;; error rate:
(select {from: (.sys.querylog) by: status total: (count status)})
Capturing a row adds no measurable cost to a query, and logging is off by default — the disabled path is a single predicted branch.
.sys.querylog.enable
Signature: (.sys.querylog.enable [flag]). Toggles query-statistics logging.
No argument flips the current state; 0 disables; anything non-zero enables.
Returns the new state as i64 (0/1). Restricted under -U — it changes
server-wide behaviour.
The -Q startup flag is the equivalent for enabling logging from launch.
.sys.gc
Signature: (.sys.gc [mode]). Runs allocator maintenance: drains cross-thread frees,
flushes slab caches, coalesces free buddy blocks, reclaims empty oversized
pools, and incrementally returns aged free pages to the operating system.
Rayforce values remain deterministically reference-counted—this is allocator
GC, not a tracing object collector. Returns 0.
mode is 0 (the same as no argument) or 1. (.sys.gc 1) also runs the
ten-second idle decay's sweep now, without waiting for the process to go
quiet or for free blocks to age, and unmaps the cached large blocks, which
the decay keeps. A server that a timer or a client keeps busy never goes
quiet for that long, so this is how it gives back a peak it has finished
with; the next use of that memory faults the released pages in again. It is
best effort, not a promise that every free page leaves the process: the
sweep covers the calling thread's heap and the idle workers' heaps, a block
keeps its first page and one under 8 KiB keeps all of them, a spill pool's
memory goes back only with the pool, and when resident memory drops is up
to the operating system. It returns 0, as (.sys.gc) does. Another mode
is a domain error, a non-i64 a type error, and more than one argument
an arity error. See
Giving a peak back.
.sys.env
Signature: (.sys.env [compat-arg]). From a script / IPC context returns the count of globally bound names (i64). In a REPL context the same dispatcher prints one line per binding (name + type label) and returns null. At most one optional compatibility argument is accepted; additional arguments return an arity error.
.sys.exec
Signature: (.sys.exec "command") or (.sys.exec "command" 'out).
Runs command through the shell. The one-argument form returns the exit
code as an i64 — the code the shell itself would report, not the
waitpid-encoded status. A command killed by a signal reports 128 + signum,
following the same shell convention.
The 'out form returns {code: i64, out: str}, capturing the command's
standard output. Output is read until EOF and is not sized from stat, so
pipes and procfs reads come back whole. stderr is not captured — it stays
on the process's own stderr, so a command's diagnostics land in the log rather
than corrupting a value you are about to parse.
Errors: type (command not a string, or mode not a symbol), domain (null
pointer, or a mode other than 'out), io (the command could not be started
or did not complete), oom.
(.sys.exec "rm -rf /tmp/scratch") ;; => 0
(.sys.exec "false") ;; => 1
(.sys.exec "exit 3") ;; => 3
(.sys.exec "echo hi" 'out) ;; => {code: 0, out: "hi\n"}
(.sys.exec "echo partial; exit 4" 'out) ;; => {code: 4, out: "partial\n"}
'out waits for every process holding the pipe
The capture reads until end-of-file on the command's stdout, and a
backgrounded child inherits that pipe. So 'out waits for the background
process too, not just for the shell:
(.sys.exec "(sleep 3 &) ; true" 'out) ;; returns after 3s
(.sys.exec "./server &" 'out) ;; waits for the server
Detach the child's stdout and the call returns at once:
This is inherent to capturing output — popen(3) behaves the same way, as
do the equivalents in other languages. The one-argument form does not
have it, since it only waits for the shell itself, so prefer plain
(.sys.exec "./server &") to launch something that outlives the call.
Behaviour change
Before this, the one-argument form returned the raw waitpid-encoded
status, so (.sys.exec "false") was 256 and (.sys.exec "exit 3") was
768. Success was 0 either way, so code that only checks for zero is
unaffected; code comparing against an encoded value needs updating.
.sys.cmd
Signature: (.sys.cmd "name args"). Parses a colon-style command name (without the leading :) and dispatches. Names supported: timeit/t, listen, env, clear, help/?, q/quit. Unknown names fall back to system() (so (.sys.cmd "ls /tmp") works too).
clear, help, and q are REPL-only — invoking them from a script returns a domain error rather than silently no-op'ing.
(.sys.cmd "listen 5000") ;; => listener id
(.sys.cmd "env") ;; => binding count
(.sys.cmd "timeit 1") ;; => 1 (profiler enabled)
.sys.listen
Signature: (.sys.listen port). Binds an IPC listener on port using the runtime's main poll. Returns the listener id (i64).
Errors: type (port not an int / not parseable from string), domain (port outside [1, 65535]), nyi (no main loop — embedded library without a poll), io (bind failure, e.g. port already in use).
.sys.timeit
Signature: (.sys.timeit [flag]). Toggles the per-expression profiler. Calling with no argument flips the current state; passing 0 disables, anything non-zero enables. Returns the new state as i64 (0/1). More than one argument returns an arity error.
(.sys.timeit 1) ;; enable profiling
(sum (til 1000))
;; (timings printed for each subexpression)
(.sys.timeit 0) ;; disable