Functions Reference
Complete reference for all Rayfall built-in functions. Each entry shows the function name, arity type (unary/binary/variadic), flags, description, and example usage.
Legend
Functions marked atomic auto-map element-wise over vectors. Functions marked aggr reduce vectors to scalars. Functions marked special receive unevaluated arguments.
Arithmetic
Arithmetic operators auto-map over vectors and broadcast scalars. Numeric typed-vector calls and query expressions lower to morsel-based DAG kernels where available.
| Function | Type | Description | Example |
|---|---|---|---|
+ |
binary | Addition | (+ 3 4) → 7 |
- |
binary | Subtraction | (- 10 3) → 7 |
* |
binary | Multiplication | (* 3 4) → 12 |
/ |
binary | Float division (always f64) | (/ 7 2) → 3.5 |
% |
binary | Modulo | (% 7 3) → 1 |
div |
binary | Integer division (floor) | (div 7 2) → 3 |
neg |
unary | Negate | (neg 5) → -5 |
round |
unary | Round to nearest integer | (round 3.7) → 4.0 |
floor |
unary | Floor (round down) | (floor 3.7) → 3.0 |
ceil |
unary | Ceiling (round up) | (ceil 3.2) → 4.0 |
abs |
unary | Absolute value | (abs -7) → 7 |
sqrt |
unary | Square root (returns f64) | (sqrt 9) → 3.0 |
log |
unary | Natural logarithm | (log 2.718) → ~1.0 |
exp |
unary | Exponential (e^x) | (exp 1) → 2.718... |
sin |
unary | Sine, radians; returns f64 | (sin 0.0) → 0.0 |
asin |
unary | Arcsine, radians; out-of-domain values return 0Nf |
(asin 1.0) → 1.570... |
cos |
unary | Cosine, radians; returns f64 | (cos 0.0) → 1.0 |
acos |
unary | Arccosine, radians; out-of-domain values return 0Nf |
(acos 1.0) → 0.0 |
tan |
unary | Tangent, radians; returns f64 | (tan 0.0) → 0.0 |
atan |
unary | Arctangent, radians; returns f64 | (atan 1.0) → 0.785... |
reciprocal |
unary | Reciprocal 1/x; zero returns 0Nf |
(reciprocal 4) → 0.25 |
signum |
unary | Sign as i64: -1, 0, or 1; null stays null |
(signum -3.5) → -1 |
pow |
binary | Power (x^y, returns f64; DAG-lowered in queries) | (pow 2 10) → 1024.0 |
xbar |
binary | Round down to nearest multiple (bucketing) | (xbar [3 7 12] 5) → [0 5 10] |
Vector examples:
(+ [1 2 3] [10 20 30]) ; [11 22 33]
(* [1 2 3] 10) ; [10 20 30] scalar broadcast
(neg [1 -2 3]) ; [-1 2 -3]
(sqrt [4 9 16]) ; [2.0 3.0 4.0]
(sin [0.0 1.57079632679]) ; [0.0 1.0]
(reciprocal [2 0 4]) ; [0.5 0Nf 0.25]
(signum [-2.5 0.0 4.0]) ; [-1 0 1]
(xbar [3 15 27] 10) ; [0 10 20]
Comparison
All comparison operators are atomic and return boolean results.
| Function | Type | Description | Example |
|---|---|---|---|
> |
binary | Greater than | (> 5 3) → true |
< |
binary | Less than | (< 3 5) → true |
>= |
binary | Greater than or equal | (>= 5 5) → true |
<= |
binary | Less than or equal | (<= 3 5) → true |
== |
binary | Equal | (== 3 3) → true |
!= |
binary | Not equal | (!= 3 4) → true |
within |
binary | Check which vector elements fall within a range | (within [1 5 10] [3 7]) → [false true false] |
Logic
| Function | Type | Description | Example |
|---|---|---|---|
and |
binary | Logical AND | (and true false) → false |
or |
binary | Logical OR | (or true false) → true |
not |
unary | Logical NOT | (not true) → false |
Aggregation
Aggregation functions are marked aggr and reduce vectors to scalar values. Used in select with by: for group-by aggregation.
| Function | Type | Description | Example |
|---|---|---|---|
sum |
unary, aggr | Sum of all elements | (sum [1 2 3]) → 6 |
prod |
unary, aggr | Product of all non-null numeric elements | (prod [2 3 4]) → 24 |
all |
unary, aggr | True if every non-null numeric element is truthy; empty/all-null returns true |
(all [1 2 3]) → true |
any |
unary, aggr | True if any non-null numeric element is truthy; empty/all-null returns false |
(any [0 0 3]) → true |
count |
unary, aggr | Count of elements | (count [1 2 3]) → 3 |
avg |
unary, aggr | Arithmetic mean | (avg [1 2 3]) → 2.0 |
min |
unary aggr, or binary atomic | Reduce one vector, or choose the element-wise minimum of two values | (min [3 9 1] [2 10 4]) → [2 9 1] |
max |
unary aggr, or binary atomic | Reduce one vector, or choose the element-wise maximum of two values | (max [3 9 1] 5) → [5 9 5] |
med |
unary, aggr | Median value (returns f64) | (med [1 3 2]) → 2.0 |
mode |
unary, aggr | Most frequent non-null value; ties keep the first encountered value | (mode [1 2 2 3]) → 2 |
dev |
unary, aggr | Population standard deviation | (dev [1 2 3]) → 0.816... |
stddev |
unary, aggr | Sample standard deviation | (stddev [1 2 3]) → 1.0 |
stddev_pop |
unary, aggr | Population standard deviation | (stddev_pop [1 2 3]) |
dev_pop |
unary, aggr | Population standard deviation alias | (dev_pop [1 2 3]) |
var |
unary, aggr | Sample variance | (var [1 2 3]) → 1.0 |
var_pop |
unary, aggr | Population variance | (var_pop [1 2 3]) |
pearson_corr |
binary, aggr | Pearson correlation over paired non-null numeric inputs | (pearson_corr [1 2 3] [2 4 6]) → 1.0 |
cov |
binary, aggr | Population covariance over paired non-null numeric inputs | (cov [1 3] [2 6]) → 2.0 |
scov |
binary, aggr | Sample covariance over paired non-null numeric inputs | (scov [1 3] [2 6]) → 4.0 |
wsum |
binary, aggr | Weighted sum, sum(weights * values), skipping null pairs |
(wsum [1 3] [10 20]) → 70.0 |
wavg |
binary, aggr | Weighted average, wsum / sum(weights); null for zero total weight |
(wavg [1 3] [10 20]) → 17.5 |
quantile |
binary, aggr | Exact linear-interpolated quantile; probability is 0.0..1.0 |
(quantile [1 2 3 4] 0.5) → 2.5 |
percentile |
binary, aggr | Exact linear-interpolated percentile; probability is 0..100 |
(percentile [1 2 3 4] 50) → 2.5 |
first |
unary | First element of vector | (first [10 20 30]) → 10 |
last |
unary | Last element of vector | (last [10 20 30]) → 30 |
Numeric reducers skip nulls where applicable. Pairwise binary reducers skip a row when either vector input is null. Inside select/by:, aggregate reducers lower to morsel-based DAG paths that can run in parallel and poll for cancellation.
The examples below use this small in-memory table:
(set trades (table [sym price size time]
(list [AAPL GOOG AAPL]
[150.0 280.0 151.0]
[100 50 200]
[10 20 15])))
; Group-by aggregation example
(select {from: trades
by: {sym: sym}
hi: (max price) lo: (min price) n: (count price)})
Higher-Order Functions
Functions that take other functions as arguments.
| Function | Type | Description | Example |
|---|---|---|---|
map |
variadic | Apply function to each element (returns a list) | (map (fn [x] (* x 2)) [1 2 3]) → (2 4 6) |
pmap |
variadic | Parallel map (multi-threaded, returns a list) | (pmap (fn [x] (* x x)) [1 2 3]) → (1 4 9) |
filter |
binary | Keep elements where boolean mask is true | (filter [1 2 3 4] (> [1 2 3 4] 2)) → [3 4] |
fold |
variadic | Reduce with function and initial value | (fold + 0 [1 2 3]) → 6 |
fold-left |
variadic | Left-associative fold | (fold-left - 10 [1 2 3]) → 4 |
fold-right |
variadic | Right-associative fold | (fold-right - 10 [1 2 3]) → -8 |
scan |
variadic | Running fold (returns all intermediate results) | (scan + (enlist 1 2 3)) → [1 3 6] |
scan-left |
variadic | Left-to-right running fold | (scan-left + (enlist 1 2 3)) → [1 3 6] |
scan-right |
variadic | Right-to-left running fold (returns a list) | (scan-right + (enlist 1 2 3)) → (6 5 3) |
apply |
variadic | Zip-apply function pairwise over two lists | (apply + (enlist 1 2) (enlist 3 4)) → (4 6) |
map-left |
variadic | Map each element of the left over the whole right | (map-left + 10 [1 2 3]) → [11 12 13] |
map-right |
variadic | Map the whole left over each element of the right | (map-right - [10 20 30] 5) → [5 15 25] |
Collection Operations
Operations on vectors as collections.
| Function | Type | Description | Example |
|---|---|---|---|
distinct |
unary | Remove duplicates | (distinct [1 2 2 3]) → [1 2 3] |
in |
binary | Membership test (element in vector) | (in 2 [1 2 3]) → true |
except |
binary | Set difference | (except [1 2 3] [2]) → [1 3] |
union |
binary | Set union | (union [1 2] [2 3]) → [1 2 3] |
sect |
binary | Set intersection | (sect [1 2 3] [2 3 4]) → [2 3] |
take |
binary | Take first/last N elements | (take [10 20 30] 2) → [10 20] |
drop |
binary | Drop first/last N elements | (drop [10 20 30] 1) → [20 30] |
rotate |
binary | Rotate left by N positions; negative rotates right | (rotate [1 2 3 4] 1) → [2 3 4 1] |
cut |
binary | Split at sorted 0-based indices | (cut [1 2 3 4] [2]) → ([1 2] [3 4]) |
cross |
binary | Cartesian product as pairs | (cross [1 2] ['a 'b]) → ((1 'a) (1 'b) (2 'a) (2 'b)) |
at |
binary | Index a collection; vector keys batch dictionary lookups | (at [10 20 30] 1) → 20 |
find |
binary | Find index of value | (find [10 20 30] 20) → 1 |
fill |
binary, atomic | Replace null values; the replacement is the first argument | (fill 0 [1 0Nl 3]) → [1 0 3] |
reverse |
unary | Reverse order | (reverse [1 2 3]) → [3 2 1] |
til |
unary | Range [0..n) | (til 5) → [0 1 2 3 4] |
lag |
unary | Shift values one row back; first row is null/sentinel | (lag [10 20 30]) → [0Nl 10 20] |
lead |
unary | Shift values one row forward; last row is null/sentinel | (lead [10 20 30]) → [20 30 0Nl] |
deltas |
unary | Adjacent differences; first row is null | (deltas [10 15 13]) → [0Nl 5 -2] |
ratios |
unary | Adjacent ratios as f64; first row is null | (ratios [2 4 8]) → [0Nf 2.0 2.0] |
fills |
unary | Forward-fill nullable vectors, or every column of a table | (fills (as 'I64 (list 0N 2 0N))) → [0Nl 2 2] |
sums |
unary | Running sum; nulls are skipped | (sums [1 2 3]) → [1 3 6] |
avgs |
unary | Running average over non-null values | (avgs [2 4 6]) → [2.0 3.0 4.0] |
mins |
unary | Running minimum | (mins [3 1 2]) → [3 1 1] |
maxs |
unary | Running maximum | (maxs [3 1 2]) → [3 3 3] |
prds |
unary | Running product; nulls are skipped | (prds [2 3 4]) → [2 6 24] |
differ |
unary | Boolean change flag versus previous row; first row is true | (differ [1 1 2]) → [true false true] |
msum |
binary | Moving sum over trailing N rows; nulls are skipped | (msum 3 [1 2 3 4]) → [1 3 6 9] |
mavg |
binary | Moving average over trailing N rows and non-null values | (mavg 3 [1 2 3 4]) → [1.0 1.5 2.0 3.0] |
mmin |
binary | Moving minimum over trailing N rows | (mmin 3 [3 2 4 1]) → [3 2 2 1] |
mmax |
binary | Moving maximum over trailing N rows | (mmax 3 [3 2 4 1]) → [3 3 4 4] |
mcount |
binary | Moving non-null count over trailing N rows | (mcount 3 [1 2 3 4]) → [1 2 3 3] |
mvar |
binary | Moving population variance over trailing N rows and non-null values | (mvar 2 [1 3 5]) → [0.0 1.0 1.0] |
mdev |
binary | Moving population standard deviation over trailing N rows and non-null values | (mdev 2 [1 3 5]) → [0.0 1.0 1.0] |
enlist |
variadic | Wrap value(s) in a vector | (enlist 1 2 3) → [1 2 3] |
concat |
binary | Concatenate two vectors | (concat [1 2] [3 4]) → [1 2 3 4] |
raze |
unary | Flatten a list of vectors into one | (raze (list [1 2] [3 4])) → [1 2 3 4] |
where |
unary | Indices where boolean vector is true | (where [true false true]) → [0 2] |
group |
unary | Group indices by value | (group [A B A]) → dict of groups |
diverse |
unary | Check if all elements are unique | (diverse [1 2 3]) → true |
rand |
binary | N random values from range or vector | (rand 3 100) → 3 random ints 0..99 |
bin |
binary | Binary search (left boundary) | (bin [10 20 30] 25) → 1 |
binr |
binary | Binary search (right boundary) | (binr [10 20 30] 25) → 2 |
unify |
binary | Merge two tables/dicts, second takes precedence | (unify d1 d2) |
The time-series vector functions above are lazy-aware DAG operations for vector inputs. Moving-window helpers take a positive integer window first, then the vector. Constant windows inside select lower into DAG nodes; dynamic windows evaluate through the normal function path. These functions materialize through morsel-based kernels, can run in parallel, and poll the query cancellation flag during execution.
Sorting & Ordering
| Function | Type | Description | Example |
|---|---|---|---|
asc |
unary | Sort ascending | (asc [3 1 2]) → [1 2 3] |
desc |
unary | Sort descending | (desc [3 1 2]) → [3 2 1] |
iasc |
unary | Indices that would sort ascending | (iasc [30 10 20]) → [1 2 0] |
idesc |
unary | Indices that would sort descending | (idesc [30 10 20]) → [0 2 1] |
rank |
unary | Rank of each element | (rank [30 10 20]) → [2 0 1] |
xasc |
binary | Sort table ascending by column(s) | (xasc trades 'price) |
xdesc |
binary | Sort table descending by column(s) | (xdesc trades 'price) |
xrank |
binary | Assign rank buckets (quantiles) | (xrank 4 [10 20 30 40]) → [0 1 2 3] |
Table Operations
| Function | Type | Description | Example |
|---|---|---|---|
list |
variadic | Create a list from vectors | (list [1 2] [A B]) |
table |
binary | Create table from column names + list of vectors | (table [x y] (list [1 2] [A B])) |
key |
unary | Get column names (table) or keys (dict) | (key trades) → [sym price size] |
cols |
unary | Get table column names | (cols trades) → [sym price size] |
value |
unary | Get column data (table) or values (dict) | (value trades) |
dict |
binary | Create a dictionary from evaluated keys and values | (dict [a b] [1 2]) |
get |
binary | Lookup key in dict/table | (get d 'a) → 1 |
remove |
binary | Remove key from dict | (remove d 'a) |
row |
binary | Extract single row from table as dict | (row trades 0) |
meta |
unary | Get metadata (column types, lengths) | (meta trades) |
xcol |
binary | Rename all table columns | (xcol trades [ticker px sz]) |
xcols |
binary | Project/reorder named columns | (xcols trades [size sym]) |
xkey |
binary | Build a dict keyed by unique column value(s), with row dict values for remaining columns | (xkey trades 'time) |
xgroup |
binary | Group rows by column value(s), returning dict key -> table slice | (xgroup trades 'sym) |
fkeys |
unary | Inspect linked-column metadata as column -> target table |
(fkeys fact) |
alter |
variadic, special | In-place mutation of table column | (alter trades 'price (* price 1.1)) |
del |
variadic, special | Delete columns or rows from table | (del trades 'temp_col) |
modify |
variadic | Functional table update (returns new table) | (modify trades 'price (fn [p] (* p 1.1))) |
xkey returns a dictionary keyed by unique key column value(s). Duplicate keys are a domain error; use xgroup to get a dictionary whose values are grouped table slices. Passing a typed vector of keys to at performs a batch lookup and returns a list of the corresponding values.
Dictionary literals are literals: their entries are not evaluated, just as the
elements of other literals are not evaluated. Use (dict keys values) to
construct a dictionary from evaluated expressions.
Query Operations
These are special forms that bridge to the Rayforce DAG executor.
| Function | Type | Description |
|---|---|---|
select |
variadic, special | Query table with optional filter, projection, grouping, and aggregation |
update |
variadic, special | Add or modify columns in a table |
insert |
variadic, special | Append to a flat table/collection, or grow the live tail of a parted table |
upsert |
variadic, special | Insert or update rows (by key) |
; Select with filter and projection
(select {from: trades
where: (> price 100)
sym: sym notional: (* price size)})
; Group-by with multiple aggregates
(select {from: trades
by: {sym: sym}
vwap: (/ (sum (* price size)) (sum size))
count: (count price)})
; Update: add a column
(update {from: trades
notional: (* price size)})
insert has two table forms:
(insert target rows)is the ordinary two-argument form. It returns a new flat table, or rebinds and returns a quoted target symbol such as'trades. The row payload can be a list, table, or dictionary, as described by the target's physical columns.(insert parted partition-key rows)grows the in-memory live tail of a parted table and returns a fresh logical view, leavingpartedunchanged. Quote a symbol —(insert 'parted partition-key rows)— to rebind that symbol to the fresh view and return the symbol. A validated zero-row batch is a no-op and returns the existing view without rebinding. This is distinct from the three-argument positional form for vectors and lists, where the second argument is an insertion index.
; Append a row to a table
(insert 'trades (list 'AAPL 150.0 100 12))
; Append two rows to the live 2024.01.16 partition of a parted table.
; `date` is virtual, so the payload contains only physical data columns.
(insert 'trades 2024.01.16
(list ['AAPL 'MSFT] [151.0 410.0] [200 75] [13 14]))
; Vector / list operations
(set v (til 5)) ; [0 1 2 3 4]
(insert 'v 99) ; append: [0 1 2 3 4 99]
(insert 'v 0 -1) ; head: [-1 0 1 2 3 4 99]
(insert 'v 3 [100 200]) ; splice: [-1 0 1 100 200 2 3 4 99]
(insert 'v [0 3] 77) ; broadcast 77 at pre-positions 0 and 3
(insert 'v [1 3] [10 30]) ; parallel: 10 at pos 1, 30 at pos 3
Indices are pre-insertion positions in [0, count]; idx == count is equivalent to append. Vector positional inserts of a same-typed vector splice that vector in. List positional inserts always add the value as a single slot — use concat to splice. Multi-insert is stable on duplicate indices, preserving input order. Typed-null atoms (0Nl, 0Nf, …) carry their null flag through — the inserted slot is marked null, not zero.
For a parted target, rows must match the physical data-column schema exactly in name and concrete vector type. Do not include the virtual date or part column. A list maps values in physical column order; a table or dictionary may reorder columns, but must contain every physical name exactly once. Atoms append one row. Equal-length vectors append a batch, and atom values alongside them broadcast across that batch. Generic null is accepted for sentinel-nullable columns; SYM and STR map it to their empty value because those types have no distinct null, while BOOL and U8 reject it because they are non-nullable.
The partition key must equal the current last key, which grows that live segment, or be strictly later, which starts a new live segment. Earlier keys — including an existing historical partition — are immutable and cannot be inserted into. Existing partition metadata must already be strictly increasing in logical key order; use zero-padded integer directory names when lexical directory order would otherwise disagree with numeric order. Every historical physical segment must be present. A missing segment in the last partition can be repaired only by a non-empty same-key append; it blocks an empty insert or a move to a later key. BOOL/U8 cannot be null-backfilled when that missing segment already represents existing rows, but a missing zero-row segment needs no backfill.
A non-empty functional insert returns a new logical table view; the quoted-symbol form publishes that view by rebinding the symbol. Historical mmap segments are retained without copying, while a segment receiving rows becomes heap-backed. Queries and values that already retained the previous table continue to see that snapshot; subsequent resolution of a rebound target symbol sees the newly appended rows. A non-empty same-key append rebuilds partition metadata and copies the active segment; a later key materializes a new tail instead. Batch incoming rows to avoid repeatedly copying a growing intraday segment.
Live-tail insert is not persistence
Parted insert changes memory only. It does not modify partition directories, column files, .d, or .sym; process exit loses an unpersisted tail. Persist the completed physical partition explicitly at rollover, then reload the parted table if it should return to a fully mmap-backed view. See .db.parted.get for the production pattern.
upsert is not supported on parted tables. Materialize a flat table or use an application-level update strategy when key-based replacement is required.
upsert into a named flat table ((upsert 'book 'px row)) writes matched rows in place and appends new keys, so its cost is proportional to the payload rather than to the rows already present. A multi-row payload (a table, or a list of column vectors) is applied in one pass; a key repeated inside the payload resolves to its last row. A single-row upsert locates its row through a .idx.hash index on the key column when one is attached and fresh, and by one scan of the key column otherwise. A payload with a mistyped value is rejected before anything is written.
update with where: into a named flat table writes the matched rows in place: the predicate is resolved through a fresh .idx.hash index on the column when it is a plain equality ((== sym 'AAPL)), elementwise update expressions (arithmetic, comparisons, and other atomic builtins over columns and literals) are evaluated over the matched rows only, and the results are scattered into the live columns. Any other expression, such as one with an aggregate, an order-dependent function like deltas, or a free variable, is evaluated over the whole table as before, so both spellings of from: agree. A mistyped or wrong-length value is rejected before anything is written.
Joins
Rayforce supports equi-joins, outer joins, anti-joins, and time-series-aware joins.
| Function | Type | Description |
|---|---|---|
left-join |
variadic | Left join on matching columns. Unmatched rows filled with nulls. |
inner-join |
variadic | Inner join — only matching rows. |
full-join |
variadic | Full outer join. All left and right rows are preserved; unmatched columns are null. |
anti-join |
variadic | Anti-semi-join. Keep left rows with no right match. |
window-join |
variadic, special | Join with time window constraint. Match rows within a time range. |
asof-join |
variadic | As-of join — match the most recent preceding value. |
; Fixtures for join examples
(set trades_j (table [sym price size time]
(list [AAPL GOOG AAPL]
[150.0 280.0 151.0]
[100 50 200]
[10 20 15])))
(set quotes (table [sym time bid]
(list [AAPL GOOG AAPL]
[9 19 16]
[149.5 279.5 150.5])))
(set orders (table [product_id qty] (list [10 20 10] [2 1 5])))
(set products (table [product_id name] (list [10 20] [widget gadget])))
; Left join two tables on the sym column (join keys are a symbol list)
(left-join [sym] trades_j quotes)
; Inner join
(inner-join [product_id] orders products)
; Full outer join keeps rows from both sides
(full-join [sym] trades_j quotes)
; Window join: keys are [equality-keys... time-key]; intervals is
; a two-vector list with one [lo hi] window bound per left row.
; For each left row, aggregate the right rows whose time key falls in the window.
(set intervals (map-left + [-2 2] (at trades_j 'time)))
(window-join [sym time]
intervals
trades_j quotes
{avg_bid: (avg bid)})
; As-of join: keys come first, the last key is the time key
(asof-join [sym time] trades_j quotes)
Pivot & Window
| Function | Type | Description | Example |
|---|---|---|---|
pivot |
variadic | Pivot table — reshape long to wide. Args: table, index col, pivot col, value col, agg fn | (pivot trades 'sym 'date 'price sum) |
xbar |
binary, atomic | Bucket values (time bucketing for OHLC bars) | (xbar [3 7 12] 5) → [0 5 10] |
xrank |
binary | Assign N rank buckets (quantile ranking) | (xrank 4 [10 20 30 40]) |
; OHLC bars: bucket trades by 5-minute intervals
(select {from: trades
by: {sym: sym bucket: (xbar time 300000)}
open: (first price)
high: (max price)
low: (min price)
close: (last price)
vol: (sum size)})
String Operations
| Function | Type | Description | Example |
|---|---|---|---|
split |
binary | Split string by delimiter (returns a list) | (split "a,b,c" ",") → ("a" "b" "c") |
str-find |
binary | First byte index of a substring; returns 0Nl when absent |
(str-find "banana" "na") → 2 |
str-join |
binary | Join strings or symbols with a delimiter | (str-join ["a" "b"] ",") → "a,b" |
strlen |
unary | Length of each string | (strlen "hello") → 5 |
upper |
unary | Uppercase string or symbol atoms/vectors; lazy/DAG for vectors | (upper ["ab" "Cd"]) → ["AB" "CD"] |
lower |
unary | Lowercase string or symbol atoms/vectors; lazy/DAG for vectors | (lower 'AbC) → 'abc |
trim |
unary | Strip leading/trailing whitespace; lazy/DAG for vectors | (trim " abc ") → "abc" |
substr |
variadic | Substring by 1-based start and length; lazy/DAG for vectors | (substr "abcdef" 2 3) → "bcd" |
replace |
variadic | Replace all occurrences; lazy/DAG for vectors | (replace "a-b" "-" "_") → "a_b" |
like |
binary | Glob pattern match: * any, ? one, [abc]/[a-z]/[!abc] char class |
(like "hello" "hel*") → true |
concat |
binary | Concatenate two strings or vectors | (concat "hello" " world") → "hello world" |
format |
variadic | Format values as string (% is placeholder) | (format "x=%" 42) → "x=42" |
Note
String transforms are available as direct builtins and use the same DAG opcodes inside select/update. All string transformations propagate nulls: null input rows produce null output rows.
Date & Time
| Function | Type | Description | Example |
|---|---|---|---|
date |
unary | Current date or extract date from timestamp | (date 0) → today's date |
time |
unary | Current time or extract time from timestamp | (time 0) → current time |
timestamp |
unary | Current timestamp (nanosecond precision) | (timestamp 0) |
Calendar/Clock Field Extraction
Unary functions that pull a single calendar or clock field out of a DATE / TIME / TIMESTAMP atom or vector. Null input rows propagate to null output rows (the null sentinel bit pattern is not decoded as a bogus year / second). The same set of fields is reachable via dotted access on a temporal value (e.g. ts.yyyy — see Dotted Namespaces).
| Function | Range | Example |
|---|---|---|
yyyy |
year | (yyyy 2024.03.15) → 2024 |
mm |
1..12 | (mm 2024.03.15) → 3 |
dd |
1..31 | (dd 2024.03.15) → 15 |
hh |
0..23 | (hh 12:34:56) → 12 |
minute |
0..59 | (minute 12:34:56) → 34 |
ss |
0..59 | (ss 12:34:56) → 56 |
dow |
1..7 (Mon=1) | (dow 2024.03.15) → 5 |
doy |
1..366 | (doy 2024.03.15) → 75 |
mm is unambiguously MONTH; the minute spelling stays long-form because a two-letter token can't serve both meanings in a uniform dotted walk. Two dotted-only truncations exist: .date drops the time-of-day component (keeps the day), and .time drops the date component (keeps the microseconds within the day).
Cross-temporal comparisons are supported: dates, times, and timestamps are all converted to nanoseconds internally for comparison operations.
Type Operations
| Function | Type | Description | Example |
|---|---|---|---|
type |
unary | Get type name of a value | (type 42) → i64 |
as |
binary | Cast value to another type | (as 'i64 "42") → 42 |
nil? |
unary | Test if value is null; element-wise inside query expressions | (nil? x) |
rc |
unary | Reference count of an object | (rc x) → 1 |
guid |
unary | Generate a vector of N GUIDs ((guid 0) → []) |
(guid 1) |
I/O & File Operations
| Function | Type | Description | Example |
|---|---|---|---|
println |
variadic | Print values with newline | (println "hello" 42) |
print |
unary | Print value without newline | (print "hello") |
show |
variadic | Pretty-print a value (tables formatted) | (show trades) |
format |
variadic | Format value to string (% is placeholder) | (format "val=%" 42) → "val=42" |
.csv.read |
variadic | Load CSV file into table | (.csv.read "data.csv") |
.csv.write |
variadic | Write table to CSV file | (.csv.write trades "out.csv") |
read |
unary | Read file contents as string | (read "file.txt") |
read-bytes |
unary | Read file contents as a U8 byte vector |
(read-bytes "file.bin") |
write |
binary | Write a string to a file | (write "file.txt" "content") |
write-bytes |
binary | Write a U8 byte vector to a file |
(write-bytes "file.bin" bytes) |
load |
unary | Load and evaluate a Rayfall script | (load "lib.rfl") |
Control Flow
| Function | Type | Description | Example |
|---|---|---|---|
set |
binary, special | Bind value to global variable | (set x 42) |
let |
binary, special | Bind value to local variable | (let y (+ x 1)) |
if |
variadic, special | Conditional (if/then/else) | (if (> x 0) "pos" "neg") |
do |
variadic, special | Sequential execution, returns last | (do (set x 1) (set y 2) (+ x y)) |
fn |
variadic, special | Create lambda function | (fn [x] (* x x)) |
try |
binary, special | Error handling (expr handler-or-fallback) | (try (/ 1 0) (fn [e] 0)) |
raise |
unary | Throw an error | (raise "bad input") |
return |
variadic | Early return from compiled lambda (0 args → null) | (return 42) |
quote |
variadic, special | Return argument unevaluated; a bare name becomes a literal symbol ((quote x) ≡ 'x) |
(quote (+ 1 2)) → (+ 1 2) |
resolve |
variadic, special | Resolve a symbol in current scope | (resolve 'x) |
System & Utility
| Function | Type | Description | Example |
|---|---|---|---|
eval |
unary | Evaluate a parsed expression | (eval (parse "(+ 1 2)")) → 3 |
parse |
unary | Parse string into Rayfall expression | (parse "(+ 1 2)") |
.sys.gc |
variadic | Trigger garbage collection, returns 0 |
(.sys.gc) |
.sys.exec |
unary | Execute shell command, return exit code | (.sys.exec "ls -la") |
.os.getenv |
unary | Get environment variable | (.os.getenv "HOME") |
.os.setenv |
binary | Set environment variable | (.os.setenv "KEY" "value") |
exit |
unary | Exit with status code | (exit 0) |
timeit |
variadic, special | Benchmark an expression | (timeit (sum (til 1000000))) |
.time.now |
variadic | Current monotonic time in milliseconds | (.time.now) |
.time.timer.set |
variadic, restricted | Schedule a callback every ms milliseconds, num times (0 = forever). Returns timer id. |
(.time.timer.set 1000 0 (fn [t] (println t))) |
.time.timer.del |
unary, restricted | Cancel a scheduled timer by id. Returns null. | (.time.timer.del 0) |
.sys.args |
nullary | Application arguments as a typed dict (user subdict for post--- args) |
(.sys.args) |
env |
unary | List all global environment bindings | (env 0) |
.sys.build |
variadic | Build metadata: version + build-date |
(.sys.build) |
.sys.mem |
variadic | Memory allocator statistics (alloc/peak/slab) | (.sys.mem) |
.sys.info |
variadic | System information (cores, page size, memory) | (.sys.info) |
Serialization & Storage
| Function | Type | Description | Example |
|---|---|---|---|
ser |
unary | Serialize value to binary format | (ser [1 2 3]) |
de |
unary | Deserialize from binary format | (de bytes) |
.db.splayed.set |
variadic | Save table as splayed columns to directory | (.db.splayed.set "db/trades" trades) |
.db.splayed.get |
variadic | Load splayed table from directory | (.db.splayed.get "db/trades") |
.db.parted.get |
variadic | Load partitioned table by name from root directory | (.db.parted.get "db" 'trades) |
.db.parted.tables |
variadic | List table names under a parted root | (.db.parted.tables "db") |
.db.parted.fill |
variadic | Backfill missing tables across partitions | (.db.parted.fill "db") |
EAV (Entity-Attribute-Value)
Built-in support for triple stores. The EAV table has three columns: e (entity, i64), a (attribute, sym), v (value, i64).
| Function | Type | Description | Example |
|---|---|---|---|
datoms |
nullary | Create empty EAV table | (datoms) |
assert-fact |
variadic | Append triple to datoms | (assert-fact db 1 'name 100) |
retract-fact |
variadic | Remove triple from datoms | (retract-fact db 1 'name 100) |
scan-eav |
binary | Query EAV by attribute | (scan-eav db 'name) |
pull |
binary | Entity-centric retrieval (returns dict) | (pull db 1) → {name:100 age:30} |
sym-name |
unary | Convert sym intern ID to symbol string | (sym-name 0) |
; Build an EAV store
(set db (datoms))
(set db (assert-fact db 1 'name 100))
(set db (assert-fact db 1 'age 30))
(pull db 1) ; {name:100 age:30}
(scan-eav db 'name) ; table of (e, v) pairs where a='name
Table Set Operations
| Function | Type | Description | Example |
|---|---|---|---|
union-all |
binary | Concatenate two tables (all rows) | (union-all t1 t2) |
distinct |
unary | Remove duplicate rows from a table | (distinct t) |
anti-join |
variadic | Anti-semi-join: rows in left not in right | (anti-join [x] t1 t2) |
; Table concatenation and deduplication
(set t1 (table [x] (list [1 2])))
(set t2 (table [x] (list [2 3])))
(union-all t1 t2) ; 4 rows: 1 2 2 3
(distinct (union-all t1 t2)) ; 3 rows: 1 2 3
(anti-join [x] t1 t2) ; 1 row: 1
Datalog
Datalog rules and queries integrate with the EAV store. Rules use the (?entity :attribute ?value) pattern to match triples.
| Function | Type | Description | Example |
|---|---|---|---|
rule |
variadic, special | Define Datalog rule | (rule (path ?x ?y) (?x :edge ?y)) |
query |
variadic, special | Compile and execute Datalog query | (query db (find ?x ?y) (where (path ?x ?y))) |
not |
special (in Datalog WHERE) | Stratified negation in WHERE clause | (not (?y :edge 3)) |
; Define rules and query
(rule (path ?x ?y) (?x :edge ?y))
(rule (path ?x ?z) (?x :edge ?y) (path ?y ?z))
(set db (datoms))
(set db (assert-fact db 1 'edge 2))
(set db (assert-fact db 2 'edge 3))
(query db (find ?x ?y) (where (path ?x ?y)))
; returns table: (1,2) (1,3) (2,3)
Datalog Program API
Low-level API for building and evaluating Datalog programs directly, bypassing the EAV store.
| Function | Type | Description | Example |
|---|---|---|---|
dl-program |
nullary | Create Datalog program | (dl-program) |
dl-add-edb |
variadic | Register base relation (table + arity) | (dl-add-edb prog 'edge tbl 2) |
dl-stratify |
unary | Compute strata for the program | (dl-stratify prog) |
dl-eval |
unary | Evaluate program to fixpoint | (dl-eval prog) |
dl-query |
binary | Query a derived or base relation | (dl-query prog 'edge) |
dl-provenance |
binary | Reserved provenance hook; currently returns domain: not available |
(dl-provenance prog 'rel) |