Skip to content

Memory & Monitoring

1. Memory Architecture Overview

Rayforce uses a custom memory subsystem — no calls to malloc or free ever reach the C library. Every allocation flows through one of these layers:

  • Buddy allocator with thread-local heaps — Each VM thread gets its own heap (identified by a heap_id). Allocations are fast, lock-free within a thread. Cross-heap frees are deferred to a lock-free queue and reclaimed lazily.
  • Slab cache — Small allocations (common for atoms and short vectors) are served from pre-sized slab pools, avoiding buddy-tree overhead.
  • COW ref counting — Vectors use copy-on-write semantics via ray_retain/ray_release. Shared vectors are only copied when mutated. Note that ray_retain/ray_release/ray_cow are no-ops on RAY_ERROR objects, so an error block must be reclaimed with ray_error_free() rather than ray_release().
  • Arena allocator — For bulk short-lived blocks (e.g., intermediate query results). Arena objects carry an RAY_ATTR_ARENA flag that makes retain/release no-ops. The entire arena is freed at once when work completes.
  • Out-of-core spill — There is no enforced memory ceiling. The heap tracks how much anonymous (RAM) memory it has committed; once an allocation would push that past the anon watermark (total physical RAM by default), it is backed by a preallocated disk file instead of anonymous RAM. File-backed pages are always reclaimable to disk, so they can't trigger the OOM killer — the working set spills and the query completes (slowly) rather than being killed. This never rejects work. Total physical RAM is detected at startup for this threshold and for reporting (see .sys.info → total-mem).

For a deep dive into the allocator internals, see Memory Model.

2. The .sys.mem Function

Call (.sys.mem 0) to get a snapshot of the current heap's allocation statistics. It returns a dictionary with the following fields:

Field Type Description
alloc-count i64 Total number of allocations performed since init
bytes-allocated i64 Live bytes in buddy-pool blocks (sub-32 MB objects)
direct-bytes i64 Live bytes in direct mmaps (objects ≥ 32 MB, mapped at exact size)
peak-bytes i64 High-water mark of bytes-allocated
slab-hits i64 Number of allocations served from the slab cache
sys-current i64 Committed RAM: every anonymous mapping (buddy pools, sys allocations, swap-fallback pool)
sys-mapped i64 File-backed bytes currently mapped (splayed columns, symbol file, parse buffers)
sys-mapped-peak i64 High-water mark of sys-mapped

Basic Usage

;; Check current memory stats
(.sys.mem)

alloc-count     | 14523
bytes-allocated | 2621440
peak-bytes      | 5242880
slab-hits       | 9870
sys-current     | 8388608

Checking Memory Before and After an Operation

;; Snapshot before
(set before (.sys.mem))

;; Load a large CSV
(set trades (.csv.read "trades-10M.csv"))

;; Snapshot after
(set after (.sys.mem))

;; See how much memory the load consumed
(- (after 'bytes-allocated) (before 'bytes-allocated))

838860800

In this example, loading 10 million rows consumed roughly 800 MB of heap memory.

3. The .sys.gc Function

Call (.sys.gc 0) to signal that the runtime should reclaim unused memory. Currently this is a lightweight hook that returns 0 — Rayforce uses deterministic ref counting and eager page release via madvise during buddy coalescing, so most memory is reclaimed automatically when references are dropped.

(.sys.gc)  ; 0

Note

Because Rayforce uses deterministic ref counting (not tracing GC), memory is freed immediately when the last reference is released. The buddy allocator coalesces blocks and releases pages back to the OS automatically. (.sys.gc 0) exists as a hook for future use.

4. The .sys.info Function

Call (.sys.info 0) to see system-level information about the Rayforce runtime:

(.sys.info)

cores     | 16
page-size | 4096
total-mem | 16777216000
Field Description
cores Number of logical CPU cores available
page-size OS page size in bytes
total-mem Total physical RAM in bytes

Note

On Windows, only cores is currently reported.

5. Progress Monitoring

Long-running queries display a progress bar automatically in the REPL. The bar appears after approximately 2 seconds of execution and shows real-time feedback.

Progress Bar Format

[████████░░░░░░░░]  52% · group: hash · 3.2s · 1.2G/12.8G

The bar displays:

  • Percentage — Estimated completion based on rows processed
  • Operation name — The current phase (e.g., group: hash, sort: merge, join: probe)
  • Elapsed time — Wall-clock time since query start
  • Memory — Live object footprint (used / total) against total physical RAM, so you can watch a large query climb toward the point where it no longer fits

Example Session

;; This query processes 50 million rows -- progress bar appears automatically
(select {from: trades
         by: {sym: sym}
         total: (sum price)
                n: (count price)
                hi: (max price)})

[████████████████]  100% · group: merge · 4.1s · 3.8G/12.8G
┌──────┬───────────────┬──────────┬──────────┐
│  sym │     total     │    n     │    hi    │
│  SYM │      F64      │   I64    │   F64    │
├──────┼───────────────┼──────────┼──────────┤
│ AAPL │ 8825431692.50 │ 50120832 │   502.39 │
│ GOOG │ 6129847201.75 │ 49879168 │   501.97 │
├──────┴───────────────┴──────────┴──────────┤
│ 2 rows                        4 columns    │
└────────────────────────────────────────────┘

The progress bar is cleared automatically when the query completes and the result is displayed. In non-interactive mode (file execution), progress output is suppressed.

6. The timeit Function

Wrap any expression in (timeit ...) to measure its execution time. It evaluates the expression, discards the result, and returns the elapsed time in milliseconds as an f64 value.

;; Time a simple vector operation
(timeit (sum (til 10000000)))

12.4

;; Time a grouped aggregation
(timeit (select {from: trades
                 by: {sym: sym}
                 n: (count price)}))

291.0

Note

timeit returns only the elapsed milliseconds — it does not return the expression's result. To see the result and the timing, evaluate the expression separately and use timeit for benchmarking.

7. Profiling with :t

The REPL command :t (or :timeit) toggles profiling mode. When active, every expression displays a detailed timing span tree showing where time was spent.

Example Session

;; Toggle profiling on
:t

. Timeit is on.

;; Now every expression shows timing spans
(sum (til 1000000))

499999500000
── top-level ─────────── 1.8ms

;; Toggle profiling off
:t

. Timeit is off.

This is the fastest way to identify slow expressions in an interactive session. Combine it with .sys.mem snapshots to see both time and memory costs.

8. Total RAM and Out-of-Core Spill

Rayforce imposes no enforced memory ceiling — it never rejects or throttles work. Instead it is out-of-core: memory that does not fit in RAM spills to disk.

Why not just rely on the OS. An anonymous (RAM) mapping is dangerous: under Linux's default overcommit the kernel accepts a mapping larger than it can actually back, then invokes the OOM killer when the pages fault in. You cannot tell at allocation time whether that will happen. A file-backed mapping over a preallocated disk file is safe: its pages are always reclaimable to the file, so it can never trigger the OOM killer — worst case is slower I/O or a clean disk-full error.

The anon watermark. So the heap tracks the anonymous (RAM-resident) bytes it has committed, and when a new pool or large allocation would push that past the watermark — total physical RAM by default — it backs that allocation with a disk spill file instead of anonymous RAM. This never rejects work; it just routes the overflow to disk so it spills rather than getting OOM-killed. A query like (til 10000000000) (a 74 GiB vector) on a smaller machine now spills to disk and completes (slowly) instead of being terminated. The progress bar's used / total figure shows the footprint approaching total RAM — the point where spill begins.

Checking total RAM

;; The total-mem field shows total physical RAM
(.sys.info)

cores     | 16
page-size | 4096
total-mem | 16777216000

Gauging headroom

To see how close a workload is to spilling to disk, compare the live object footprint (bytes-allocated + direct-bytes from (.sys.mem)) against total-mem from (.sys.info):

;; Live footprint as a percentage of physical RAM
(set stats (.sys.mem))
(set info (.sys.info))
(* 100.0 (/ (+ (stats bytes-allocated) (stats direct-bytes)) (info total-mem)))

29.4

This shows 29.4% of physical RAM is in use — plenty of headroom before the heap begins spilling to disk.

9. Practical Patterns

Pattern 1: Monitor Memory During CSV Loading

;; Check baseline
(set m0 (.sys.mem))

;; Load data
(set trades (.csv.read "trades-50M.csv"))

;; Check cost
(set m1 (.sys.mem))
(println (format "Loaded: {} bytes, peak: {} bytes"
  (- (m1 bytes-allocated) (m0 bytes-allocated))
  (m1 peak-bytes)))

Loaded: 4194304000 bytes, peak: 4831838208 bytes

Pattern 2: GC Between Independent Queries

;; First analysis pass
(set result1 (select {from: trades
                       by: {date: date}
                       vol: (sum qty)})
(.csv.write result1 "daily-volume.csv")
(set result1 0)

;; Free intermediates before the next heavy query
(.sys.gc)

;; Second analysis pass with maximum headroom
(set result2 (select {from: trades
                       by: {sym: sym}
                       vwap: (/ (sum (* price qty)) (sum qty))}))

Pattern 3: Profile a Slow Query

;; Enable REPL profiling
:timeit

;; Break the query into parts to find the bottleneck
(set filtered (select {from: trades where: (> price 100.0)}))

elapsed: 42.1ms

(set grouped (select {from: filtered
                        by: {sym: sym}
                        avg_price: (avg price)
                               n: (count price)}))

elapsed: 203.7ms

(set sorted (select {from: grouped by: {n: (desc n)}}))

elapsed: 0.8ms

;; The group-by at 204ms is the bottleneck, not the filter (42ms) or sort (<1ms)
:timeit

Pattern 4: Check Peak Memory After a Join

;; Reset peak tracking with .sys.gc
(.sys.gc)
(set before-peak ((.sys.mem 0) peak-bytes))

;; Run a memory-intensive join
(set joined (select {from: trades join: quotes on: [sym time]}))

;; Check the peak
(set after-peak ((.sys.mem 0) peak-bytes))
(println (format "Join peak overhead: {} bytes"
  (- after-peak before-peak)))

Join peak overhead: 1073741824 bytes

This tells you the join needed about 1 GB of temporary memory beyond what was already allocated.

Summary

Tool What It Does When to Use
(.sys.mem 0) Returns heap allocation statistics Monitor memory usage, detect leaks
(.sys.gc 0) Flushes caches, releases pages Between heavy queries, before benchmarks
(.sys.info 0) Shows system and runtime info Check total RAM, CPU count, OS details
(timeit expr) Measures execution time of one expression Benchmark a specific operation
:timeit Toggles profiling for all REPL expressions Interactive performance exploration
Progress bar Automatic during long queries Visual feedback on query progress

Next Steps