Memory & Monitoring

Understand Rayforce's memory model, inspect allocator statistics, trigger garbage collection, and profile query performance from the REPL.

1. Memory Architecture Overview

Rayforce uses a custom memory subsystem — no calls to malloc or free ever reach the C library. Every allocation flows through one of these layers:

• Buddy allocator with thread-local heaps — Each VM thread gets its own heap (identified by a heap_id). Allocations are fast, lock-free within a thread. Cross-heap frees are deferred to a lock-free queue and reclaimed lazily.
• Slab cache — Small allocations (common for atoms and short vectors) are served from pre-sized slab pools, avoiding buddy-tree overhead.
• COW ref counting — Vectors use copy-on-write semantics via ray_retain/ray_release. Shared vectors are only copied when mutated.
• Arena allocator — For bulk short-lived blocks (e.g., intermediate query results). Arena objects carry an RAY_ATTR_ARENA flag that makes retain/release no-ops. The entire arena is freed at once when work completes.
• Memory budget — Auto-detected at initialization as 80% of physical RAM (via sysconf on POSIX, GlobalMemoryStatusEx on Windows). Queries that approach the budget may trigger spilling or offloading.

For a deep dive into the allocator internals, see Memory Model.

2. The .sys.mem Function

Call (.sys.mem 0) to get a snapshot of the current heap’s allocation statistics. It returns a dictionary with the following fields:

Basic Usage

;; Check current memory stats
(.sys.mem)

alloc-count     | 14523
bytes-allocated | 2621440
peak-bytes      | 5242880
slab-hits       | 9870
sys-current     | 8388608

Checking Memory Before and After an Operation

;; Snapshot before
(set before (.sys.mem))

;; Load a large CSV
(set trades (.csv.read "trades-10M.csv"))

;; Snapshot after
(set after (.sys.mem))

;; See how much memory the load consumed
(- (after 'bytes-allocated) (before 'bytes-allocated))

838860800

In this example, loading 10 million rows consumed roughly 800 MB of heap memory.

3. The .sys.gc Function

Call (.sys.gc 0) to signal that the runtime should reclaim unused memory. Currently this is a lightweight hook that returns 0 — Rayforce uses deterministic ref counting and eager page release via madvise during buddy coalescing, so most memory is reclaimed automatically when references are dropped.

(.sys.gc)  ; 0

4. The .sys.info Function

Call (.sys.info 0) to see system-level information about the Rayforce runtime:

(.sys.info)

cores     | 16
page-size | 4096
total-mem | 16777216000

5. Progress Monitoring

Long-running queries display a progress bar automatically in the REPL. The bar appears after approximately 2 seconds of execution and shows real-time feedback.

Progress Bar Format

[████████░░░░░░░░]  52% · group: hash · 3.2s · 1.2G/12.8G

The bar displays:

• Percentage — Estimated completion based on rows processed
• Operation name — The current phase (e.g., group: hash, sort: merge, join: probe)
• Elapsed time — Wall-clock time since query start
• Memory usage — Current heap usage vs. the memory budget

Example Session

;; This query processes 50 million rows -- progress bar appears automatically
(select {from: trades
         by: {sym: sym}
         cols: {total: (sum price)
                n: (count price)
                hi: (max price)}})

[████████████████]  100% · group: merge · 4.1s · 3.8G/12.8G
┌──────┬───────────────┬──────────┬──────────┐
│  sym │     total     │    n     │    hi    │
│  sym │      f64      │   i64    │   f64    │
├──────┼───────────────┼──────────┼──────────┤
│ AAPL │ 8825431692.50 │ 50120832 │   502.39 │
│ GOOG │ 6129847201.75 │ 49879168 │   501.97 │
├──────┴───────────────┴──────────┴──────────┤
│ 2 rows                        4 columns    │
└────────────────────────────────────────────┘

The progress bar is cleared automatically when the query completes and the result is displayed. In non-interactive mode (file execution), progress output is suppressed.

6. The timeit Function

Wrap any expression in (timeit ...) to measure its execution time. It evaluates the expression, discards the result, and returns the elapsed time in milliseconds as an f64 value.

;; Time a simple vector operation
(timeit (sum (til 10000000)))

12.4

;; Time a grouped aggregation
(timeit (select {from: trades
                 by: {sym: sym}
                 cols: {n: (count price)}}))

291.0

7. Profiling with :t

The REPL command :t (or :timeit) toggles profiling mode. When active, every expression displays a detailed timing span tree showing where time was spent.

Example Session

;; Toggle profiling on
:t

. Timeit is on.

;; Now every expression shows timing spans
(sum (til 1000000))

499999500000
── top-level ─────────── 1.8ms

;; Toggle profiling off
:t

. Timeit is off.

This is the fastest way to identify slow expressions in an interactive session. Combine it with .sys.mem snapshots to see both time and memory costs.

8. Memory Budget and Pressure

Rayforce auto-detects the memory budget at startup as 80% of physical RAM. This budget governs when the runtime begins applying back-pressure to avoid exhausting system memory.

Checking the Budget

;; The total-mem field shows total physical RAM
(.sys.info)

cores     | 16
page-size | 4096
total-mem | 16777216000

On this machine with 16 GB of RAM, the memory budget (80%) is approximately 12.5 GB.

How Pressure Works

When heap usage exceeds the memory budget:

• The executor may switch to streaming/morsel-at-a-time processing for large intermediate results
• Block offloading can spill cold data to disk (see Block Offloading)
• The progress bar shows current memory relative to the budget so you can see pressure building

You can monitor pressure by comparing bytes-allocated from (.sys.mem 0) against total-mem from (.sys.info 0):

;; Calculate memory pressure as a percentage
(set stats (.sys.mem))
(set info (.sys.info))
(* 100.0 (/ (stats bytes-allocated) (info total-mem)))

29.4

This shows 29.4% of the budget is in use — plenty of headroom.

9. Practical Patterns

Pattern 1: Monitor Memory During CSV Loading

;; Check baseline
(set m0 (.sys.mem))

;; Load data
(set trades (.csv.read "trades-50M.csv"))

;; Check cost
(set m1 (.sys.mem))
(println (format "Loaded: {} bytes, peak: {} bytes"
  (- (m1 bytes-allocated) (m0 bytes-allocated))
  (m1 peak-bytes)))

Loaded: 4194304000 bytes, peak: 4831838208 bytes

Pattern 2: GC Between Independent Queries

;; First analysis pass
(set result1 (select {from: trades
                       by: {date: date}
                       cols: {vol: (sum qty)}})
(.csv.write result1 "daily-volume.csv")
(set result1 0)

;; Free intermediates before the next heavy query
(.sys.gc)

;; Second analysis pass with maximum headroom
(set result2 (select {from: trades
                       by: {sym: sym}
                       cols: {vwap: (/ (sum (* price qty)) (sum qty))}}))

Pattern 3: Profile a Slow Query

;; Enable REPL profiling
:timeit

;; Break the query into parts to find the bottleneck
(set filtered (select {from: trades where: (> price 100.0)}))

elapsed: 42.1ms

(set grouped (select {from: filtered
                        by: {sym: sym}
                        cols: {avg_price: (avg price)
                               n: (count price)}}))

elapsed: 203.7ms

(set sorted (select {from: grouped by: {n: (desc n)}}))

elapsed: 0.8ms

;; The group-by at 204ms is the bottleneck, not the filter (42ms) or sort (<1ms)
:timeit

Pattern 4: Check Peak Memory After a Join

;; Reset peak tracking with .sys.gc
(.sys.gc)
(set before-peak ((.sys.mem 0) peak-bytes))

;; Run a memory-intensive join
(set joined (select {from: trades join: quotes on: [sym time]}))

;; Check the peak
(set after-peak ((.sys.mem 0) peak-bytes))
(println (format "Join peak overhead: {} bytes"
  (- after-peak before-peak)))

Join peak overhead: 1073741824 bytes

This tells you the join needed about 1 GB of temporary memory beyond what was already allocated.

Summary

Next Steps

• Memory Model — Deep dive into the buddy allocator, arenas, COW, and per-VM heaps
• Block Offloading — How Rayforce handles datasets larger than RAM
• Pipeline & Optimizer — How queries are planned, optimized, and executed
• REPL Reference — All REPL commands and configuration