1  # NeuralOS & Laya-Core: Test-First Systems Engineering, Kernel Tuning & Benchmark Specification
   2  
   3  **Document ID:** `NEURALOS-TDD-SPEC-v2.0`  
   4  **Chief Architect:** Shubham Sharma  
   5  **Systems Architecture & Audit:** Autonomous Kernel & Runtime Engineering  
   6  **Target Appliance:** Dell Node-01 (x86_64, 8,192 MB DDR4, ADATA 240 GB SATA-III SSD)  
   7  **Development Host:** Apple Silicon Workstation (M4 Pro, 48 GB Unified Memory) + Cross-Target CI  
   8  **Date:** November 2026  
   9  **Methodology:** Strict Test-First Driven Development (TFDD) & Hardware-in-the-Loop (HIL) Gatekeeping  
  10  
  11  ---
  12  
  13  ## 1. Executive Summary & Architectural Pivot
  14  
  15  Following the **November 2026 Kernel & Hardware Audit**, NeuralOS discards the high-maintenance "Custom Linux Distro / Custom PID 1" model in favor of the **Kernel-Coupled AI Appliance Architecture**:
  16  
  17  1. **Host OS Substrate:** Stripped, headless Linux (`multi-user.target`, Linux 6.12+ mainline kernel) retaining battle-tested Wi-Fi (`NetworkManager`), OpenSSH, and Tailscale mesh networking while purging all desktop, Snap, and background daemon bloat (**< 130 MB idle OS RAM**).
  18  2. **Kernel & Memory Control Plane:** Multi-Generational LRU (`MGLRU`), Transparent HugePages (`madvise`), 1.2 GB capped `ZRAM` (`zstd` for text/KV/heap only), hardware IRQ pinning to Core 0, and `sched_ext` (`scx_lavd`) BPF latency-critical CPU scheduling on Cores 1–7.
  19  3. **Laya-Core Runtime (`laya-daemon`):** A single compiled Rust (Edition 2024) service running inside a dedicated `cgroup v2` slice (`laya.slice`) with `CAP_IPC_LOCK` and `CAP_SYS_NICE`.
  20  4. **Zero-Copy Shared Memory Fabric:** A 64 MB `/dev/shm/laya_arena` backed by 2 MB HugePages using 64-byte cache-line-aligned lockless ring descriptors and `futex_waitv` wakeups (**< 5 µs IPC latency**).
  21  5. **Three-Tier Intent Routing:**
  22     - **Tier-0A (Deterministic SIMD):** Rust `daachorse` (SIMD Aho-Corasick) exact/prefix matcher (**< 2 µs**).
  23     - **Tier-0B (Semantic Fast-Path):** Int8 Static Embeddings (`Model2Vec` / AVX2 dot-product) classifying 80% of natural language commands without neural layer traversal (**< 25 µs**).
  24     - **Tier-1 (Deep Reasoning / DAG):** Fine-grained Mixture-of-Experts (MoE) / Small Language Model (`Granite-3.0-3B-A800M` or `Qwen2.5-1.5B` Q4_K_M) with `Q8_0` quantized KV-cache mapped via `mmap(MAP_POPULATE)` + `madvise(MADV_HUGEPAGE)`.
  25  6. **Subagent Execution & Storage:** Embedded **Wasmtime** micro-sandboxes (**< 1 ms cold start, < 2 MB RAM**) + **ICSR-V** (Interleaved CSR + Vector) memory-mapped hybrid storage for GraphRAG paging.
  26  7. **Native Display Surface:** `Ratatui` TUI bound directly to `/dev/tty1` framebuffer (4 MB RAM) + remote terminal/stream access.
  27  
  28  ---
  29  
  30  ## 2. Test-First Driven Development (TFDD) Philosophy for Systems Engineering
  31  
  32  In systems and kernel engineering, writing implementation code before verifying hardware/kernel preconditions leads to silent performance regressions, false-sharing cache stalls, and Out-Of-Memory (OOM) kernel panics.
  33  
  34  NeuralOS enforces a **4-Stage Gatekeeping Protocol** for every phase. **No implementation code for Phase $N$ may be written until the Phase $N$ Pre-Flight & Red-Gate Tests are written and executing.**
  35  
  36  ```
  37  ┌──────────────────────────────────────────────────────────────────────────────┐
  38  │ STAGE 0: PRE-FLIGHT BUILDABILITY GATE ("Is It Buildable on Target Hardware?")│
  39  │ • Probes kernel symbols, CPU flags (AVX2/FMA), SATA IOPS, cgroup2 & BPF BTF. │
  40  │ • Fails fast BEFORE a single line of Rust or C code is compiled.             │
  41  └──────────────────────────────────────┬───────────────────────────────────────┘
  42                                         ▼
  43  ┌──────────────────────────────────────────────────────────────────────────────┐
  44  │ STAGE 1: RED-GATE HARNESS (Write the Failing Functional & Property Tests)    │
  45  │ • Concurrency proofs (Loom), memory alignment assertions, WASI bounds tests. │
  46  └──────────────────────────────────────┬───────────────────────────────────────┘
  47                                         ▼
  48  ┌──────────────────────────────────────────────────────────────────────────────┐
  49  │ STAGE 2: IMPLEMENTATION (Write Minimal Production Rust / Kernel Config)      │
  50  │ • Zero-allocation hot paths, cache-aligned structs, SIMD intrinsics.         │
  51  └──────────────────────────────────────┬───────────────────────────────────────┘
  52                                         ▼
  53  ┌──────────────────────────────────────────────────────────────────────────────┐
  54  │ STAGE 3: THE SCOREBOARD GATE ("Score to Beat" Hardware Benchmark)            │
  55  │ • Criterion.rs / perf stat hardware counters (TLB misses, IPC, P99 latency). │
  56  │ • Merge is BLOCKED unless P99 beats the hard target threshold on Dell Node-01│
  57  └──────────────────────────────────────────────────────────────────────────────┘
  58  ```
  59  
  60  ---
  61  
  62  ## 3. Stage 0: Pre-Flight Buildability Gate ("Test First If Buildable")
  63  
  64  ### 3.1 Pre-Flight Buildability Matrix
  65  
  66  | Subsystem Requirement | Kernel / Hardware Probe | Minimum Buildable Threshold | Fallback if Gate Fails |
  67  | :--- | :--- | :--- | :--- |
  68  | **CPU Vector Intrinsics** | `grep -E "avx2.*fma" /proc/cpuinfo` | Both `avx2` and `fma` flags present | Abort build; x86_64-v3 target required for Tier-0B & `ggml` |
  69  | **Kernel Version & BTF** | `uname -r` & `/sys/kernel/btf/vmlinux` | Linux $\ge 6.8$ (for MGLRU/futex) or $\ge 6.12$ (for `sched_ext`) | Install Ubuntu Mainline Kernel 6.12 deb before proceeding |
  70  | **`sched_ext` Support** | `grep CONFIG_SCHED_CLASS_EXT /boot/config-$(uname -r)` | `=y` and `/sys/kernel/sched_ext` exists | Use `chrt -f 80` (`SCHED_FIFO`) + `cgroup v2 cpu.weight` until kernel upgrade |
  71  | **MGLRU Support** | `/sys/kernel/mm/lru_gen/enabled` | File exists and returns `0x0007` (`y`) | Upgrade kernel; legacy active/inactive LRU causes swap stalls |
  72  | **Cgroups v2 Unified** | `stat -fc %T /sys/fs/cgroup/` | Outputs `cgroup2fs` | Add `systemd.unified_cgroup_hierarchy=1` to GRUB |
  73  | **SATA SSD Direct I/O** | `fio --direct=1 --rw=randread --bs=128k` | $\ge 350\text{ MB/s}$ sequential, $\ge 120\text{ MB/s}$ 128K random | Disable FFN SSD streaming; strictly enforce in-RAM MoE models |
  74  
  75  ---
  76  
  77  ## 4. Master "Score to Beat" Benchmark Matrix
  78  
  79  | ID | Benchmark Metric | Legacy Baseline (Ubuntu GUI + Python + HTTP) | Minimum Passing Threshold (Gate) | Target "Score to Beat" (NeuralOS v2.0) | Measurement Tool / Command |
  80  | :--- | :--- | :--- | :--- | :--- | :--- |
  81  | **M-01** | **Host Idle RAM Footprint** (Post-Boot, No LLM Loaded) | `1,850 MB` | `< 220 MB` | **`<= 135 MB`** | `awk '/MemAvailable/ {print 8192 - int($2/1024)}' /proc/meminfo` |
  82  | **M-02** | **Available Physical RAM for AI & Agents** | `4,800 MB` | `>= 7,400 MB` | **`>= 7,850 MB`** | `awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo` |
  83  | **M-03** | **IPC Round-Trip Latency** (512B Payload, Warm Ring) | `28,400 µs` (28.4 ms HTTP/JSON) | `< 15.0 µs` (P99) | **`<= 1.2 µs` (P50) / `<= 4.5 µs` (P99)** | `cargo bench --bench ipc_ring` (`Criterion` RDTSC timer) |
  84  | **M-04** | **Tier-0A Deterministic Router Latency** | `18,000 µs` (Python Regex) | `< 5.0 µs` (P99) | **`<= 0.8 µs` (800 ns P99)** | `cargo bench --bench router_tier0a` |
  85  | **M-05** | **Tier-0B Static Embedding Router Latency** (500 Intents) | `45,000 µs` (PyTorch CPU) | `< 100.0 µs` (P99) | **`<= 25.0 µs` (P99)** | `cargo bench --bench router_tier0b` |
  86  | **M-06** | **Tier-0B Classification Accuracy** (Golden 200-Query Set) | `100%` (Full LLM) | `>= 92.0%` F1 Score | **`>= 96.5%` F1 Score** (0% False Executions) | `cargo test --test router_accuracy_eval -- --nocapture` |
  87  | **M-07** | **Data-TLB Load Miss Rate** (During Vector/LLM Sweep) | `4.80%` (4 KB Pages) | `< 0.25%` | **`<= 0.04%`** (via 2 MB THP) | `perf stat -e dTLB-loads,dTLB-load-misses ./laya-bench` |
  88  | **M-08** | **Wasmtime Subagent Cold-Start Latency** | `1,450,000 µs` (1.45s Docker) | `< 2,000 µs` (2 ms) | **`<= 350 µs`** (Pre-compiled `.cwasm` AOT) | `cargo bench --bench wasm_sandbox` |
  89  | **M-09** | **Wasmtime Per-Instance RAM Overhead** | `410 MB` (Docker + Python) | `< 5.0 MB` | **`<= 1.8 MB`** | `/proc/<pid>/smaps_rollup` (`Pss_Anon` delta) |
  90  | **M-10** | **Token Generation Jitter under 100% Background Load** | `+340%` Inter-Token Latency spike | `< +15%` ITL spike | **`<= +3.5%` ITL spike** | Run `stress-ng --cpu 8 --io 4` (`SCHED_IDLE`) during LLM bench |
  91  | **M-11** | **ICSR-V Graph+Vector 2-Hop Hybrid Lookup** (100k Nodes) | `62.0 ms` (SQLite + Python) | `< 3.5 ms` (P99) | **`<= 0.85 ms` (850 µs P99)** | `cargo bench --bench icsrv_hybrid` |
  92  | **M-12** | **OOM Survival Under 11.0 GB Synthetic Allocation** | Kernel Freeze / SSH Drop | Graceful Reject (`ENOMEM` inside slice) | **Zero SSH drop; `laya-core` survives in < 10ms** | `systemd-run --slice=agent.slice stress-ng --vm-bytes 6G` |
  93  
  94  ---
  95  
  96  ## 5. Phase-by-Phase Build & Test Implementation Plan
  97  
  98  ### PHASE 1: Host OS Pruning, Memory Hierarchy & Cgroup v2 Isolation
  99  * Purge desktop environments (`gdm3`, `gnome`, `snapd`).
 100  * Switch default target to `multi-user.target`.
 101  * Enable `MGLRU` and Transparent HugePages (`madvise`).
 102  * Configure 1.2 GB physical-capped `ZRAM` with `zstd` compression.
 103  * Create isolated `laya.slice` (`MemoryMin=4500M`, `CPUWeight=10000`) and `agent-sandbox.slice`.
 104  
 105  ### PHASE 2: Lockless 64-Byte Cache-Aligned Shared Memory IPC
 106  * Implement `LayaIpcDescriptor` strictly cache-line aligned (`#[repr(C, align(64))]`).
 107  * Single 64 MB arena at `/dev/shm/laya_arena` using `memfd_create` and `futex_waitv`.
 108  * Zero allocation during hot-path message transmission.
 109  
 110  ### PHASE 3: Tier-0A (SIMD) & Tier-0B (Int8 Static Embedding) Router
 111  * Tier-0A: `daachorse` SIMD Aho-Corasick automaton (< 2 µs).
 112  * Tier-0B: Static Int8 quantized embeddings (`Model2Vec` / AVX2 dot-product, < 25 µs).
 113  * Route ambiguous queries ($\tau < 0.82$) to Tier-1 SLM.
 114  
 115  ### PHASE 4: Tier-1 MoE/SLM Engine, ICSR-V Storage & Wasmtime Sandbox
 116  * MoE Model: `Granite-3.0-3B-A800M` or `Qwen2.5-1.5B` Q4_K_M (100% in RAM).
 117  * PagedAttention / `Q8_0` quantized KV-cache (cuts KV RAM by 50%).
 118  * Pre-allocated Wasmtime pooling allocator for subagents (< 350 µs cold start, < 2 MB RAM).
 119  
 120  ### PHASE 5: `sched_ext` CPU Isolation & `/dev/tty1` Native TUI Surface
 121  * Pin hardware IRQs to Core 0 (`irqaffinity=0`).
 122  * Pin inference threads to Cores 1–7 using `scx_lavd` (Latency-Criticality Aware Virtual Deadline).
 123  * Native Ratatui interactive TUI bound to `/dev/tty1` (4 MB RAM).
 124