Rendering markdown & diagrams...
1 # NeuralOS & Laya-Core: Architecture, Systems Audit & Hardened Reference Specification
2
3 **Document ID:** `NEURALOS-SPEC-v2.1-AUDITED`
4 **Chief Architect:** Shubham Sharma
5 **Systems Architecture & Audit:** Autonomous Kernel & Runtime Engineering
6 **Target Appliance:** Dell Node-01 (Intel Core i5-3337U Ivy Bridge @ 1.80GHz, 2C/4T, 7,305 MB Available DDR3/4, ADATA 240 GB SATA-III SSD)
7 **Development Host:** Apple Silicon Workstation (M4 Pro, 48 GB Unified Memory)
8 **Date:** November 2026
9 **Methodology:** Strict Test-First Driven Development (TFDD) & Hardware-in-the-Loop (HIL) Gatekeeping
10
11 ---
12
13 ## 1. Executive Summary & Architectural Audit
14
15 In November 2026, an exhaustive kernel-and-hardware audit was conducted on the initial NeuralOS specification. While the vision of bypassing Chromium, Python GIL, and TCP loopback was validated, the audit identified **six critical engineering flaws** where theoretical assumptions collided with Linux kernel mechanics and Dell Node-01's physical hardware constraints.
16
17 This document serves as the **Audited Reference Specification**, incorporating the corrected systems architecture, the Ivy Bridge hardware realities, and a rigorous Test-First Driven Development (TFDD) plan.
18
19 ---
20
21 ## 2. Part 1: The Six Critical Engineering Flaws & Resolutions
22
23 ### Flaw 1: ZRAM + `mlockall()` Deadlock & Quantization Entropy Trap
24 * **The Theoretical Flaw:** Splitting 8 GB RAM into 4.5 GB pinned DRAM via `mlockall(MCL_CURRENT | MCL_FUTURE)` while simultaneously configuring a 3.5 GB physical ZRAM backing pool.
25 * **The Kernel Reality:**
26 1. ZRAM does not statically carve out RAM; it dynamically consumes unpinned physical pages as swap fills. If 4.5 GB is pinned and the OS uses 200 MB, only ~2.6 GB remains for heap allocations and ZRAM compressed storage. Passing `MCL_FUTURE` forces every subsequent heap allocation in PID 1 to lock physical frames, triggering an immediate OOM panic.
27 2. Quantized model weights (`Q4_K_M`, `Q8_0`) possess near-maximum Shannon entropy (pseudo-random bitstreams) and are **completely incompressible** (`zstd` achieves ~1.01x, wasting CPU cycles). Only UTF-8 text contexts, JSON payloads, SQLite pages, and sparse FP16 KV-caches compress effectively (2x–3x).
28 * **The Resolution:**
29 * **Never `mlockall()` the entire process.** Only `mlock()` the Tier-0 Static Router (15 MB), the Attention Layer Weights, and the Active KV-Cache.
30 * Map Transformer FFN weights via `mmap(MAP_POPULATE)` with `madvise(MADV_WILLNEED | MADV_HUGEPAGE)`. This allows the kernel's **Multi-Generational LRU (MGLRU)** to gracefully evict cold FFN pages under memory pressure instead of OOM killing the machine.
31 * Cap ZRAM at **1.2 GB physical RAM** (2.4 GB uncompressed) dedicated strictly to anonymous heap, context strings, and tmpfs.
32
33 ---
34
35 ### Flaw 2: The "LLM in a Flash" Sparsity Fallacy on Modern Models
36 * **The Theoretical Flaw:** Relying on Apple's *LLM in a Flash* (Alizadeh et al., 2023) to stream 60–80% sparse FFN weights dynamically from the ADATA 240 GB SSD via `io_uring`.
37 * **The Hardware Reality:**
38 1. **ReLU vs. SwiGLU:** Apple's paper evaluated older ReLU-based models (OPT, Falcon) where activations produce exact `0.0` values (80% sparsity). Modern architectures (**Granite, Llama-3, Qwen-2.5, Gemma-2**) use **SwiGLU / GeGLU** activations with **0% exact zero sparsity**.
39 2. **SATA-III Bottleneck:** Dell Node-01 runs over a **SATA-III bus (max ~520 MB/s sequential, ~150 MB/s random 128 KB blocks)**—not PCIe Gen4 NVMe (7,000 MB/s). Streaming even 30% of a 4 GB model per token requires reading ~1.2 GB of weights per token, capping generation at **~0.4 tokens/second** and rapidly degrading the ADATA TLC NAND endurance.
40 * **The Resolution:**
41 * **Abandon Flash Weight Streaming on SATA.**
42 * Use **Fine-Grained Mixture-of-Experts (MoE)** or compact SLMs:
43 * **`Granite-3.0-3B-A800M-Instruct-Q4_K_M`** (~1.9 GB total file size): Fits **100% inside physical DRAM** on Dell Node-01 with zero SSD streaming bottleneck, activating only **800M parameters per token**.
44 * Alternative: **`Qwen2.5-1.5B-Instruct-Q4_K_M`** (~1.1 GB total file size).
45 * Quantize the KV-cache to **`Q8_0`** in `llama.cpp` (`type_k = GGML_TYPE_Q8_0`, `type_v = GGML_TYPE_Q8_0`), cutting KV memory consumption by **50% with zero perplexity loss**.
46
47 ---
48
49 ### Flaw 3: `/dev/laya` Character Driver vs. `shm_open` Conflation & Cache-Line Alignment
50 * **The Theoretical Flaw:** Writing an out-of-tree character driver (`/dev/laya`) using `char payload[512]` with `__attribute__((packed))`.
51 * **The Kernel Reality:**
52 1. An out-of-tree kernel driver introduces severe maintenance debt. Zero-copy IPC can be achieved entirely in userspace via **`memfd_create` / `/dev/shm` lockless SPSC/MPSC ring buffers** paired with **`futex_waitv`** or `eventfd`.
53 2. `__attribute__((packed))` forces unaligned memory accesses on x86_64, breaking atomic operations (`std::sync::atomic` in Rust) on fields crossing 64-byte cache lines. A 512-byte fixed payload cannot hold RAG context chunks without fragmentation.
54 * **The Resolution:**
55 * Drop the custom kernel driver. Create a 64 MB arena at `/dev/shm/laya_arena` backed by 2 MB HugePages.
56 * Use strictly **64-byte cache-line-aligned descriptors** (`#[repr(C, align(64))]`) separating producer and consumer atomic indices onto isolated cache lines to prevent false sharing.
57 * Large payloads reside in data slabs; the 64-byte descriptor only transfers `arena_offset` and `payload_len`.
58
59 ---
60
61 ### Flaw 4: eBPF Verifier Limits & Syscall Execution Fallacy
62 * **The Theoretical Flaw:** Expecting in-kernel eBPF to parse regex intents and directly spawn processes or execute system commands in < 15 µs.
63 * **The Kernel Reality:** The Linux eBPF verifier strictly prohibits unbounded loops, floating-point math, arbitrary memory mutation, and process creation. eBPF cannot execute system commands from Ring 0; it can only inspect buffers, update maps, or send ring buffer events.
64 * **The Resolution:**
65 * Move the **Fast-Path Classifier to Userspace in Rust**:
66 * **Tier-0A (Deterministic):** SIMD Aho-Corasick (`daachorse` in Rust) exact/prefix matcher (**< 2 µs**).
67 * **Tier-0B (Semantic Fast-Path):** Int8 Static Embeddings (`Model2Vec` / AVX dot-product against 500 pre-normalized intent centroids) (**< 25 µs**). Resolves 80% of natural language intents without a neural network forward pass!
68 * **Tier-1 (Deep Reasoning):** Dispatches ambiguous prompts to the in-memory MoE/SLM.
69
70 ---
71
72 ### Flaw 5: `init=/usr/local/bin/neuralos-core` (PID 1) Fragility
73 * **The Theoretical Flaw:** Launching the complex AI inference engine directly as PID 1.
74 * **The Systems Reality:** If `llama.cpp` segfaults, hits an out-of-bounds pointer, or panics, the Linux kernel halts immediately with `Kernel panic - not syncing: Attempted to kill init!`.
75 * **The Resolution:**
76 * **The Micro-Supervisor Model:**
77 * **PID 1 (`neuralos-init`):** A tiny, rock-solid supervisor (< 1,500 lines of pure Rust, zero unsafe C/FFI) whose sole job is mounting virtual filesystems, reaping zombie processes (`waitpid`), and supervising PID 2.
78 * **PID 2 (`laya-daemon`):** The isolated AI runtime running in a dedicated `cgroup v2` slice (`laya.slice`). If it ever crashes, PID 1 cleanly restarts it in < 200 ms without rebooting the kernel.
79
80 ---
81
82 ### Flaw 6: Core 0 Pinning Contention & Hardware Discovery (Ivy Bridge AVX vs AVX2)
83 * **The Theoretical Flaw:** Pinning Laya to CPU Core 0, passing `hugepages=1G`, and assuming AVX2 instructions.
84 * **The Hardware Reality (Probed on Dell Node-01):**
85 1. **CPU Model:** `Intel(R) Core(TM) i5-3337U CPU @ 1.80GHz` (Ivy Bridge, 2 Cores / 4 Threads).
86 2. **Vector Intrinsics:** Supports **AVX, F16C, SSE4.2**, but **NO AVX2 or FMA**. Compiling with `target-cpu=native` or `x86-64-v3` triggers illegal instruction crashes.
87 3. **Core 0 Contention:** In Linux x86_64, CPU Core 0 handles all hardware IRQs (SATA, Wi-Fi, timers). Pinning inference to Core 0 causes heavy latency jitter.
88 4. **RAM Size:** 7,305 MB available (887 MB reserved by BIOS/Intel HD Graphics 4000). Static 1 GB HugePages cause severe fragmentation.
89 * **The Resolution:**
90 * Compile code targeting **`x86-64-v2 + avx + f16c`** with `GGML_AVX=ON`, `GGML_F16C=ON`, `GGML_AVX2=OFF`.
91 * Use **Transparent HugePages in `madvise` mode (2 MB pages)**.
92 * Pin hardware IRQs to **Core 0 (`irqaffinity=0`)**; pin Laya inference to **Threads 1..3** with tickless kernel mode (`nohz_full=1-3`).
93
94 ---
95
96 ## 3. Part 2: Hardened System Architecture & Memory Budget
97
98 ### Process & Ring Hierarchy
99
100 ```
101 ┌──────────────────────────────────────────────────────────────────────────────┐
102 │ USERSPACE (Ring 3) │
103 │ │
104 │ ┌────────────────────────────────────────────────────────────────────────┐ │
105 │ │ PID 1: neuralos-init (Tiny Rust Supervisor, < 4 MB RAM, Zero FFI) │ │
106 │ │ • Mounts /proc, /sys, /dev, /run, cgroup2; reaps zombies (SIGCHLD) │ │
107 │ └───────────────────────────────────┬────────────────────────────────────┘ │
108 │ │ supervises │
109 │ ┌───────────────────────────────────▼────────────────────────────────────┐ │
110 │ │ PID 2: laya-daemon (Rust Edition 2024, CAP_SYS_NICE, CAP_IPC_LOCK) │ │
111 │ │ • Direct Framebuffer Surface (Ratatui /dev/tty1, 4 MB RAM) │ │
112 │ │ • Tier-0A SIMD Router (daachorse Aho-Corasick, < 2 µs) │ │
113 │ │ • Tier-0B Int8 Static Embeddings (AVX/SSE4.2 dot-product, < 25 µs) │ │
114 │ │ • Tier-1 MoE Engine (Granite-3.0-3B-A800M Q4_K_M, mmap + madvise) │ │
115 │ │ • Embedded Wasmtime Subagent Sandbox Pool (< 350 µs cold start) │ │
116 │ └─────────────────▲────────────────────────────────────▲─────────────────┘ │
117 └────────────────────┼────────────────────────────────────┼────────────────────┘
118 │ Zero-Copy Shared Arena │ io_uring_enter()
119 │ (/dev/shm/laya_arena + futex) │ (Async SATA + Net)
120 ┌────────────────────▼────────────────────────────────────▼────────────────────┐
121 │ LINUX KERNEL 7.0 / 6.12+ (Ring 0) │
122 │ │
123 │ • IRQ Pinning: Core 0 handles SATA, Wi-Fi, timers (irqaffinity=0) │
124 │ • Tickless Cores 1..3: Dedicated to inference (nohz_full=1-3, rcu_nocbs=1-3)│
125 │ • MGLRU (Multi-Gen LRU enabled) + ZRAM (1.2 GB physical cap, zstd) │
126 │ • Transparent HugePages: madvise mode (2 MB pages) │
127 └──────────────────────────────────────────────────────────────────────────────┘
128 ```
129
130 ### Physical Memory Allocation (7,305 MB Available DRAM)
131
132 | Memory Region | Allocation | Management Strategy | Purpose |
133 | :--- | :--- | :--- | :--- |
134 | **Kernel + PID 1 + TTY1 Console** | 160 MB | Fixed Physical DRAM | Base operating system substrate |
135 | **Tier-0 Router + Graph Index** | 240 MB | Pinned (`mlock`) | Instant intent routing (< 25 µs) & ICSR-V index |
136 | **Primary MoE Model Weights** | 1,900 MB | `mmap` + `madvise(MADV_HUGEPAGE)` | Granite-3B-A800M (800M active params per token) |
137 | **Active KV-Cache (Q8_0 Quantized)**| 1,200 MB | Pinned (`mlock`) | 8,192 token context window with zero paging stutter |
138 | **Dynamic Agent Workspace & WASM** | 2,100 MB | Pageable DRAM (`laya.slice`) | Wasmtime pooling allocator for subagents |
139 | **ZRAM Backing Pool** | 1,200 MB | Capped Physical (`zstd`) | Expands to ~2.4 GB for anonymous heap & context |
140 | **Page Cache / I/O Read-Ahead** | 505 MB | Elastic | SATA disk buffering |
141
142 ---
143
144 ## 4. Part 3: Test-First Driven Development (TFDD) Gatekeeping
145
146 NeuralOS strictly enforces: **No production code may be committed until its Pre-Flight & Red-Gate tests are written and verified on hardware.**
147
148 ### The 4-Stage Gatekeeping Workflow
149
150 ```
151 STAGE 0: Hardware & Kernel Pre-Flight Probe (scripts/00_preflight_buildable.sh)
152 │
153 ▼
154 STAGE 1: Red-Gate Test Harness (Failing property & concurrency tests)
155 │
156 ▼
157 STAGE 2: Minimal Zero-Allocation Production Implementation
158 │
159 ▼
160 STAGE 3: Scoreboard Hardware Benchmark (perf stat, TLB miss rate, P99 latency)
161 ```
162
163 ### Master Benchmark Scoreboard
164
165 | Metric ID | Target Metric | Legacy Baseline | Gate Threshold | Score to Beat | Validation Command |
166 | :--- | :--- | :--- | :--- | :--- | :--- |
167 | **M-01** | Host Idle RAM (No LLM) | 1,850 MB | < 220 MB | **<= 135 MB** | `awk '/MemAvailable/ {print 7305 - int($2/1024)}' /proc/meminfo` |
168 | **M-02** | Available RAM for Models | 4,800 MB | >= 6,800 MB | **>= 7,100 MB** | `awk '/MemAvailable/ {print int($2/1024)}' /proc/meminfo` |
169 | **M-03** | IPC Round-Trip Latency (P99)| 28,400 µs | < 15.0 µs | **<= 4.5 µs** | `cargo bench --bench ipc_ring` |
170 | **M-04** | Tier-0A SIMD Router (P99) | 18,000 µs | < 5.0 µs | **<= 0.8 µs** | `cargo bench --bench router_tier0a` |
171 | **M-05** | Tier-0B Embedding Router | 45,000 µs | < 100.0 µs | **<= 25.0 µs** | `cargo bench --bench router_tier0b` |
172 | **M-06** | Tier-0B F1 Classification | 100% (Full LLM) | >= 92.0% | **>= 96.5%** | `cargo test --test router_accuracy_eval` |
173 | **M-07** | Data-TLB Miss Rate (2MB THP)| 4.80% | < 0.25% | **<= 0.04%** | `perf stat -e dTLB-loads,dTLB-load-misses ./laya-bench` |
174 | **M-08** | Wasmtime Cold-Start Latency | 1,450,000 µs | < 2,000 µs | **<= 350 µs** | `cargo bench --bench wasm_sandbox` |
175 | **M-09** | Wasmtime RAM Overhead | 410 MB (Docker) | < 5.0 MB | **<= 1.8 MB** | `/proc/<pid>/smaps_rollup` (`Pss_Anon`) |
176 | **M-10** | ITL Jitter under 100% Load | +340% spike | < +15% spike | **<= +3.5%** | Run `stress-ng` during token generation |
177
178 ---
179
180 ## 5. Part 4: Phase-by-Phase Execution Plan
181
182 ### Phase 1: Host OS Pruning, MGLRU, ZRAM & Cgroup v2 Isolation (Days 1–2)
183 1. Write `tests/test_phase1_kernel.sh` (asserts headless target, MGLRU 0x0007, THP madvise, ZRAM zstd, idle RAM <= 220 MB).
184 2. Execute `scripts/01_apply_phase1_kernel.sh`:
185 - Purge desktop environments (`gdm3`, `gnome`, `snapd`).
186 - Enable MGLRU: `echo y > /sys/kernel/mm/lru_gen/enabled`.
187 - Set THP to `madvise`: `echo madvise > /sys/kernel/mm/transparent_hugepage/enabled`.
188 - Setup 1.2 GB physical-capped ZRAM with `zstd`.
189 - Create systemd slices `/etc/systemd/system/laya.slice` and `agent-sandbox.slice`.
190 3. Verify Phase 1 Red-to-Green gate.
191
192 ### Phase 2: Lockless 64-Byte Cache-Aligned Shared Memory IPC (Days 3–6)
193 1. Write `crates/laya-ipc/tests/ipc_torture.rs`:
194 - Assert compile-time 64-byte alignment (`size_of == 64`, `align_of == 64`).
195 - Run 10-million packet torture test with zero dropped packets and zero heap allocations.
196 2. Implement `LayaIpcDescriptor` and `memfd_create` arena at `/dev/shm/laya_arena`.
197 3. Implement hybrid adaptive spin-then-futex polling via `futex_waitv`.
198 4. Validate Benchmark M-03 (P99 <= 4.5 µs).
199
200 ### Phase 3: Tier-0A (SIMD) & Tier-0B (Int8 Static Embedding) Router (Days 7–10)
201 1. Create `tests/golden_intents_200.jsonl` (200 labeled real-world prompts across 4 categories).
202 2. Implement Tier-0A using `daachorse` (SIMD Double-Array Aho-Corasick).
203 3. Implement Tier-0B using distilled static embeddings (`Model2Vec` quantized to Int8, AVX/SSE4.2 dot-product).
204 4. Validate Benchmark M-04 (P99 <= 0.8 µs), M-05 (P99 <= 25.0 µs), and M-06 (F1 >= 96.5%).
205
206 ### Phase 4: Tier-1 MoE/SLM Engine, ICSR-V Storage & Wasmtime Sandbox (Days 11–16)
207 1. Setup `llama.cpp` Rust bindings with Ivy Bridge flags: `GGML_AVX=ON`, `GGML_F16C=ON`, `GGML_AVX2=OFF`.
208 2. Load `Granite-3.0-3B-A800M-Instruct-Q4_K_M.gguf` with `Q8_0` quantized KV-cache.
209 3. Configure Wasmtime pooling allocator (`PoolingAllocationConfig`, 16 pre-allocated slots).
210 4. Validate Benchmark M-08 (Cold start <= 350 µs) and M-09 (RAM <= 1.8 MB).
211
212 ### Phase 5: Hardware IRQ Isolation & `/dev/tty1` Native TUI Surface (Days 17–20)
213 1. Route hardware IRQs to Core 0 (`irqaffinity=0`) and isolate Threads 1..3 for inference.
214 2. Build `laya-tui` in Rust using `ratatui` + `crossterm`.
215 3. Bind to `/dev/tty1` as replacement for `getty@tty1.service` (boots to cockpit in 2.8s, 4 MB RAM).
216 4. Run master test suite `scripts/run_scoreboard.sh` to produce final verified scorecard.
217