The NX bit is not just about security
Sonya recounts a months-long debugging odyssey on an ARM64 bare-metal hypervisor for postmarketOS, where enabling a CTR_EL0 intercept caused random phone lock-ups. The complex saga involved systematically eliminating hypotheses like MRS emulation errors, out-of-spec hardware, and kernel panics, ultimately narrowing the issue to a subtle difference between static and dynamic branch calls. The breakthrough revealed that the Non-Executable (NX) bit, typically a security feature, unexpectedly prevented speculative instruction fetches from MMIO regions, solving a baffling problem and showcasing a deep architectural quirk.
The Lowdown
This story details Sonya's arduous, months-long debugging journey while developing a bare-metal hypervisor on ARM64 for postmarketOS. The core problem manifested as random phone lock-ups immediately after enabling a CTR_EL0 intercept, a feature crucial for the hypervisor's function. The experience illustrates the intricate challenges of low-level ARM system development and the unexpected behaviors one can encounter with hardware and memory management.
- Initial investigations focused on verifying the correctness of
MRS/MSRinstruction emulation and stack handling within the exception handler, both of which appeared sound. - A key early hurdle was ARM's lack of Icache/Dcache coherence, which meant dynamically sorting an array of executable instructions at runtime led to issues; moving this to a build-time step resolved it.
- Hypotheses about out-of-spec hardware or kernel panics were explored through extensive logging and analyzing crash logs, but provided no definitive answers.
- Manual patching of
ctr_el0instructions in the kernel and rewriting specific handlers in assembly helped isolate the problem to the C handler's execution. - Binary bisecting of the C code revealed the subtle but critical difference between a static
bl get_ctr_el0call and a dynamicblr x0call; the dynamic call, relying on branch prediction, was the culprit. - The root cause was uncovered: speculative instruction fetches from the
0x0address, which in the 1:1 memory mapping corresponded to an unmapped bootrom region. - The ultimate solution, confirmed by ARM documentation, was to mark the relevant MMIO pages as non-executable, preventing these speculative instruction fetches.
The debugging saga concludes with the surprising revelation that the Non-Executable (NX) bit, often seen purely as a security mechanism to prevent code execution from data segments, plays an equally vital role on ARM in preventing microarchitectural speculative instruction fetches from device memory. This highlights a nuanced aspect of ARM architecture where memory attributes serve functions beyond their typical security implications.