OpenDLSS: A Vulkan Reimplementation of Nvidia's DLSS 5 Neural Rendering Network
A developer has reverse-engineered NVIDIA's proprietary DLSS 5 Neural Rendering network, creating OpenDLSS, a bit-exact Vulkan reimplementation. This technical deep dive into generative neural rendering offers an open-source alternative, intriguing the HN crowd with its precision and potential for broader accessibility to advanced graphics tech.
The Lowdown
OpenDLSS is a remarkable open-source project that meticulously reimplements NVIDIA's DLSS 5 Neural Rendering (NR) network in Vulkan, achieving bit-exact parity with the original. This project provides a transparent and accessible alternative to a powerful, proprietary graphics technology.
- Bit-Exact Reproduction: OpenDLSS precisely mirrors NVIDIA's DLSS 5 NR build 310.8.0, utilizing the same 71-block Swin / ViT network and running FP8 on tensor cores. Crucially, it matches all 75 block boundaries byte for byte, ensuring identical intermediate results.
- Generative Neural Rendering: Unlike traditional upscaling, DLSS-NR re-renders existing frames, injecting detail from noise and adjusting elements like tone, structure, and skin, based on a style setting. It takes a rendered frame, Gaussian noise, and conditioning scalars to produce an RGB residual and a temporal-blend logit.
- Technical Implementations: The project features a Vulkan implementation with optimized GLSL and PTX kernels, leveraging cooperative-matrix operations, FP8 GEMMs, and advanced chaining techniques. It also includes a separate WebGPU port, which runs the same network in a browser without tensor cores or FP8, albeit slower.
- Performance Metrics: On an RTX 4070 SUPER, the network processes a 1080p frame in approximately 7.8 ms and a 4K frame in 29.3 ms.
- User-Supplied Weights: Users are required to supply their own model weights, as the repository contains no NVIDIA software or intellectual property.
- Validation Tools: The project includes
parityandverifytools to confirm bit-exactness against recorded captures of the original DLSS-NR output. - System Requirements: It requires Windows, an NVIDIA Ada or newer GPU with specific Vulkan extensions, Visual Studio 2022+, Python, and Node.js.
This project stands out as a significant reverse-engineering effort, offering a deep, verifiable insight into a complex neural rendering pipeline and democratizing access to the underlying technology through an open-source, non-affiliated implementation.