OpenJev
OpenJev brings sophisticated decision modeling to your browser, running local AI directly on your device. This experiment, inspired by the proprietary Jev, sparks debate on open-source emulation and client-side AI performance. HN is captivated by the technical ingenuity and the implications of powerful AI models running directly on consumer hardware.
The Lowdown
OpenJev is a fascinating web-based experiment demonstrating the feasibility of running AI-powered decision models entirely within your browser, without any backend servers. It aims to replicate the core functionality of Jev, a system known for its specific approach to providing choices and options, by leveraging small, local language models.
Key aspects of this browser-based AI demonstration include:
- Local Execution: The entire process, from model loading to inference, occurs client-side, ensuring user privacy as inputs never leave the page.
- Method Comparison: It showcases two distinct methods for obtaining decision probabilities: directly reading model logits or prompting the model to generate probabilities in JSON format.
- Optimized Models: It utilizes pre-quantized, smaller models like MiniCPM5 2B (downloaded from Hugging Face) to minimize resource usage, though loading times can vary.
- Real-time Performance Metrics: The tool measures actual wall-time performance on the user's GPU, highlighting the speed differences between the two approaches.
OpenJev pushes the boundaries of what's possible with in-browser AI, offering a hands-on look at how advanced AI decision-making can be democratized and made instantly accessible.
The Gossip
Jev Jousting: Naming & Novelty
Commenters immediately pounced on the 'Jev' in OpenJev, questioning its relation (or lack thereof) to the original Jev project. The discussion veered into whether the core 'Jev-like' approach—constraining output and using parallel prompts—is truly novel or merely an application-layer trick executable with any small LLM. Some felt the naming was potentially misleading, while others saw it as a legitimate open-source attempt to replicate and explore a cutting-edge concept.
Performance Ponderings & Practical Pits
Users shared their experiences with OpenJev's performance, noting varying speeds on different hardware (e.g., M2 Max) and acknowledging that while it's not 'super fast,' it's quicker than typical LLM interactions. A practical snag emerged for some, with corporate firewalls blocking access to Hugging Face, preventing model downloads and sparking a wish for mirrored repositories.
Choice Conundrums: Scoring & Certainty
The method by which OpenJev assigns probabilities to choices, like 'Foo' scoring 98%, raised questions. Users wondered why a model would assign such a high score to one option without a clear basis, and debated the absence of an explicit 'uncertainty' parameter. One commenter humorously pondered the meaning of a 'real choice' versus a 'fake choice' in this AI context.
Broader Breathtaking Benefits
Beyond the technical implementation, commenters expressed excitement about the rapid innovation exemplified by OpenJev. Discussions touched upon the 'Jevons Paradox' in relation to potential new use cases, applications, and even job creation stemming from such technologies. The fact that the original Jev was reportedly trained on 100% synthetic data also captured attention, highlighting advancements in AI data generation and its implications for future real-world decision-making.