I turned Jev into a (lousy) chatbot
This project transforms Jev, an AI model, into a character-by-character chatbot, using an array of clever sampling strategies to construct responses one symbol at a time. It's a delightfully 'lousy' yet deeply technical exploration of AI generation mechanics, prompting discussions on model limitations, rapid prototyping, and the peculiar humor of early AI outputs. While fun and experimental, its high cost and unconventional approach spark both amusement and thoughtful technical debate.
The Lowdown
Jevchat is an experimental project that repurposes Jev, an AI model designed for probabilistic predictions, into an interactive chatbot. Instead of generating full tokens or sentences, Jevchat constructs replies symbol-by-symbol, asking Jev a question at each step: "Given the user's question and the reply written so far, which symbol comes next?"
- Jev returns probabilities for various symbols (e.g., letters, words, stop signals), from which the next symbol is drawn.
- The system offers multiple alphabets (e.g., lowercase, ASCII, BPE tokens) and sampling strategies (e.g.,
choice,bisect,buckets,refine) to influence generation. - A key innovation is the "hypothesis" presentation, where Jev evaluates complete potential strings rather than individual symbols, significantly improving output quality.
- Though the results are described as "hilarious" and "lousy" and the cost "impractical," the project serves as a playful yet insightful demonstration of granular AI control.
- The author notes that parts of the implementation were accelerated using Claude.
Ultimately, Jevchat stands as a quirky testament to pushing the boundaries of AI models, demonstrating that even a seemingly inefficient method can yield both entertaining outcomes and valuable technical insights into how models process and generate language.
The Gossip
Laughter and Lousy Language Models
Users are highly amused by the chatbot's often comically literal or nonsensical responses, highlighting specific examples like 'write me a short story' yielding 'a story.' Commenters liken the experience to interacting with a 'death crystal' (a Rick and Morty reference) or an 'ADHD heavy friend,' appreciating the unexpected humor derived from the model's peculiar generation process and limitations.
Jev's Jigsaws: Understanding the Mechanism
Discussion delves into Jev's internal workings and how its symbol-by-symbol prediction differs from traditional LLM token generation. Commenters ponder whether Jev possesses a 'secret sauce' in its well-calibrated probabilities, question its advantage over using embedding models for similarity, and debate if a similar outcome could be achieved by forcing JSON tokens onto conventional LLMs. Some users also admit to not being familiar with Jev itself, prompting clarification.
The Swiftness of Software Spawning
A recurring sentiment is the accelerated pace of AI-related development and idea generation. Multiple users recount having similar 'aha!' moments about turning Jev into a character-based chatbot, only to find others have already implemented and published their versions, sometimes multiple times over. This phenomenon leads to reflections on how quickly novel ideas are now executed and shared, with one commenter observing that 'time from idea to output has dropped off a fucking cliff.'
Nostalgic Nods to Early AI Oddities
The 'lousy' and amusing nature of Jevchat's output sparks nostalgia for the quirky, often flawed, but entertaining generations of earlier AI models. Commenters recall the 'sheer demented horror' of early image models and the 'funny completions' of text-davinci and early ChatGPT, finding a similar charm in Jevchat's less-than-perfect, yet often humorous, attempts at conversation. This harkens back to a time when AI outputs were less polished but often more surprising.