The “small web” is bigger than you might think
The author investigates the true scale of the "small web" – non-commercial, personal sites free of ads and tracking – discovering it's far more active than its name suggests. Inspired by aggregated feeds in the tiny Gemini protocol, he attempts to build a similar aggregator for the regular web's indie corners. The unexpected volume of daily updates reveals a thriving, vibrant ecosystem, challenging preconceived notions about the internet's non-corporate spaces.
The Lowdown
Kevin Boone, inspired by the aggregated feeds of the small-scale Gemini protocol, set out to quantify and potentially aggregate content from the "small web" – a collection of personal, non-commercial websites eschewing ads and corporate tracking. He initially hypothesized that, like Gemini, the small web would be manageable enough for a single feed aggregator.
- Boone leveraged Kagi's "smallweb" initiative list, which had grown from 6,000 to 32,000 sites.
- He filtered this list, requiring valid update feeds with timestamps, and then further culled sites with less than one update per month.
- This process yielded approximately 9,000 active small web sites, which collectively produced over 1,200 new content updates on a typical day.
- Despite the "small" moniker, this volume of activity proved too large for the single-page aggregation Boone initially envisioned, demonstrating a vigorous and growing independent web presence.
Ultimately, Boone concludes that while his aggregation project proved impractical due to scale, the discovery itself is a positive one: the non-commercial web is thriving. Its definition lies not in its size, but in its underlying ethos of personal expression over corporate monetization.
The Gossip
Defining the Indie Web Spirit
Commenters debated the core essence of the 'small web' and similar movements like the 'Indie Web' and Gemini. Some emphasize it as a 'mindset' focused on sharing content for its own sake, free from optimization for attention or monetization. Others argue it's about reclaiming the 'non-commerciality' of the early internet. However, one critique posits that these movements 'miss the point' if they only focus on retro aesthetics or tech, suggesting the original 90s web attracted people through experimentation, diverse content, and a free-spirited social dynamic that modern initiatives need to replicate, rather than just old forms. The Fediverse is also cited as another example of a small web 'mindset'.
Search Engine Squelch & Discoverability
A significant thread of discussion revolved around the role of search engines, particularly Google, in burying the small web. Many feel Google's algorithms now prioritize monetization over genuine usefulness, making independent blogs and personal sites difficult to find. Marginalia Search is frequently highlighted as a promising alternative, explicitly designed to unearth human-written content and act as a public good. Conversely, some argue that Google *does* give users what they want, contending that 'usefulness' often implies commercial intent, and that the average user now prefers 'professional-looking' (even if AI-generated) content over plain, old-school blogs.
Nostalgia vs. Novelty in Non-Commercial Spaces
Commenters explored the blend of nostalgia and forward-thinking in seeking alternatives to the commercialized internet. Many acknowledge a yearning for the 'whatever-it-was' of the internet before mass exploitation, seeing the small web and Gemini as a reaction to the ad-riddled, corporate-dominated landscape. This isn't just about recreating the past, but building something new that prioritizes content creation over endless scrolling and ad consumption. The desire for non-commercial spaces is strong, leading to inquiries about sustainable models like 'free-but-subscriber-only' blogs, highlighting the ongoing tension between passion projects and financial viability.