Neurowire: an open feed engine, now with full docs

Neurowire is an open-source feed engine: it takes content from blogs, RSS, Atom, or even websites with no feed at all, normalizes it into one canonical model, and serializes that model to whatever format you need. As of today the full documentation site is live at neurowire.starside.io, covering the concepts, the formats, and the API. This post is the short version of why it exists and what it does.

The problem: the open web forgot about feeds

Feeds were how the web stayed interoperable. You published once, and anything could read it. That quietly eroded. Plenty of sites ship no RSS or Atom at all, the formats that survive are verbose, and “follow us” increasingly means an account on someone else’s platform. If you want to actually own what you read, you end up writing the same brittle scraper twice and reinventing merge-and-dedupe logic every time.

Neurowire treats that as a data-modeling problem, not a scraping problem. One canonical feed model sits in the middle. Everything reads from it, everything writes to it.

One model, many formats

Every parser produces the same canonical feed, and every serializer reads it. That single decision is what makes the rest composable. From one model, Neurowire emits:

Atom and RSS 2.0, for anything that already speaks feeds
JSON Feed 1.1, for modern clients
Markdown, for a readable digest
OPML, for moving subscriptions around
Self-contained HTML pages
NWF, its own compact native format

Because the model is the contract, adding a new output is writing one serializer, not touching every parser.

NWF: small on purpose

NWF (Neurowire Feed) is the native format, and it is roughly 62% smaller than the equivalent JSON Feed while staying fully round-trippable back to the model. It gets there with three tricks: it interns repeated values (authors, tags, sources are listed once and referenced by index), it relativizes links against a shared base prefix, and it stores dates as deltas from the feed’s update time instead of full timestamps. The wire format is line-oriented (LF-separated lines, TAB-separated cells, a one-letter kind per line), media type text/x-neurowire, extension .nwf. Compact for machines, still legible for humans debugging it.

Feeds for feed-less sites

The piece I like most is taps: per-host, CSS-selector recipes that turn a plain HTML listing page into a real feed. A site with no RSS becomes a first-class source. That is the difference between “we support feeds” and “we make anything into a feed.”

On top of sources, two concepts compose:

Meshes fetch multiple sources in parallel and merge them into one feed.
Constructs bundle multiple meshes into a single repository, either grouped or flattened.

So you go from scattered sources, to a merged topic feed, to a curated bundle, without leaving the model.

The packages

Neurowire ships as focused packages rather than one monolith:

@neurowire/core: the canonical model, the serializers, and the format helpers
@neurowire/ingest: fetching and parsing sources
@neurowire/cli: the command-line interface
@neurowire/web: the HTTP API and web layer

Delivery is not limited to files either: feeds can be pushed to Slack, Discord, webhooks, or served over the HTTP API.

Dogfooded here

This site uses @neurowire/core to publish its own writing feeds. The Atom, JSON Feed, and NWF links in the footer are all serialized from the same canonical model the docs describe. If you want to see the output before reading the spec, those are real.

Read the docs

The full documentation is live at neurowire.starside.io: getting started, concepts, the format specs, and the API reference. The code lives under Starside Labs on GitHub. If you have ever wanted to own your reading instead of renting it from a platform, this is the toolkit for it.

More on the project itself on the Neurowire project page, and more open-source work under projects.