Designing NWF: a feed format 62% smaller than JSON Feed
NWF is Neurowire's native feed format, roughly 62% smaller than the equivalent JSON Feed yet fully round-trippable, using interning, relative links, and date deltas.
NWF (Neurowire Feed) is Neurowire’s native format, and it lands roughly 62% smaller than the equivalent JSON Feed while staying fully round-trippable back to the canonical model. It gets there with three techniques: interning repeated values, relativising links against a shared base prefix, and storing dates as deltas from the feed’s update time. The wire format is line-oriented, with LF-separated lines, TAB-separated cells, and a single-letter kind per line.
What does the wire format actually look like?
NWF is deliberately boring on the surface. Each line is one record. The first character of the line is its kind: a feed header, an interned value, an item, a link. Within a line, cells are separated by TAB. Lines are separated by LF. That is the whole grammar. The media type is text/x-neurowire and the extension is .nwf.
This line orientation is not just for compactness. It means a parser can stream the file one line at a time, and a human debugging a feed can read it without a JSON pretty-printer. The format is compact for machines and still legible for the person staring at it at 2am. The full spec sits alongside the rest of the Neurowire docs, and the docs launch post covers where it fits in the engine.
How does it get 62% smaller?
JSON Feed is verbose because it repeats itself. The same author appears on every item. The same tag string is written out dozens of times. Every link carries the full origin. Every date is a complete ISO timestamp. NWF removes that redundancy with three techniques.
- Interning. Authors, tags, and sources are listed once near the top of the feed and then referenced by index. If one author wrote forty items, their name appears once, not forty times.
- Relative links. Links are stored against a shared base prefix declared once for the feed. An item link becomes the short tail after the common origin, not the whole URL.
- Date deltas. Each item date is stored as a delta from the feed’s update time, not a full timestamp. Recent items become tiny integers.
None of these are lossy. The base prefix, the interning table, and the feed update time are all in the file, so a parser reconstructs the exact original model. Round-trip in, round-trip out, byte-for-byte on the model.
The savings hold as feeds grow. Measured with the @neurowire/core serializers on a representative single-source feed, NWF stays around two thirds smaller from ten items up to a thousand. One honest caveat: heavily repeated single-source feeds compress the most (around 67 to 69%), because the interning table earns its keep when the same author and tags recur on every item. Mixed multi-source feeds save closer to 50 to 54%, since there is less repetition to fold away. The roughly 62% figure quoted elsewhere is a typical midpoint between those two cases. You can see NWF in action across real feeds in the Neurowire App.
When should you reach for NWF over JSON Feed?
If you are publishing to clients you do not control, keep emitting JSON Feed 1.1. NWF is for the cases where size and round-trip fidelity matter: syncing feeds between machines, caching large merged feeds, or storing the output of a mesh that pulled hundreds of items together. Because the canonical model is the contract, you are never locked in. The same feed serializes to both, and Neurowire reads NWF straight back into the model the project is built around.
The format lives in @neurowire/core with the other serializers. If you want to see real NWF before reading the spec, this site publishes its own feed in it.