Designing NWF: a feed format 62% smaller than JSON Feed
NWF is Neurowire's native feed format, roughly 62% smaller than the equivalent JSON Feed yet fully round-trippable, running interning, relative links, and date deltas. Lean chrome for any feed rig, choom.
NWF (Neurowire Feed) is Neurowire’s native format, and it lands roughly 62% smaller than the equivalent JSON Feed while staying fully round-trippable back to the canonical model. It gets there with three moves any netrunner would respect: interning repeated values, relativising links against a shared base prefix, and storing dates as deltas from the feed’s update time. The wire format is line-oriented, with LF-separated lines, TAB-separated cells, and a single-letter kind per line. Stripped-down chrome, no wasted bytes, choom, the kind of rig a fixer keeps lean on purpose.
What does the wire format actually look like?
NWF is deliberately boring on the surface, and that is the whole play. Each line is one record. The first character of the line is its kind: a feed header, an interned value, an item, a link. Within a line, cells are split by TAB. Lines are split by LF. That is the whole grammar, choom. The media type is text/x-neurowire and the extension is .nwf.
This line orientation is not just for compactness. It means a parser can stream the file one line at a time, and a netrunner debugging a feed can read it raw without a JSON pretty-printer. The format is compact for machines and still legible for the merc staring at it at 2am with a cold cup of synth-coffee. The full spec sits alongside the rest of the Neurowire docs, and the docs launch post covers where it fits in the engine.
How does it get 62% smaller?
JSON Feed is verbose because it repeats itself like a gonk telling the same story twice. The same author shows on every item. The same tag string gets written out dozens of times. Every link hauls the full origin. Every date is a complete ISO timestamp. NWF strips that redundancy with three moves, clean as a ripperdoc’s cut.
- Interning. Authors, tags, and sources are listed once near the top of the feed and then referenced by index. If one author wrote forty items, their name shows once, not forty times. That is eddies saved on every byte, choom.
- Relative links. Links are stored against a shared base prefix declared once for the feed. An item link becomes the short tail after the common origin, not the whole URL. No merc hauls the full address around twice.
- Date deltas. Each item date is stored as a delta from the feed’s update time, not a full timestamp. Recent items shrink to tiny integers, lean as street chrome.
None of these are lossy, no data flatlines on the way through. The base prefix, the interning table, and the feed update time are all in the file, so a parser rebuilds the exact original model. Round-trip in, round-trip out, byte-for-byte on the model, choom.
The savings hold as feeds grow. Measured with the @neurowire/core serializers on a representative single-source feed, NWF stays around two thirds smaller from ten items up to a thousand. One honest caveat, straight from the crew: heavily repeated single-source feeds compress the most (around 67 to 69%), because the interning table earns its keep when the same author and tags recur on every item. Mixed multi-source feeds save closer to 50 to 54%, since there is less repetition to fold away. The roughly 62% figure quoted elsewhere is a typical midpoint between those two cases. You can watch NWF run across real feeds in the Neurowire App, choom.
When should you reach for NWF over JSON Feed?
If you are publishing to clients you do not control, keep emitting JSON Feed 1.1. NWF is for the cases where size and round-trip fidelity matter: syncing feeds between machines, caching large merged feeds, or storing the output of a mesh that pulled hundreds of items together. Because the canonical model is the contract, you are never locked in by any corpo format, choom. The same feed serializes to both, and Neurowire jacks NWF straight back into the model the project is built around, no data lost in the wire, no netrunner needed to pry it loose. Preem either way, your call which one you jack in.
The format lives in @neurowire/core with the other serializers. If you want to eyeball real NWF before reading the spec, choom, this site publishes its own feed in it. Preem proof, running live.