Why Open Quest Format froze its file format at v0.1 while the packages keep changing

Open Quest Format has two version numbers, and they measure different things. The packages are at 0.1.0, which is what you would expect from a young project. The files, on the other hand, are versioned as OQF1, the four bytes on the first line of every compact quest, and I froze that version in the first release, before any game had shipped with the format.

I know that looks backwards. The usual advice is to settle the API first, run it in production for a while, and only then start making promises about the data. For a file format I think that order is wrong, so OQF is built on the opposite bet: the files get a stability promise from the first release, and the code keeps the freedom to change while it matures.

Two version numbers because there are two kinds of user

The packages are used by people who write code. If @oqf/runtime renames a method in 0.2, the person using it fixes an import, reruns the type checker and gets on with their day, because the tooling tells them exactly what broke. The README is explicit about this: the TypeScript API may change between minor versions while it settles, so if that matters to you, you should pin a minor range.

The format is used by people who write quests, and that is a very different situation. A quest file can represent hours of somebody’s writing. It sits in a repo, later in a game’s data folder, and maybe gets copied into an engine project by someone who has never run npm install in their life. If the format changes under that file, there is no type checker to warn anyone. The file either stops loading or, which is worse, it loads but reads the wrong values, and nobody notices until something behaves strangely in the game.

So I gave the two numbers different rules. OQF1 only changes when a file written today would stop parsing. The packages (core, runtime, dialogue, i18n, cli, examples and the editor) all share one version, and pnpm version:set bumps them together. I did that because I didn’t want anybody to have to work out which @oqf/core pairs with which @oqf/runtime: if the numbers match, you know that set of packages was tested together.

If I had tied the format version and the package version together, I would have ended up with one of two bad outcomes. Either every API cleanup during 0.x would look like a format break, which would make users think their files were at risk when they weren’t, or every promise about the format would also lock in an API I’m still not happy with. Keeping the two numbers separate means I don’t have to accept either of those.

What “frozen” means for a TAB-separated file

The compact format is line oriented, in the same family as NWF. Each line is a list of TAB-separated cells, the first cell names the record type, and the column order for every record type is written down in the spec: docs/12-compact-spec.md for quests and docs/13-dialogue-spec.md for dialogue. On top of that sits a single rule, and it is what freezing actually means here: new columns can be appended to the end of a record, but existing columns are never reordered and never removed.

For that promise to hold in practice, the parser has to follow two rules of its own. A record with fewer cells than the spec lists is valid, because it was produced by an older writer that didn’t know about the newer columns. A record with more cells than the spec lists is also valid, because it was produced by a newer writer, and the parser keeps those extra cells in the extension bag under x-oqf.extra so that they survive a round trip instead of being dropped.

I wanted to see the second rule work on a real fixture, since a unit test’s toy input can hide problems that real files would expose. So I took templates/linear.oqf from the examples package and appended three cells to every S line: two empty ones, then 2h, which stands in for a column that doesn’t exist yet. Then I ran the file through the built 0.1.0 parseCompact and toCompact. The output was byte-identical to the input, and the first step’s ext came back as {"x-oqf.extra":"\t\t2h"}. The 0.1.0 tool has no idea what that extra cell means, and it doesn’t need to, because all it has to do is keep the cell and write it back out unchanged.

The magic line is the one place where the parser is allowed to refuse a file. If you change it to OQF2, the same parser stops on line 1 with expected the magic line "OQF1", got "OQF2". That is the kind of failure I want when a file really is incompatible, because the error is explicit and it happens before anything has been half-loaded.

The column changes I made before the freeze

docs/06-formats.md still holds the sketch I wrote while I was planning the format. If you put it next to the frozen spec, you can see that three records changed shape in the middle. The Q header gained unlock between tagRefs and title. The R reward line gained outcome between amount and payload, because the sketch had left quest-level rewards as “exact encoding to be settled”. And the S step line gained actorRefs just before title.

Every one of those changes is an insertion, which is exactly the move the rule now forbids, and that is the reason I froze at v0.1 instead of waiting for v1.0. While the format was still a sketch, I could shuffle columns freely, because nothing outside my own checkout depended on their order. Once 0.1.0 was published, an insertion would break files written by other people, so the cheapest moment to lock the order was right then, before anyone else had written files against it.

How the planned v2 features fit the append-only rule

The roadmap lists four v2 items under the line “planned for now so v1 files stay valid when they land”. A claim like that is easy to write and hard to check, so here is how each of the four fits without touching the order of existing columns.

Timers are the first one, and at the moment the clock for them lives in the engine. The failure template, for example, closes a sale before nightfall with a fail cell of event world.nightfall, which means the game emits that event and the step fails. v2 adds a deadline on steps and var.time.* in conditions. A deadline is a new step column, so it goes on the end of the record like any other new column. The condition half already parses today: var.time.hour >= 18 comes out of the 0.1.0 parser as {">=":[{"var":"var.time.hour"},18]}, because var.* is the namespace the game supplies and the validator already treats its contents as unknown.

Factions are mostly there already, since reputation exists as a reward kind with an opaque ref, and the reference quest grants reputation harbor 10 on the returned outcome and -10 on sold. What v2 adds is a faction registry and standing thresholds exposed as variables. A threshold is just a condition that reads a variable, and conditions can already read variables today, which means thresholds fit into what the format can express now.

Radiant quests get a template header with parameters like $target and $location, followed by a substitution pass before load. The runtime never sees a template, because what it loads is the output of that pass, which is an ordinary OQF1 quest. From the runtime’s point of view, nothing about the file format changes.

Tabletop export (printable quest sheets, a Foundry export) is an exporter. It reads the model and writes some other format, so the quest file itself never changes.

None of the four needs a new cell in the middle of a record. I checked that before freezing the format, and it is the reason a v2 list can exist at all while the project is still at 0.1.0.

What’s incoming

Append-only covers the order of cells in the compact format. There are two pieces that sit outside that rule, and both are on the way.

The first is forward-compatible JSON reading. The compact parser already tolerates files from a newer writer, but today the JSON parser rejects keys it doesn’t know, and the published schema says additionalProperties: false. To check, I added a deadline key to a step in the JSON form of the same template, and 0.1.0 refused it with unknown key "deadline" at quests[0].steps[0].deadline. So the current situation is that old files load in new tools in both forms, while new files load in old tools only in the compact form. JSON reading that behaves the same way as the compact parser is incoming.

The second is multiple objectives per step, and this one lives inside a cell. Append-only protects the order of cells, but it says nothing about the grammar inside a single cell, so this change needs its own plan. One objective per step is also the only open item in the decisions log. The reference quest wants the trade step to require five silver koi and a talk with the otter, and since v0.1 only holds a single objective, the talk half is handled today through a dialogue binding and a flag. If you type the obvious spelling, collect fish.rare 5 and talk otter otter_trade, 0.1.0 answers with unknown variable namespace "talk". A list of objectives is incoming and lined up as a candidate for v0.2. When it lands it will come with its own column, or with a new magic line that declares the change, and it will leave the current grammar of the complete cell alone so that existing files keep meaning the same thing.

Serializers must reproduce the fixtures byte for byte

There is one rule in CONTRIBUTING that I won’t bend: every serializer has to reproduce the fixtures byte for byte after normalization. packages/examples/quests/ is the shared source of truth, and the round-trip test takes eleven of those fixtures through three checks each. First, the compact form reserializes to the same bytes. Second, the compact and JSON forms parse to the same model. Third, the JSON model serializes back to the same compact bytes.

The consequence of that rule is where people tend to push back. CONTRIBUTING treats any parser change that alters a single byte of output as a breaking change to the format, and it requires a decision written into docs/00-decisions.md before it can land. That includes changes that look harmless, like writing dictionaries in a different order or picking another canonical spelling for a condition. Both would pass every type check in the repo, and both would still change the format.

The reason this matters is that v1.5 is about engine loaders, starting with Godot, and a GDScript loader can’t import my TypeScript. What it can do is read the same fixtures and compare bytes. If the reference implementation drifted by one byte, the two implementations would disagree about what a quest file is, and the whole purpose of a shared format is that every implementation agrees on exactly that.

Incoming. The Cozy Coast integration is on its way as the first game to ship with OQF, and engine exporters for Godot, Unity and Unreal will follow in v1.5. Until those land, the only OQF1 files out there are the fixtures in the repo and whatever someone writes after reading the spec. That is exactly why I wanted the rule in place now: at this stage keeping it costs nothing, whereas later every change to the column order would affect real files.