Adding leaderboards and global stats to steamworks.js: what Steam doesn't tell you

Version 0.7.0 of Cozy Steamworks, the steamworks.js fork I maintain for Cozy Coast, added two namespaces, leaderboard and global_stats, and version 0.8.0 added screenshots on top of that. None of the Steamworks calls underneath are long, and a leaderboard upload is a single function call. Where I actually spent my time was on everything around those calls, because some requests fail by never answering at all, and some results look like a success while carrying nothing useful.

This post goes through the problems I ran into one at a time, along with the code I wrote to deal with each of them. Everything I mention is in the repo, mostly in src/api/ and in the scripts under test/.

Leaderboard handles are only valid for one Steam session

When you call findLeaderboard('Feet Traveled'), you get back a Leaderboard object with an id on it, which is a bigint. It looks like exactly the kind of value you would want to cache in a save file, so that the next time the game launches you can skip the lookup.

You should not do that, because the id is a raw SteamLeaderboard_t and it is only valid for the current Steam session. That is why the doc comment on the class tells you to store the name and look the board up again on every run, and why the id field is marked as diagnostic only.

If you ignore that and reuse an old id, what you get is a hang rather than an error. Steam answers a request made against a stale handle with k_uAPICallInvalid, but the steamworks crate registers its callback under that invalid call handle anyway, so the callback never fires. In my first version the promise simply stayed pending forever, and a client that is logged out of Steam produces exactly the same silence. To make sure a caller always gets an answer, every Steam await in the leaderboard and global stats modules now runs under a 15 second tokio timeout, and when the timeout expires the promise rejects with a readable message such as Timed out waiting for Steam after 15s (score upload).

Leaderboard names are also checked before they ever reach Steam, because a few bad inputs cause much worse problems than a clear error would. An empty name is refused, and so is a name longer than 128 bytes (k_cchLeaderboardNameMax), since it would trigger the same k_uAPICallInvalid hang described above. The most dangerous case is a NUL byte in the name: the crate builds a CString from the name and panics, and a panic inside the Steam callback machinery takes the whole process down with it. The binding now rejects that case in Rust, which means you get an error you can catch instead of a crash.

findOrCreateLeaderboard only applies its settings once

findOrCreateLeaderboard(name, sortMethod, displayType) is convenient when you do not want to set a board up by hand, but its arguments do not behave the way you might expect. The sort method and the display type are only applied at the moment the board is created. After that, the board keeps whatever settings it was created with, and later calls that pass different values have no effect on it.

In practice this means that if the very first build that ever ran shipped Descending for a speedrun board, fixing the code afterwards does not fix the board. You have to change it on the Steamworks site, because that is the only place where a board that already exists can be modified. There is a second thing to know about boards created from code, which is that they do not show up in the Steam Community until someone sets a Community Name for them in App Admin. I got that detail wrong in the first version of the doc comment and corrected it in a follow-up commit.

Uploading scores: KeepBest and the rate limit

await board.uploadScore(lb.UploadScoreMethod.KeepBest, 4200, [level, seed])

The first argument decides what happens when the player already has a score on the board. KeepBest keeps the stored score if the new one does not beat it, while ForceUpdate always replaces it. The object the promise resolves to tells you which of the two happened: wasChanged is false when KeepBest discarded your upload, and you also get globalRankNew and globalRankPrevious, so you can show the player how their rank moved.

The promise can also resolve to null, which means Steam reported the upload as unsuccessful. The most common reason for that is the rate limit, which is roughly 10 uploads per 10 minutes per user. The important part is that a throttled upload is not treated as an error. The promise resolves normally with null, so if your code does not check for it, you will assume the score was saved when it was not. The practical rule that follows is to upload when a run ends and never on every tick of a counter. The test script handles null as a skip for this reason, and it only uploads anything when you set LEADERBOARD_UPLOAD=1, because each run of the script uses up one of those ten uploads.

The third argument is the details payload, an array of at most 64 ints that Steam stores alongside the score. If you pass 65, the binding rejects the call before Steam ever sees it. Reading the details back has its own trap, because downloadEntries takes a maxDetails argument that defaults to 0, and with 0 every entry comes back with details: []. The data is still stored on Steam’s side, so when you see empty arrays it usually means you did not ask for the details, and passing a maxDetails value is enough to get them back.

How download ranges work for each request type

// Absolute 1 based ranks.
const top10 = await board.downloadEntries(lb.LeaderboardDataRequest.Global, 1, 10, 2)
// Offsets relative to your own rank.
const around = await board.downloadEntries(lb.LeaderboardDataRequest.GlobalAroundUser, -4, 5, 2)
const friends = await board.downloadEntries(lb.LeaderboardDataRequest.Friends, 0, 0)

downloadEntries always takes the same two numbers, start and end, but what they mean depends on the request type you pass in. For Global, they are absolute ranks counting from 1, so 1, 10 gives you the top ten. For GlobalAroundUser, they are offsets from the local player’s own rank, which is why they are allowed to be negative: -4, 5 returns your own row plus the four entries above it and the five below it. For Friends, Steam ignores the range completely and returns every friend who has an entry on the board. Whatever the request type, a single request is capped at 5000 entries.

Supporting the negative offsets required a small workaround on the Rust side. The steamworks crate at 0.13.1 takes the range as usize, which cannot hold -4, and then casts it down to the SDK’s int before calling Steam. So the binding accepts i32 from JavaScript and does start as usize before the call. The sign extends on the way in and survives the cast back down, so Steam still receives the negative value it expects. It is the only way to express that range through this version of the crate, and I left a comment in leaderboard.rs explaining it, because anyone reading that line without the context would reasonably assume it is a bug. A start greater than end, on the other hand, is rejected outright, since a call like downloadEntries(Global, 10, 1) is almost certainly a mistake.

Global stats run about a day behind

The global_stats namespace reads totals aggregated across every player, and most of the difficulty is in the setup on Steam’s side. A stat can only be read through this API if it is marked as aggregated in the Steamworks App Admin, and Steam only starts counting from the moment you switch that on. On top of that, the totals trail live play by roughly a day. That rules out something like a “fish caught worldwide” counter that ticks up while the player watches, because this API cannot give you numbers that fresh.

// Totals, plus seven days of day-by-day history. Resolve before reading.
await client.global_stats.requestGlobalStats(7)

const total = client.global_stats.getGlobalStatInt64('NumGames')     // bigint or null
const rate = client.global_stats.getGlobalStatDouble('Distance')     // number or null
const daily = client.global_stats.getGlobalStatHistoryInt64('NumGames', 7)  // index 0 is today

Until requestGlobalStats resolves, the getters return null and [], and they keep returning those values when a stat is not aggregated for the app. What surprised me is that the request also resolves when Steam answered it with a failure. Steam does report a per-request EResult, but the crate does not pass it through, so the binding has no way of telling a successful reply from a failed one. All a resolved promise tells you is that Steam replied, not that any data came back, which is why you need to check for null every time you read a stat.

The history array is ordered most recent first, so index 0 is today, and it is only as long as the number of days Steam actually returned. historyDays is clamped to 60 because that is the SDK maximum, and the days value you pass to the history getter should not be larger than what you requested.

Per-user stats work differently from all of this since SDK 1.64. The Steam client now syncs them before the game process starts and the SDK removed RequestCurrentStats altogether, which is why init in the fork no longer calls it.

64-bit values come back as bigints

getGlobalStatInt64 returns a bigint, and so does every value in the history array, a leaderboard’s id, and the steamId64 on each downloaded entry. Two things break the first time you forget about this. JSON.stringify throws a TypeError when it meets a bigint, and an expression like total + 1 throws because JavaScript does not let you mix bigint and number in arithmetic. Because of the first problem, the leaderboard and global stats test scripts both carry the same one-line replacer, which they need in order to print their results at all:

const json = (v) => JSON.stringify(v, (_, x) => typeof x === 'bigint' ? String(x) : x)

Float and average-rate stats are not affected by this, because they go through getGlobalStatDouble and come back as plain numbers.

Adding screenshots without the overlay

The screenshots namespace came later, but it belongs in this post because it has the same kind of problem: it is an async write, and you only learn the result from a callback. addToLibrary(path, width, height) adds an image that the game rendered itself to the player’s Steam screenshot library, without involving the overlay. It accepts JPG, PNG or TGA files, and Steam generates the thumbnail for you.

The path you pass has to be absolute. The crate canonicalizes the path against the working directory, so it would happily accept a relative path and resolve it against wherever the process happened to be launched from. The binding returns null in that case instead, and it does the same for a path containing a NUL byte and for any file that Steam refuses. When you do get a number back, it means Steam accepted the file, but it does not mean the file has been written yet.

const me = client.localplayer.getSteamId().steamId64
const sub = client.callback.register(steamworks.SteamCallback.ScreenshotReady, ({ handle, result }) => {
    if (handle !== shot || result !== 1) return
    client.screenshots.setLocation(shot, 'Deep Sea')
    client.screenshots.tagUser(shot, me)
})
const shot = client.screenshots.addToLibrary(file, 320, 180)

That is why the example waits for ScreenshotReady before calling setLocation or tagUser. The callback carries the raw EResult number, and a value of 1 (k_EResultOK) means the screenshot has been written and can be tagged. The crate does have its own ScreenshotReady type, but it does not derive Serialize, and it reduces the EResult to either Ok or Fail, which throws away the actual code. So the fork reads the two fields directly from memory with offset_of! instead of going through the bindgen EResult enum. I avoided the enum on purpose, because if Steam sent a value that the enum does not list, reading it as that enum would be undefined behaviour inside SteamAPI_RunCallbacks.

Running the test scripts

Each namespace has its own script, node test/leaderboard.js, node test/global_stats.js and node test/screenshots.js, and all three run against Spacewar, app 480, with a Steam client that is logged in. The two that write to a real account are opt in, so that running them never costs anything by accident: LEADERBOARD_UPLOAD=1 enables a real score upload, and SCREENSHOT_UPLOAD=1 puts a generated gradient PNG into your actual screenshot library. The global stats test also races the request against a 20 second timer of its own. It needs that timer because index.js keeps a 30 Hz callback pump alive on an interval, so a promise that never settled would keep the process running forever instead of letting the test fail. It is the same problem as the stale leaderboard handle at the start of this post, a request that never answers, showing up one layer higher in the test harness.