cache: Short TTL negative cache for 404/410 depot objects #50

2026-09-08T11:42:27-05:00

pike commented

2026-09-08 11:42:27 -05:00

Why

A 404/410 depot chunk was re-fetched on every miss, hammering Steam for definitive gone objects while looking like normal cache misses.

What changed

Store short-TTL negative markers for upstream 404/410 in the same VFS cache under the same depot-path key (SC2C optional 4th expires-unix field; positives stay 3-field).
Within TTL, serve the cached 404/410 without re-hitting upstream; expired negatives are deleted and treated as miss.
Do not retry 404/410 with backoff; do not cache 5xx as negative.
New /metrics counter negative_cache_hits (also still counts as cache_hits).
Config: cache.negative_ttl (Go duration, default 5m; 0/0s disables storing).
Brief README docs for TTL/behavior near existing metrics/cache notes.

How tested

go test -count=1 ./steamcache/ ./config/ on jumpbox (pass)
TestNegativeCache404: first 404 hits upstream once; second within TTL does not; negative_cache_hits present
Also 410, TTL expiry refetch, SC2C 4th-field roundtrip, metrics emission

Risks / rollback

Negative entries occupy the same cache capacity as positives (short TTL + small bodies mitigate).
Clients now receive real 404/410 for gone objects instead of a generic 500 on the first miss (intentional).
Rollback: revert this PR / unset or set cache.negative_ttl: 0s.

Fixes #40

Waterfall

Produced by xAI grok (waterfall step 1: grok -p --effort medium).

## Why A 404/410 depot chunk was re-fetched on every miss, hammering Steam for definitive gone objects while looking like normal cache misses. ## What changed - Store short-TTL negative markers for upstream **404/410** in the **same** VFS cache under the same depot-path key (SC2C optional 4th `expires-unix` field; positives stay 3-field). - Within TTL, serve the cached 404/410 without re-hitting upstream; expired negatives are deleted and treated as miss. - Do **not** retry 404/410 with backoff; do **not** cache 5xx as negative. - New `/metrics` counter `negative_cache_hits` (also still counts as `cache_hits`). - Config: `cache.negative_ttl` (Go duration, default `5m`; `0`/`0s` disables storing). - Brief README docs for TTL/behavior near existing metrics/cache notes. ## How tested - `go test -count=1 ./steamcache/ ./config/` on jumpbox (pass) - `TestNegativeCache404`: first 404 hits upstream once; second within TTL does not; `negative_cache_hits` present - Also 410, TTL expiry refetch, SC2C 4th-field roundtrip, metrics emission ## Risks / rollback - Negative entries occupy the same cache capacity as positives (short TTL + small bodies mitigate). - Clients now receive real 404/410 for gone objects instead of a generic 500 on the first miss (intentional). - Rollback: revert this PR / unset or set `cache.negative_ttl: 0s`. Fixes #40 ## Waterfall Produced by **xAI grok** (waterfall step 1: `grok -p --effort medium`).

pike added 1 commit 2026-09-08 11:42:27 -05:00

cache: Short TTL negative cache for 404/410 depot objects

CI / vulncheck (pull_request) Successful in 14s

Details

CI / check-and-test (pull_request) Successful in 41s

Details

036ea1ea7f

Stop re-fetching gone depot objects on every miss: store 404/410 in the
existing VFS cache under the same key with a short TTL (default 5m).

pike requested review from eva 2026-09-08 11:42:38 -05:00

eva merged commit a0e929c525 into main

2026-09-08 11:49:07 -05:00

eva deleted branch cache/negative-ttl-404

2026-09-08 11:49:07 -05:00

eva referenced this pull request

2026-09-08 13:14:12 -05:00

ops: Signal disk-full and eviction capacity pressure #46

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: s1d3sw1ped/steamcache2#50