171 Commits

Author SHA1 Message Date
Blake 97af0f829b ops: Assert Host allowlist reject in validate-check
CI / vulncheck (pull_request) Successful in 16s
CI / check-and-test (pull_request) Successful in 53s
CI / vulncheck (push) Successful in 22s
CI / check-and-test (push) Successful in 1m0s
Release Tag / release (push) Successful in 15s
Empty-upstream Host allowlist is load-bearing: if that gate regresses,
the cache becomes an open LAN reverse proxy again. Unit tests already
cover hostAllowedForDirectFetch, but make validate-check did not probe
the live reject path.

Extend validate-check to GET a depot-like path with Host: evil.example
and a Steam User-Agent, requiring HTTP 400 Invalid URL. Document the
expected reject in README and the validate-config comment.

Fixes #37
1.0.31
2026-09-14 13:39:32 +00:00
pike 25622cbf25 ops: Fix goimports on fair-share bandwidth PR
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 51s
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 50s
Release Tag / release (push) Successful in 20s
1.0.30
2026-09-09 20:27:32 +00:00
pike b7710de0ca cache: Per-client fair-share bandwidth on table uplink
CI / vulncheck (pull_request) Successful in 20s
CI / check-and-test (pull_request) Failing after 21s
2026-09-09 20:22:23 +00:00
pike 50ca0a071c ops: Disk tier occupancy on /metrics
CI / vulncheck (pull_request) Successful in 19s
CI / check-and-test (pull_request) Successful in 47s
CI / vulncheck (push) Successful in 16s
CI / check-and-test (push) Successful in 46s
Release Tag / release (push) Successful in 14s
Operators could see attach-ready and capacity-pressure events but not how
full the configured disk (or memory) tier was without reading filesystems.
Expose size/capacity gauges and disk_cache_full_ratio next to disk_tier_ready.
Capacity is a config read, so it stays available while attach is pending.
1.0.29
2026-09-09 16:06:12 +00:00
linus a3ea4806a7 Merge pull request 'cache: Coalesce in-flight identical upstream fetches' (#52) from cache/coalesce-inflight-fetches into develop
CI / vulncheck (push) Successful in 16s
CI / check-and-test (push) Successful in 48s
Release Tag / release (push) Successful in 15s
cache: Coalesce in-flight identical upstream fetches

Fixes #35
1.0.28
2026-09-08 15:19:26 -05:00
ash ea195993de cache: Coalesce in-flight identical upstream fetches
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 48s
In-flight coalescing already dedups identical misses, but acceptance
still lacked a gated test (without a hold, waiters can become sequential
HITs after the first fill) and the Quick check metrics table omitted
cache_coalesced.

Add steamcache/coalesce_test.go: N concurrent identical GETs share one
upstream fill (leader MISS, waiters HIT-COALESCED) and a 5xx sibling.
Document cache_coalesced next to the other hit/miss fields.

Fixes #35
2026-09-08 20:00:53 +00:00
eva 7c34ff4538 Merge pull request 'metrics: Prometheus text exposition for /metrics' (#51) from metrics/prometheus-exposition into main
CI / vulncheck (push) Successful in 15s
CI / check-and-test (push) Successful in 41s
Release Tag / release (push) Successful in 16s
metrics: Prometheus text exposition for /metrics (#51)

Closes #49
1.0.27
2026-09-08 14:57:22 -05:00
pike dd72668c2d metrics: Prometheus text exposition for /metrics
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 42s
/metrics was Prometheus-ish name/value text with a custom banner and
text/plain Content-Type, so strict scrapers could fail.

Emit Prometheus text 0.0.4 (# HELP, # TYPE, counter|gauge) with stable
metric names, and set Content-Type to text/plain; version=0.0.4; charset=utf-8.
2026-09-08 19:53:58 +00:00
linus 0db9943436 ops: Signal disk-full and eviction capacity pressure (#46)
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 43s
Release Tag / release (push) Successful in 14s
capacity_pressure_events + ENOSPC/eviction logs. Closes #36.
1.0.26
2026-09-08 13:20:22 -05:00
ash e3b2b8de1e test: Hold disk init in DiskOnlyDelayedAttach too
CI / vulncheck (pull_request) Successful in 15s
CI / check-and-test (pull_request) Successful in 42s
Empty-dir attach can finish before the pending DiskTierReady
assertion under CI load, same race MixedPendingReady already
fixed with RegisterInitHold. Drop t.Parallel and hold the
barrier through the pending checks so disk-only stays green.

Link: #36
2026-09-08 18:16:18 +00:00
ash fe04fb9fbb vfs/disk: Harden pending-window test against empty-dir race
Empty-dir disk init closes initDone almost immediately, so
TestDiskTierSignalMixedPendingReady can observe DiskTierReady=0 then
a heartbeat that already saw ready. Register a per-root init hold so
the test can keep the pending window open without slowing production.

Link: #36
2026-09-08 18:16:18 +00:00
ash de43a71929 ops: Signal disk-full and eviction capacity pressure
When the disk (or memory) tier is at cap or the volume returns ENOSPC,
ops currently look like random misses with no clear "we are dropping
data." Count those events as capacity_pressure_events on /metrics and
log tier plus reason so operators can tell capacity pressure from a
cold cache, without changing the existing evictions counter.

Link: #36
2026-09-08 18:15:51 +00:00
eva a0e929c525 Merge pull request 'cache: Short TTL negative cache for 404/410 depot objects' (#50) from cache/negative-ttl-404 into main
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Failing after 40s
2026-09-08 11:49:07 -05:00
pike 036ea1ea7f cache: Short TTL negative cache for 404/410 depot objects
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
Stop re-fetching gone depot objects on every miss: store 404/410 in the
existing VFS cache under the same key with a short TTL (default 5m).
2026-09-08 16:40:03 +00:00
linus 8eb31143e5 cache: Key by depot path across CDN host aliases (#48)
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 41s
Release Tag / release (push) Successful in 15s
Cache key from Path only; Host aliases share one entry. Closes #39.
1.0.25
2026-09-07 15:18:26 -05:00
moss 337741e06e cache: Key by depot path across CDN host aliases
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
2026-09-07 20:14:30 +00:00
linus 2c8ec276e7 ops: Widen validate-check for upstream/write/rate failures (#47)
CI / vulncheck (push) Successful in 15s
CI / check-and-test (push) Successful in 41s
Highlight upstream_errors/cache_write_failures/rate_limited. Closes #34.
2026-09-07 14:53:37 -05:00
pike 144a68dcd7 ops: Widen validate-check for upstream/write/rate failures
CI / vulncheck (pull_request) Successful in 15s
CI / check-and-test (pull_request) Successful in 41s
Surface upstream_errors, cache_write_failures, and rate_limited in
make validate-check highlights and Quick check docs so a hot failure
path cannot look fine when hits exist.

Closes #34
2026-09-07 19:51:50 +00:00
linus 8cebc1f96c ops: Signal disk-tier attach pending vs ready (#45)
CI / vulncheck (push) Successful in 15s
CI / check-and-test (push) Successful in 41s
Release Tag / release (push) Successful in 15s
disk_tier_ready + X-SteamCache-Disk-Tier. Closes #33.
1.0.24
2026-09-07 12:02:10 -05:00
pike 12ea3ee4f6 ops: Signal disk-tier attach pending vs ready
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 40s
Large disk caches can look memory-only/broken during async DiskFS attach.
Expose disk_tier_ready in /metrics, pending/ready logs for disk-only and
mixed modes, and X-SteamCache-Disk-Tier on /lancache-heartbeat. GetMetrics
skips blocking disk.Size() while attach is pending so /metrics stays usable.

Closes #33
2026-09-07 16:58:13 +00:00
linus a500f51f17 cache: Serve Range GET from disk when present (#44)
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 41s
Range HIT/MISS as 206; range_cache/range_upstream metrics. Closes #38.
2026-09-07 11:37:06 -05:00
moss ac2d36f1ad cache: Serve Range GET from disk when present
CI / vulncheck (pull_request) Successful in 15s
CI / check-and-test (pull_request) Successful in 41s
Steam clients lean on Range requests. Hit/miss looked fine while downloads
still felt cold when a Range miss returned the full upstream body as 200.

Add range_cache / range_upstream metrics. HIT path already sliced 206 from
the cached full object; count range_cache there. On MISS, keep stripping
Range for the upstream fetch (full object still cached) but serve the
client's requested slice as 206 and count range_upstream.

Tests in steamcache/range_test.go cover Range HIT (local, no upstream),
Range MISS (206 + full object cached), and /metrics emission.
2026-09-07 16:32:36 +00:00
linus c43bfba568 Merge pull request 'docs: Real DNS + empty upstream Quick Start' (#43) from docs/quick-start-real-dns into main
docs: Real DNS + empty upstream Quick Start

Fixes #32
2026-09-07 10:00:58 -05:00
eva 3411defd51 docs: Real DNS + empty upstream Quick Start
CI / vulncheck (pull_request) Successful in 15s
CI / check-and-test (pull_request) Successful in 40s
2026-09-07 14:58:06 +00:00
linus 71d5106777 docs: Add README operator cache check (#30)
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 41s
Release Tag / release (push) Successful in 16s
README Quick check + make validate-check using existing /metrics and heartbeat. Closes #29.
1.0.23
2026-09-03 00:19:53 -05:00
pike 8b1b229539 docs: Deduplicate validate-check helper
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
Keep a single Makefile validate-check target that prints the full
/metrics dump, highlights hit/miss fields, and curls /lancache-heartbeat.
Drop the duplicate README troubleshooting item and align the validation
chapter plus validate-config.yaml comments with that behavior.

Link: #29
2026-09-03 05:18:19 +00:00
pike ff1ab31327 docs: Add README operator cache check
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
The README buried hit/miss verification under the SteamPrefill validation
chapter, and advertised make validate-check without a Makefile target.
Operators need a short start → traffic → metrics path against the existing
/metrics and /lancache-heartbeat endpoints.

Link: #29
2026-09-03 05:16:14 +00:00
s1d3sw1ped_bot 85e14bc8af README: Replace --threads with real cobra concurrency flags
Release Tag / release (push) Successful in 17s
Docs sync: align README with current install/CLI behavior.

Co-authored-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
Co-committed-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
1.0.22
2026-09-02 10:43:52 -05:00
s1d3sw1ped_bot 3f5175b482 .gitea/workflows: Prefer job GITEA_TOKEN
CI / vulncheck (push) Successful in 15s
CI / check-and-test (push) Successful in 41s
Release Tag still used secrets.RELEASE_TOKEN. Prefer the job built-in token with contents: write, same path as vulture/teleport.

Link: #26
Co-authored-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
Co-committed-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
2026-09-02 10:08:00 -05:00
s1d3sw1ped_bot 04d1c6c368 Merge pull request 'Promote develop: vfs/cache promoteToFast race fix' (#24) from develop into main
CI / vulncheck (push) Successful in 15s
CI / check-and-test (push) Successful in 41s
Release Tag / release (push) Successful in 30s
Promote vfs/cache Stat snapshot + promoteToFast size fix.
1.0.21
2026-09-01 15:36:49 -05:00
s1d3sw1ped_bot 523a9a4782 Merge pull request 'vfs/cache: Fix promoteToFast race with WriteCloser Close' (#23) from vfs/cache-promote-race into develop
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
Stat returns FileInfo snapshots; promoteToFast sizes from len(content).

#21
2026-09-01 15:35:20 -05:00
s1d3sw1ped_bot 8e09c89e24 vfs/cache: Fix promoteToFast race with WriteCloser Close
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 40s
promoteToFast read FileInfo.Size from Stat after the VFS lock was
released, while WriteCloser.Close updated Size on the same live
object. Concurrent Create/Open on overlapping keys tripped the race
detector.

Stat now returns a FileInfo snapshot, and promotion uses the
ReadAll length for the fast-tier Create size. Promotion stays
best-effort.

#21
2026-09-01 20:30:40 +00:00
s1d3sw1ped_bot a2ac13d317 vfs/disk: Fix EvictDiskVisibilityAndRecreateSafety flake
CI / vulncheck (push) Successful in 13s
CI / check-and-test (push) Failing after 40s
2026-09-01 15:23:10 -05:00
s1d3sw1ped_bot 5d006ac44f vfs/disk: Fix EvictDiskVisibilityAndRecreateSafety flake
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
2026-09-01 15:21:05 -05:00
s1d3sw1ped_bot acd006d4a2 ci: Skip test workflow on markdown-only pushes
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 40s
Docs-only pushes to main (**.md / CONTRIBUTING.md) do not need the
full check-and-test job. Pull requests are unchanged.
2026-09-01 20:19:19 +00:00
s1d3sw1ped_bot 30a695458e vfs/disk: Fix EvictDiskVisibilityAndRecreateSafety flake
New() launches background calculateSizeAndPopulateIndex which scans
disk and calls insertBatch. Create does not wait on initDone, so a
file can be written and indexed, discovered by the scan, then removed
from d.info and disk by EvictLRU/EvictBySize. insertBatch then
re-inserted the stale discoveredFile without checking the path still
existed, so Stat succeeded from the index while os.Stat failed.

Re-stat under the lock and skip gone files so evicted keys are not
resurrected.

Fixes #18.
2026-09-01 20:19:14 +00:00
s1d3sw1ped_bot e8bcf0ddbd docs: Add CONTRIBUTING.md
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 41s
Add short contribution guide on main from develop tip content.
2026-09-01 14:11:02 -05:00
s1d3sw1ped_bot f497e71ef0 docs: Add CONTRIBUTING.md
CI / vulncheck (pull_request) Successful in 16s
CI / check-and-test (pull_request) Successful in 43s
Copy the develop tip contribution guide onto main so the default
branch carries the same short CONTRIBUTING.md.
2026-09-01 19:10:15 +00:00
s1d3sw1ped_bot 0198e8990b docs: Add CONTRIBUTING.md
CI / vulncheck (pull_request) Successful in 31s
CI / check-and-test (pull_request) Failing after 57s
Contributors need a short guide for develop-targeted PRs, commit
subject form, and Gitea issue-closing rules.
2026-09-01 13:37:56 -05:00
s1d3sw1ped_bot 2a2cd8d393 Bump Go to 1.27.0 (#14)
CI / vulncheck (push) Successful in 8s
CI / check-and-test (push) Successful in 35s
2026-08-31 20:04:49 -05:00
s1d3sw1ped_bot 81b3a7df53 Restrict Host-based fetches to Steam CDN names (#13)
CI / vulncheck (push) Successful in 7s
CI / check-and-test (push) Successful in 27s
Allowlist steamcontent/steampowered/steamstatic Hosts; do not follow upstream redirects.
2026-08-31 18:58:53 -05:00
s1d3sw1ped_bot 8e8e877533 Restrict Host-based origin fetches to Steam CDN names.
CI / vulncheck (pull_request) Successful in 7s
CI / check-and-test (pull_request) Successful in 28s
When upstream is empty the cache used the client Host as the fetch URL, so any LAN client with a spoofed Steam User-Agent could proxy to literal IPs or arbitrary names. Reject those hosts, stop following upstream redirects, and keep path-only cache keys so real Steam CDNs still share entries.
2026-08-31 23:56:03 +00:00
s1d3sw1ped d63d7b4d3c Merge pull request 'Run CI on push to main' (#11) from ci/test-on-push into main
CI / vulncheck (push) Successful in 8s
CI / check-and-test (push) Successful in 28s
Release Tag / release (push) Successful in 14s
Reviewed-on: #11
1.0.20
2026-08-31 15:47:07 -05:00
s1d3sw1ped_bot 35d698a232 Keep v1 lint rules under golangci-lint v2 (no ST/QF, no G704/G705).
CI / vulncheck (pull_request) Successful in 7s
CI / check-and-test (pull_request) Successful in 38s
2026-08-31 20:30:36 +00:00
s1d3sw1ped_bot 36b613b6cd Drop empty gosec settings so golangci-lint v2 config verifies.
CI / vulncheck (pull_request) Successful in 7s
CI / check-and-test (pull_request) Failing after 13s
2026-08-31 20:28:54 +00:00
s1d3sw1ped_bot 79f02d9868 Fix CI on Go 1.26.7: golangci-lint-action v8 and vulncheck on 1.27.
CI / vulncheck (pull_request) Successful in 19s
CI / check-and-test (pull_request) Failing after 27s
v4 installs golangci-lint v1.64.8 (built with Go 1.24) which cannot lint a
1.26 module; v8 pulls v2.12. govulncheck@latest still fails to install on
1.26.7, so run that job on Go 1.27. Keep tests on push to main.
2026-08-31 20:27:35 +00:00
s1d3sw1ped_bot 19497eba0c Pin CI to Go 1.26.7 and stable action tags.
CI / vulncheck (pull_request) Failing after 13s
CI / check-and-test (pull_request) Failing after 23s
setup-go@main plus go-version-file 1.26.0 was flaky (version: not found) and still scanned an unpatched 1.26.0 stdlib.
2026-08-31 15:18:59 -05:00
s1d3sw1ped_bot c7a2312994 Raise module Go version from EOL 1.23.0 to 1.26.0.
CI / check-and-test (pull_request) Failing after 11s
CI / vulncheck (pull_request) Failing after 21s
govulncheck fails on stdlib crypto/tls and crypto/x509 findings (including GO-2025-4008 / CVE-2025-58189) that were never patched on 1.23. 1.26 is still a supported release.
2026-08-31 15:15:48 -05:00
s1d3sw1ped_bot fbb084d824 Use latest Go 1.23 patch in CI and run tests even if govulncheck fails.
CI / vulncheck (pull_request) Failing after 18s
CI / check-and-test (pull_request) Successful in 37s
setup-go was installing go.mod's 1.23.0 exactly, so govulncheck reported stdlib x509 findings and skipped tests. check-latest gets the patched 1.23, and vulncheck is its own job.
2026-08-31 15:11:43 -05:00
s1d3sw1ped_bot e7d4a19c3f Let govulncheck@latest fetch a newer Go toolchain.
CI / check-and-test (pull_request) Failing after 22s
CI was dying on: golang.org/x/vuln@v1.7.0 requires go >= 1.25.0 (running go 1.23.0; GOTOOLCHAIN=local).
2026-08-31 15:11:18 -05:00