Operators could see attach-ready and capacity-pressure events but not how
full the configured disk (or memory) tier was without reading filesystems.
Expose size/capacity gauges and disk_cache_full_ratio next to disk_tier_ready.
Capacity is a config read, so it stays available while attach is pending.
/metrics was Prometheus-ish name/value text with a custom banner and
text/plain Content-Type, so strict scrapers could fail.
Emit Prometheus text 0.0.4 (# HELP, # TYPE, counter|gauge) with stable
metric names, and set Content-Type to text/plain; version=0.0.4; charset=utf-8.
When the disk (or memory) tier is at cap or the volume returns ENOSPC,
ops currently look like random misses with no clear "we are dropping
data." Count those events as capacity_pressure_events on /metrics and
log tier plus reason so operators can tell capacity pressure from a
cold cache, without changing the existing evictions counter.
Link: #36
Large disk caches can look memory-only/broken during async DiskFS attach.
Expose disk_tier_ready in /metrics, pending/ready logs for disk-only and
mixed modes, and X-SteamCache-Disk-Tier on /lancache-heartbeat. GetMetrics
skips blocking disk.Size() while attach is pending so /metrics stays usable.
Closes#33
Steam clients lean on Range requests. Hit/miss looked fine while downloads
still felt cold when a Range miss returned the full upstream body as 200.
Add range_cache / range_upstream metrics. HIT path already sliced 206 from
the cached full object; count range_cache there. On MISS, keep stripping
Range for the upstream fetch (full object still cached) but serve the
client's requested slice as 206 and count range_upstream.
Tests in steamcache/range_test.go cover Range HIT (local, no upstream),
Range MISS (206 + full object cached), and /metrics emission.
- Updated the Makefile to include a new `clean-disk` target for removing disk cache, and modified the `run-validation` target to clean the disk cache before starting.
- Enhanced the `check-review-labels` target to include additional file types in the search for temporary review labels, improving code hygiene checks.
- Refined the README.md to clarify the hardening section and improve the description of the `prefill` command.
- Removed the obsolete `test_cache/.gitkeep` file to clean up the repository.
- Added metrics for bytes saved from cache to improve performance insights.
- Updated cache eviction strategies in MemoryFS and DiskFS to include metrics tracking for hits and evictions.
- Enhanced README.md with updated garbage collection algorithm descriptions and recommendations for cache usage.
- Introduced new madviseSequential functionality for improved memory access hints on Unix systems.
- Adjusted validation configuration in examples to better reflect realistic usage scenarios.
- Introduced coalescing logic in `coalescing.go` to handle concurrent identical upstream fetches, including a state machine and response buffering for improved performance.
- Implemented a new cache file format in `format.go`, supporting serialization and deserialization of HTTP responses, along with range request handling.
- Developed an HTTP handler in `handler.go` to manage requests, including special endpoint handling and metrics reporting.
- Added rate limiting functionality in `ratelimit.go` to control per-client and global request rates, enhancing security and performance.
- Created service management capabilities in `service.go` to define and manage cacheable services, including user-agent detection.
- Updated tests in `steamcache_test.go` to cover new functionalities, ensuring robustness and reliability across the codebase.
- C4: Propagate request context through semaphores and upstream calls
- C5: Make client rate limiter cleanup goroutine idempotent via sync.Once
- P1: Streaming response path using io.Copy instead of full materialization
- P2: Gate promotion goroutines to files >= 64KiB
- P3: Reduce global lock contention during eviction
- P4: Demote hot-path logging and compact metrics output
- R1: Add upstream URL scheme/host validation
- R4: Add cache file format version + tolerant deserialization
- Makefile: Make build-snapshot-single practical by using -short tests
- Config: Add GetDefaultConfig + Validate for better testability and R1 coverage
All changes follow the "smallest safe diff" principle from the review.
Safe test suite now builds and runs cleanly via make build-snapshot-single.
Ref: docs/reviews/steamcache2-production-hardening-review-2026-05-26.md
- Introduced a new error handling system with custom error types for better context and clarity in error reporting.
- Implemented URL validation to prevent invalid requests and enhance security.
- Updated cache key generation functions to return errors, improving robustness in handling invalid inputs.
- Added comprehensive metrics tracking for requests, cache hits, misses, and performance metrics, allowing for better monitoring and analysis of the caching system.
- Enhanced logging to include detailed metrics and error information for improved debugging and operational insights.
pike
ash
linus
moss
s1d3sw1ped