Operators could see attach-ready and capacity-pressure events but not how
full the configured disk (or memory) tier was without reading filesystems.
Expose size/capacity gauges and disk_cache_full_ratio next to disk_tier_ready.
Capacity is a config read, so it stays available while attach is pending.
/metrics was Prometheus-ish name/value text with a custom banner and
text/plain Content-Type, so strict scrapers could fail.
Emit Prometheus text 0.0.4 (# HELP, # TYPE, counter|gauge) with stable
metric names, and set Content-Type to text/plain; version=0.0.4; charset=utf-8.
Empty-dir attach can finish before the pending DiskTierReady
assertion under CI load, same race MixedPendingReady already
fixed with RegisterInitHold. Drop t.Parallel and hold the
barrier through the pending checks so disk-only stays green.
Link: #36
Empty-dir disk init closes initDone almost immediately, so
TestDiskTierSignalMixedPendingReady can observe DiskTierReady=0 then
a heartbeat that already saw ready. Register a per-root init hold so
the test can keep the pending window open without slowing production.
Link: #36
When the disk (or memory) tier is at cap or the volume returns ENOSPC,
ops currently look like random misses with no clear "we are dropping
data." Count those events as capacity_pressure_events on /metrics and
log tier plus reason so operators can tell capacity pressure from a
cold cache, without changing the existing evictions counter.
Link: #36
Large disk caches can look memory-only/broken during async DiskFS attach.
Expose disk_tier_ready in /metrics, pending/ready logs for disk-only and
mixed modes, and X-SteamCache-Disk-Tier on /lancache-heartbeat. GetMetrics
skips blocking disk.Size() while attach is pending so /metrics stays usable.
Closes#33
When upstream is empty the cache used the client Host as the fetch URL, so any LAN client with a spoofed Steam User-Agent could proxy to literal IPs or arbitrary names. Reject those hosts, stop following upstream redirects, and keep path-only cache keys so real Steam CDNs still share entries.
- Introduced coalescing logic in `coalescing.go` to handle concurrent identical upstream fetches, including a state machine and response buffering for improved performance.
- Implemented a new cache file format in `format.go`, supporting serialization and deserialization of HTTP responses, along with range request handling.
- Developed an HTTP handler in `handler.go` to manage requests, including special endpoint handling and metrics reporting.
- Added rate limiting functionality in `ratelimit.go` to control per-client and global request rates, enhancing security and performance.
- Created service management capabilities in `service.go` to define and manage cacheable services, including user-agent detection.
- Updated tests in `steamcache_test.go` to cover new functionalities, ensuring robustness and reliability across the codebase.
- Updated `disk.New` to support asynchronous initialization for large caches, improving responsiveness during startup.
- Introduced an eviction function parameter to `disk.New`, ensuring proper handling of over-capacity scenarios.
- Enhanced error handling in various components, including memory and disk tests, to ensure robustness and clarity.
- Refactored tests to validate new behaviors, including checks for delayed attachment and proper error propagation.
- Removed obsolete error handling code and tests related to the now-deleted errors package, streamlining the codebase.
- Implemented a more robust shutdown mechanism using sync.Once to prevent multiple shutdown calls and ensure all background managers are stopped properly.
- Refactored coalesced request handling to utilize atomic operations for waiting counts, improving thread safety and performance.
- Introduced a done channel for coalesced requests to signal completion, enhancing the handling of concurrent requests and reducing potential deadlocks.
- Updated logging to provide better insights into cache request processing and error handling.
- C4: Propagate request context through semaphores and upstream calls
- C5: Make client rate limiter cleanup goroutine idempotent via sync.Once
- P1: Streaming response path using io.Copy instead of full materialization
- P2: Gate promotion goroutines to files >= 64KiB
- P3: Reduce global lock contention during eviction
- P4: Demote hot-path logging and compact metrics output
- R1: Add upstream URL scheme/host validation
- R4: Add cache file format version + tolerant deserialization
- Makefile: Make build-snapshot-single practical by using -short tests
- Config: Add GetDefaultConfig + Validate for better testability and R1 coverage
All changes follow the "smallest safe diff" principle from the review.
Safe test suite now builds and runs cleanly via make build-snapshot-single.
Ref: docs/reviews/steamcache2-production-hardening-review-2026-05-26.md
- Introduced a new error handling system with custom error types for better context and clarity in error reporting.
- Implemented URL validation to prevent invalid requests and enhance security.
- Updated cache key generation functions to return errors, improving robustness in handling invalid inputs.
- Added comprehensive metrics tracking for requests, cache hits, misses, and performance metrics, allowing for better monitoring and analysis of the caching system.
- Enhanced logging to include detailed metrics and error information for improved debugging and operational insights.
- Updated the caching logic to utilize a predictive cache warmer, enhancing content prefetching based on access patterns.
- Replaced the legacy warming system with a more efficient predictive approach, allowing for better performance and resource management.
- Refactored memory management to integrate dynamic cache size adjustments based on system memory usage, improving overall efficiency.
- Simplified the VFS interface and improved concurrency handling with sharded locks for better performance in multi-threaded environments.
- Enhanced tests to validate the new caching and memory management behaviors, ensuring reliability and performance improvements.
- Updated caching logic to support size-based promotion filtering, ensuring that not all files may be promoted based on size constraints.
- Implemented adaptive caching strategies with a new AdaptiveCacheManager to analyze access patterns and adjust caching strategies dynamically.
- Introduced predictive caching features with a PredictiveCacheManager to prefetch content based on access patterns.
- Added a CacheWarmer to preload popular content into the cache, improving access times for frequently requested files.
- Refactored memory management with a DynamicCacheManager to adjust cache sizes based on system memory usage.
- Enhanced VFS interface and file metadata handling to support new features and improve performance.
- Updated tests to validate new caching behaviors and ensure reliability of the caching system.
- Introduced integration tests for SteamCache to validate caching behavior with real Steam URLs.
- Implemented a ServiceManager to manage service configurations, allowing for dynamic detection of services based on User-Agent.
- Updated cache key generation to include service prefixes, enhancing cache organization and retrieval.
- Enhanced the caching logic to support multiple services, starting with Steam and Epic Games.
- Improved .gitignore to exclude test cache files while retaining necessary structure.
- Introduced maxConcurrentRequests and maxRequestsPerClient fields in the Config struct to manage request limits.
- Updated the SteamCache implementation to utilize these new configuration options for controlling concurrent requests.
- Enhanced the ServeHTTP method to enforce global and per-client rate limiting using semaphores.
- Modified the root command to accept new flags for configuring concurrency limits via command-line arguments.
- Updated tests to reflect changes in the SteamCache initialization and request handling logic.
- Replaced SHA1 hash calculations with SHA256 for improved security and consistency in cache key generation.
- Introduced a new TestURLHashing function to validate the new cache key generation logic.
- Removed outdated hash calculation tests and streamlined the caching process to focus on URL-based hashing.
- Implemented lightweight validation methods in ServeHTTP to enhance performance and reliability of cached responses.
- Added batched time updates in VFS implementations for better performance during access time tracking.
- Introduced a YAML-based configuration system, allowing for automatic generation of a default `config.yaml` file.
- Updated the application to load configuration settings from the YAML file, improving flexibility and ease of use.
- Added a Makefile to streamline development tasks, including running the application, testing, and managing dependencies.
- Enhanced `.gitignore` to include build artifacts and configuration files.
- Removed unused Prometheus metrics and related code to simplify the codebase.
- Updated dependencies in `go.mod` and `go.sum` for improved functionality and performance.
- Updated .gitignore to include all .exe files and ensure .smashignore is tracked.
- Expanded README.md with advanced configuration options for garbage collection algorithms, detailing available algorithms and use cases.
- Modified launch.json to include memory and disk garbage collection flags for better configuration.
- Refactored root.go to introduce memoryGC and diskGC flags for garbage collection algorithms.
- Implemented hash extraction and verification in steamcache.go to ensure data integrity during caching.
- Added new tests in steamcache_test.go for hash extraction and verification, ensuring correctness of caching behavior.
- Enhanced garbage collection strategies in gc.go, introducing LFU, FIFO, Largest, Smallest, and Hybrid algorithms with corresponding metrics.
- Updated caching logic to conditionally cache responses based on hash verification results.
- Updated disk_test.go to replace Set and Get with Create and Open methods for better clarity and functionality.
- Modified fileinfo.go to include package comment.
- Refactored gc.go to streamline garbage collection handling and removed unused statistics.
- Updated gc_test.go to comment out large random tests for future implementation.
- Enhanced memory.go to implement LRU caching and metrics for memory usage.
- Updated memory_test.go to replace Set and Get with Create and Open methods.
- Removed sync.go as it was redundant and not utilized.
- Updated vfs.go to reflect changes in the VFS interface, replacing Set and Get with Create and Open.
- Added package comments to vfserror.go for consistency.
- Added Prometheus client library for metrics collection.
- Refactored garbage collection strategy from random deletion to LRU (Least Recently Used) deletion.
- Introduced per-key locking in cache to prevent race conditions.
- Enhanced logging with structured log messages for cache hits and misses.
- Implemented a retry mechanism for upstream requests with exponential backoff.
- Updated Go modules and indirect dependencies for better compatibility and performance.
- Removed unused sync filesystem implementation.
- Added version initialization to ensure a default version string.
pike
ash
moss
s1d3sw1ped_bot
s1d3sw1ped