Update metrics tracking and enhance cache eviction strategies
Release Tag / release (push) Successful in 34s

- Added metrics for bytes saved from cache to improve performance insights.
- Updated cache eviction strategies in MemoryFS and DiskFS to include metrics tracking for hits and evictions.
- Enhanced README.md with updated garbage collection algorithm descriptions and recommendations for cache usage.
- Introduced new madviseSequential functionality for improved memory access hints on Unix systems.
- Adjusted validation configuration in examples to better reflect realistic usage scenarios.
This commit is contained in:
2026-05-28 10:31:23 -05:00
parent 3fd72705fc
commit b7e3a0da86
11 changed files with 184 additions and 41 deletions
+12 -11
View File
@@ -261,11 +261,11 @@ SteamCache2 supports different garbage collection algorithms for memory and disk
**Available GC Algorithms:**
- **`lru`** (default): Least Recently Used - evicts oldest accessed files
- **`lfu`**: Least Frequently Used (P1 real impl) - evicts by lowest AccessCount (tiebreak older ATime); uses existing FileInfo counters
- **`fifo`**: First In, First Out - evicts oldest created files (predictable)
- **`largest`**: Size-based - evicts largest files first (maximizes file count)
- **`smallest`**: Size-based - evicts smallest files first (maximizes cache hit rate)
- **`hybrid`**: Recency + frequency hybrid (P1 meaningful) - evicts by lowest time-decayed score (GetTimeDecayedScore combining ATime + AccessCount)
- **`lfu`**: Least Frequently Used - evicts by lowest AccessCount (tiebreak older ATime); uses existing FileInfo counters
- **`fifo`**: First In, First Out - evicts oldest created files (predictable and terrible all in one) don't ever use it
- **`largest`**: Size-based - evicts largest files first (maximizes small file count) if used on memory greatly improves access time
- **`smallest`**: Size-based - evicts smallest files first (maximizes large file count) probably best used for disk since there kinda slow with small files
- **`hybrid`**: Recency + frequency hybrid - evicts by lowest time-decayed score (GetTimeDecayedScore combining ATime + AccessCount)
**Recommended Algorithms by Cache Type:**
@@ -273,18 +273,19 @@ SteamCache2 supports different garbage collection algorithms for memory and disk
- **`lru`** - Best overall performance, good balance of speed and hit rate
- **`lfu`** - Excellent for gaming cafes where popular games stay cached
- **`hybrid`** - Optimal for mixed workloads with varying file sizes
- **`largest`** - Crazy good for access times since disks are slow with lots of tiny files
**For Disk Cache (Slow, Large Size):**
- **`hybrid`** - Recommended for optimal performance, balances speed and storage efficiency
- **`largest`** - Good for maximizing number of cached files
- **`smallest`** - Good for maximizing linear reads which is the only place spinning disks have performance don't expect too much though steam kinda uses small files
- **`lru`** - Reliable default with good performance
**Use Cases:**
- **Gaming Cafes**: Use `lfu` for memory, `hybrid` for disk
- **LAN Events**: Use `lfu` for memory, `hybrid` for disk
- **Home Use**: Use `lru` for memory, `hybrid` for disk
- **Testing**: Use `fifo` for predictable behavior
- **Large File Storage**: Use `largest` for disk to maximize file count
- **Gaming Cafes**: Use `largest` for memory, `hybrid` for disk
- **LAN Events**: Use `largest` for memory, `hybrid` for disk
- **Home Use**: Use `largest` for memory, `hybrid` for disk
- **Testing**: Use `fifo` for nothing its pointless
- **Large File Storage**: Use `smallest` for disk get rid of the slow tiny files first
### DNS Configuration
+12 -9
View File
@@ -6,11 +6,14 @@
# "benchmark" commands.
#
# Why these values?
# - Both memory and disk tiers are enabled but deliberately small.
# A modest benchmark workload (a few hundred MB to low GB) will exercise:
# * Memory tier promotions
# * Disk tier attach + writes
# * GC / eviction pressure on at least one tier
# - Both tiers enabled. Memory is sized large enough to survive the disk attach
# window in mixed mode (see steamcache.go: the goroutine that blocks on d.Size()
# before SetSlow). With a realistic SteamPrefill benchmark (high rate of unique
# ~1MB chunks) the old tiny 128MB mem + 512MB disk caused almost all early
# content to live only in memory, get evicted by its GC, and never reach disk.
# Result: "never hitting", hit_rate 0, memory_size 0, despite files appearing
# on disk for late-arriving chunks. Larger mem + disk makes the validation
# actually exercise hits, promotions, disk tier, and GC as intended.
# - Conservative concurrency limits suitable for a developer laptop.
# - trusted_proxies set for 127.0.0.0/8 so that an external benchmark tool
# can simulate multiple distinct clients via X-Forwarded-For if desired.
@@ -31,7 +34,7 @@
listen_address: :80
max_concurrent_requests: 100
max_concurrent_requests: 1000
max_requests_per_client: 10
max_object_size: "0" # unlimited for validation (real Steam files can be large)
@@ -39,10 +42,10 @@ trusted_proxies: ["127.0.0.0/8"]
cache:
memory:
size: 128MB
gc_algorithm: lru # lru is somewhat recommeded in the project README and is better for validation since its not just using the same again
size: 1GB
gc_algorithm: largest
disk:
size: 512MB
size: 2GB
path: ./validate-disk # ephemeral; clean between runs if you want a fresh test
gc_algorithm: hybrid # recommended for disk in the project README
+1 -1
View File
@@ -8,6 +8,7 @@ require (
github.com/rs/zerolog v1.33.0
github.com/spf13/cobra v1.8.1
golang.org/x/sync v0.16.0
golang.org/x/sys v0.12.0
gopkg.in/yaml.v3 v3.0.1
)
@@ -16,5 +17,4 @@ require (
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect
github.com/spf13/pflag v1.0.5 // indirect
golang.org/x/sys v0.12.0 // indirect
)
+7 -5
View File
@@ -110,6 +110,7 @@ func (sc *SteamCache) handleCacheHit(w http.ResponseWriter, r *http.Request, cac
sc.metrics.IncrementCacheHits()
sc.metrics.AddResponseTime(time.Since(tstart))
sc.metrics.AddBytesServed(int64(len(cachedData)))
sc.metrics.AddBytesSaved(int64(len(cachedData)))
sc.metrics.IncrementServiceRequests(service.Name)
logger.Logger.Debug().
@@ -197,6 +198,7 @@ func (sc *SteamCache) waitForCoalesced(w http.ResponseWriter, r *http.Request, c
sc.metrics.IncrementCacheCoalesced()
sc.metrics.AddResponseTime(time.Since(tstart))
sc.metrics.AddBytesServed(int64(len(responseData)))
sc.metrics.AddBytesSaved(int64(len(responseData)))
sc.metrics.IncrementServiceRequests(service.Name)
logger.Logger.Info().
@@ -233,9 +235,6 @@ func (sc *SteamCache) ServeHTTP(w http.ResponseWriter, r *http.Request) {
}
defer sc.requestSemaphore.Release(1)
// Track total requests
sc.metrics.IncrementTotalRequests()
// Apply per-client rate limiting
clientLimiter := sc.getOrCreateClientLimiter(clientIP)
@@ -305,6 +304,11 @@ func (sc *SteamCache) ServeHTTP(w http.ResponseWriter, r *http.Request) {
Str("client_ip", clientIP).
Msg("Generated cache key")
// Only count real cacheable service traffic toward total_requests / hit_rate.
// Special endpoints (/, /metrics, /lancache-heartbeat) and unsupported services
// are intentionally excluded so that idle monitoring doesn't dilute the hit rate.
sc.metrics.IncrementTotalRequests()
if sc.handleCacheHit(w, r, cachePath, cacheKey, urlPath, service, clientIP, tstart) {
return
}
@@ -588,8 +592,6 @@ func (sc *SteamCache) ServeHTTP(w http.ResponseWriter, r *http.Request) {
sc.metrics.IncrementServiceError("cache_write")
_ = sc.vfs.Delete(cachePath) // best-effort removal of partial corrupt cache entry on write failure; non-fatal. Deferred cacheWriter.Close() (from earlier in block) runs after this on error unwind path (harmless per DiskFS design)
} else {
// Track successful cache write
sc.metrics.AddBytesCached(int64(len(cacheData)))
logger.Logger.Debug().
Str("key", cacheKey).
Str("url", urlPath).
+17 -11
View File
@@ -22,7 +22,9 @@ type Metrics struct {
// Performance metrics
TotalResponseTime int64 // in nanoseconds
TotalBytesServed int64
TotalBytesCached int64
TotalBytesSaved int64 // bytes served from cache instead of being re-downloaded from upstream
// Cache metrics
MemoryCacheSize int64
@@ -98,9 +100,10 @@ func (m *Metrics) AddBytesServed(bytes int64) {
atomic.AddInt64(&m.TotalBytesServed, bytes)
}
// AddBytesCached adds bytes cached to the total
func (m *Metrics) AddBytesCached(bytes int64) {
atomic.AddInt64(&m.TotalBytesCached, bytes)
// AddBytesSaved records bytes that were served from the cache instead of being
// fetched again from the upstream (the main value metric for a cache).
func (m *Metrics) AddBytesSaved(bytes int64) {
atomic.AddInt64(&m.TotalBytesSaved, bytes)
}
// SetMemoryCacheSize sets the current memory cache size
@@ -191,8 +194,8 @@ func (m *Metrics) GetStats() *Stats {
RateLimited: atomic.LoadInt64(&m.RateLimited),
HitRate: hitRate,
AvgResponseTime: avgResponseTime,
TotalBytesServed: atomic.LoadInt64(&m.TotalBytesServed),
TotalBytesCached: atomic.LoadInt64(&m.TotalBytesCached),
TotalBytesServed: atomic.LoadInt64(&m.TotalBytesServed),
TotalBytesSaved: atomic.LoadInt64(&m.TotalBytesSaved),
MemoryCacheSize: atomic.LoadInt64(&m.MemoryCacheSize),
DiskCacheSize: atomic.LoadInt64(&m.DiskCacheSize),
MemoryCacheHits: atomic.LoadInt64(&m.MemoryCacheHits),
@@ -218,7 +221,7 @@ func (m *Metrics) Reset() {
atomic.StoreInt64(&m.RateLimited, 0)
atomic.StoreInt64(&m.TotalResponseTime, 0)
atomic.StoreInt64(&m.TotalBytesServed, 0)
atomic.StoreInt64(&m.TotalBytesCached, 0)
atomic.StoreInt64(&m.TotalBytesSaved, 0)
atomic.StoreInt64(&m.MemoryCacheHits, 0)
atomic.StoreInt64(&m.DiskCacheHits, 0)
atomic.StoreInt64(&m.Promotions, 0)
@@ -247,9 +250,11 @@ type Stats struct {
RateLimited int64
HitRate float64
AvgResponseTime time.Duration
TotalBytesServed int64
TotalBytesCached int64
MemoryCacheSize int64
TotalBytesServed int64
TotalBytesSaved int64
MemoryCacheSize int64
DiskCacheSize int64
MemoryCacheHits int64
DiskCacheHits int64
@@ -291,7 +296,8 @@ func WriteText(w http.ResponseWriter, stats *Stats) {
_, _ = fmt.Fprintf(w, "hit_rate %.4f\n", stats.HitRate)
_, _ = fmt.Fprintf(w, "avg_response_time_ms %.2f\n", float64(stats.AvgResponseTime.Nanoseconds())/1e6)
_, _ = fmt.Fprintf(w, "total_bytes_served %d\n", stats.TotalBytesServed)
_, _ = fmt.Fprintf(w, "total_bytes_cached %d\n", stats.TotalBytesCached)
_, _ = fmt.Fprintf(w, "total_bytes_saved %d\n", stats.TotalBytesSaved)
_, _ = fmt.Fprintf(w, "memory_cache_size %d\n", stats.MemoryCacheSize)
_, _ = fmt.Fprintf(w, "disk_cache_size %d\n", stats.DiskCacheSize)
_, _ = fmt.Fprintf(w, "uptime_seconds %.2f\n", stats.Uptime.Seconds())
+15
View File
@@ -179,6 +179,17 @@ func New(address string, memorySize string, diskSize string, diskPath, upstream,
metrics: metrics.NewMetrics(),
}
// Wire metrics into TieredCache so promotions are counted.
c.SetMetrics(sc.metrics)
// Wire metrics into the concrete storage tiers (MemoryFS / DiskFS) so per-tier hits and evictions are counted.
if m != nil {
m.SetMetrics(sc.metrics)
}
if d != nil {
d.SetMetrics(sc.metrics)
}
// Wire the request processor (constructor injection of interfaces for the owned wrappers + vfs + client + scalars).
// Done after all fields including wrappers are set; before attach goroutines (no impact on lifecycle paths).
sc.processor = newRequestProcessor(sc)
@@ -209,12 +220,16 @@ func New(address string, memorySize string, diskSize string, diskPath, upstream,
sc.wg.Add(1)
go func() {
defer sc.wg.Done()
t0 := time.Now()
_ = d.Size()
select {
case <-sc.shutdownCh:
return
default:
c.SetSlow(dgc)
logger.Logger.Info().
Dur("attach_delay", time.Since(t0)).
Msg("Disk slow tier attached (mixed mode); prior traffic was memory-only")
}
}()
}
+13 -2
View File
@@ -3,6 +3,7 @@ package cache
import (
"io"
"s1d3sw1ped/steamcache2/steamcache/metrics"
"s1d3sw1ped/steamcache2/vfs"
"s1d3sw1ped/steamcache2/vfs/vfserror"
"sync/atomic"
@@ -10,8 +11,9 @@ import (
// TieredCache implements a lock-free two-tier cache for better concurrency
type TieredCache struct {
fast *atomic.Value // Memory cache (fast) - atomic.Value for lock-free access
slow *atomic.Value // Disk cache (slow) - atomic.Value for lock-free access
fast *atomic.Value // Memory cache (fast) - atomic.Value for lock-free access
slow *atomic.Value // Disk cache (slow) - atomic.Value for lock-free access
metrics *metrics.Metrics
}
// New creates a new tiered cache
@@ -22,6 +24,11 @@ func New() *TieredCache {
}
}
// SetMetrics allows wiring the top-level metrics collector (called from SteamCache).
func (tc *TieredCache) SetMetrics(m *metrics.Metrics) {
tc.metrics = m
}
// SetFast sets the fast (memory) tier atomically
func (tc *TieredCache) SetFast(vfs vfs.VFS) {
tc.fast.Store(vfs)
@@ -221,6 +228,10 @@ func (tc *TieredCache) promoteToFast(key string, reader io.ReadCloser) {
// Failure (e.g. mem pressure, concurrent evict) is non-fatal and does not affect correctness of slow tier.
_, _ = writer.Write(content)
_ = writer.Close()
if tc.metrics != nil {
tc.metrics.IncrementPromotions()
}
}
}
}
+51 -2
View File
@@ -7,6 +7,7 @@ import (
"os"
"path/filepath"
"s1d3sw1ped/steamcache2/steamcache/logger"
"s1d3sw1ped/steamcache2/steamcache/metrics"
"s1d3sw1ped/steamcache2/vfs"
"s1d3sw1ped/steamcache2/vfs/locks"
"s1d3sw1ped/steamcache2/vfs/lru"
@@ -43,6 +44,7 @@ type DiskFS struct {
// initCloseOnce ensures initDone closed exactly once even on panic in bg populator (panic safety for Issue 1).
initCloseOnce sync.Once
startupEvict func(vfs.VFS, uint) uint // passed to New (via gc.GetGCAlgorithm); invoked as last step of bg init if over cap (no post-ctor race)
metrics *metrics.Metrics
}
// shardPath converts a Steam cache key to a sharded directory path to reduce inode pressure
@@ -134,6 +136,12 @@ func New(root string, capacity int64, evict func(vfs.VFS, uint) uint) (*DiskFS,
return d, nil
}
// SetMetrics allows the owner (SteamCache) to inject the metrics collector
// so that per-tier hit and eviction counters can be recorded.
func (d *DiskFS) SetMetrics(met *metrics.Metrics) {
d.metrics = met
}
// calculateSizeAndPopulateIndex runs in background from New to avoid blocking startup or O(N) RAM for large caches (millions of Steam files).
// It streams batch inserts (bounded by maxEvictBatch) to keep lock times short and eliminate giant temporary slice.
// Startup over-capacity eviction (if needed) runs as the very last step (using the evict func passed to New, selected via gc.GetGCAlgorithm).
@@ -489,8 +497,14 @@ func (d *DiskFS) Open(key string) (io.ReadCloser, error) {
return nil, err
}
// Use memory mapping for large files (>1MB) to improve performance
const mmapThreshold = 1024 * 1024 // 1MB
// Use memory mapping for large files to improve performance.
// We use 8 MiB as the threshold because:
// - Most Steam chunks are ~1 MiB (see current disk cache analysis).
// - mmap has non-trivial fixed overhead (page tables, TLB, faults).
// - For files < ~4-8 MiB the overhead often outweighs the zero-copy benefit
// on mostly sequential access patterns.
// - Larger files benefit more from kernel readahead + zero-copy.
const mmapThreshold = 8 * 1024 * 1024 // 8 MiB
if fi.Size > mmapThreshold {
// Close the regular file handle
_ = file.Close() // best-effort; mmap path takes over or falls back
@@ -505,9 +519,21 @@ func (d *DiskFS) Open(key string) (io.ReadCloser, error) {
if err != nil {
_ = mmapFile.Close() // best-effort close before fallback open
// Fallback to regular file reading (intentional 3rd open of same path after mmap failure; pre-existing pattern, no leak)
if d.metrics != nil {
d.metrics.IncrementDiskCacheHits()
}
return os.Open(path)
}
// Hint to the kernel (on supported platforms) that we will access
// this mapping sequentially. This enables better readahead.
if err := madviseSequential(mapped); err != nil {
logger.Logger.Debug().
Err(err).
Str("key", key).
Msg("madvise(MADV_SEQUENTIAL) failed on mmap'd chunk")
}
return &mmapReadCloser{
data: mapped,
file: mmapFile,
@@ -515,6 +541,9 @@ func (d *DiskFS) Open(key string) (io.ReadCloser, error) {
}, nil
}
if d.metrics != nil {
d.metrics.IncrementDiskCacheHits()
}
return file, nil
}
@@ -677,6 +706,10 @@ func (d *DiskFS) EvictLRU(bytesNeeded uint) uint {
}
}
d.mu.Unlock()
if d.metrics != nil && evicted > 0 {
d.metrics.IncrementEvictions()
}
return evicted
}
@@ -726,6 +759,10 @@ func (d *DiskFS) EvictBySize(bytesNeeded uint, ascending bool) uint {
}
}
d.mu.Unlock()
if d.metrics != nil && evicted > 0 {
d.metrics.IncrementEvictions()
}
return evicted
}
@@ -773,6 +810,10 @@ func (d *DiskFS) EvictFIFO(bytesNeeded uint) uint {
}
}
d.mu.Unlock()
if d.metrics != nil && evicted > 0 {
d.metrics.IncrementEvictions()
}
return evicted
}
@@ -825,6 +866,10 @@ func (d *DiskFS) EvictLFU(bytesNeeded uint) uint {
}
}
d.mu.Unlock()
if d.metrics != nil && evicted > 0 {
d.metrics.IncrementEvictions()
}
return evicted
}
@@ -878,5 +923,9 @@ func (d *DiskFS) EvictHybrid(bytesNeeded uint) uint {
}
}
d.mu.Unlock()
if d.metrics != nil && evicted > 0 {
d.metrics.IncrementEvictions()
}
return evicted
}
+13
View File
@@ -0,0 +1,13 @@
//go:build !windows
package disk
import (
"golang.org/x/sys/unix"
)
// madviseSequential gives the OS a hint that the memory region will be
// accessed sequentially. This is a no-op or best-effort on some platforms.
func madviseSequential(b []byte) error {
return unix.Madvise(b, unix.MADV_SEQUENTIAL)
}
+11
View File
@@ -0,0 +1,11 @@
//go:build windows
package disk
// madviseSequential is a no-op on Windows.
// Windows file mappings don't have a direct equivalent to MADV_SEQUENTIAL
// in the same way. Sequential access hints are better done via
// FILE_FLAG_SEQUENTIAL_SCAN at file open time (future improvement possible).
func madviseSequential(b []byte) error {
return nil
}
+32
View File
@@ -5,6 +5,7 @@ import (
"bytes"
"fmt"
"io"
"s1d3sw1ped/steamcache2/steamcache/metrics"
"s1d3sw1ped/steamcache2/vfs"
"s1d3sw1ped/steamcache2/vfs/locks"
"s1d3sw1ped/steamcache2/vfs/lru"
@@ -33,6 +34,7 @@ type MemoryFS struct {
keyLocks []sync.Map // Sharded lock pools for better concurrency
LRU *lru.LRUList[*types.FileInfo]
timeUpdater *types.BatchedTimeUpdate // Batched time updates for better performance
metrics *metrics.Metrics
}
// New creates a new MemoryFS
@@ -55,6 +57,12 @@ func New(capacity int64) (*MemoryFS, error) {
}, nil
}
// SetMetrics allows the owner (SteamCache) to inject the metrics collector
// so that per-tier hit and eviction counters can be recorded.
func (m *MemoryFS) SetMetrics(met *metrics.Metrics) {
m.metrics = met
}
// Name returns the name of this VFS
func (m *MemoryFS) Name() string {
return "MemoryFS"
@@ -209,6 +217,10 @@ func (m *MemoryFS) Open(key string) (io.ReadCloser, error) {
// Use zero-copy approach - return reader that reads directly from buffer
m.mu.Unlock()
if m.metrics != nil {
m.metrics.IncrementMemoryCacheHits()
}
return &memoryReadCloser{
buffer: buffer,
offset: 0,
@@ -344,6 +356,10 @@ func (m *MemoryFS) EvictLRU(bytesNeeded uint) uint {
}
}
m.mu.Unlock()
if m.metrics != nil && evicted > 0 {
m.metrics.IncrementEvictions()
}
return evicted
}
@@ -395,6 +411,10 @@ func (m *MemoryFS) EvictBySize(bytesNeeded uint, ascending bool) uint {
}
}
m.mu.Unlock()
if m.metrics != nil && evicted > 0 {
m.metrics.IncrementEvictions()
}
return evicted
}
@@ -442,6 +462,10 @@ func (m *MemoryFS) EvictFIFO(bytesNeeded uint) uint {
}
}
m.mu.Unlock()
if m.metrics != nil && evicted > 0 {
m.metrics.IncrementEvictions()
}
return evicted
}
@@ -494,6 +518,10 @@ func (m *MemoryFS) EvictLFU(bytesNeeded uint) uint {
}
}
m.mu.Unlock()
if m.metrics != nil && evicted > 0 {
m.metrics.IncrementEvictions()
}
return evicted
}
@@ -548,5 +576,9 @@ func (m *MemoryFS) EvictHybrid(bytesNeeded uint) uint {
}
}
m.mu.Unlock()
if m.metrics != nil && evicted > 0 {
m.metrics.IncrementEvictions()
}
return evicted
}