steamcache2/plans/P2.md

P2: Performance, Quality & Maintainability Improvements

Priority: P2 — Performance, refactoring, and long-term health
Theme: Make the codebase faster, smaller, easier to reason about, and production-operable at scale.
Status: Not started
Depends on: P0 strongly recommended; P1 nice-to-have for some verification steps

Goal

Turn a clever but monolithic prototype into a high-quality, maintainable Go project that is pleasant to work on and easy to operate.

Overview

After the critical stability (P0) and safety (P1) work, the project still carries technical debt that affects:

• Runtime performance under sustained load (lock contention, unnecessary copies)
• Developer velocity (huge source file, magic numbers, copy-paste)
• Operational visibility (weak metrics, no benchmarks, incomplete CI)
• Correctness confidence (very low test coverage on the storage layer)

These items are important for long-term success but are not immediate crash or security risks.

Tasks

P2-01: Refactor the monolithic ServeHTTP and split steamcache/steamcache.go

• Description: The core request handler is a single ~500+ line function with many responsibilities (authz, rate limiting, coalescing, upstream fetch, cache write, metrics, adaptive recording). The file itself is 1724 lines.
• Impact:
  • Extremely hard to test individual behaviors in isolation.
  • High risk of regression when touching any part of request handling.
  • New contributors are intimidated.
• Affected Files:
  • steamcache/steamcache.go (primary)
  • Potentially new files: steamcache/handler.go, steamcache/coalescing.go, steamcache/upstream.go, steamcache/response.go, etc.
• Approach:
  1. Extract clear types for the request context (e.g. requestContext holding clientIP, cacheKey, service, timing, etc.).
  2. Break ServeHTTP into smaller focused methods: handleCacheHit, handleCoalesced, fetchAndCache, writeCacheEntry, etc.
  3. Move pure helper logic (range parsing, response reconstruction, hash generation) into separate small files if they aren't already.
  4. Keep the SteamCache struct as the coordinator but reduce its god-object nature over time.
• Acceptance Criteria:
  • No single function in the package > 150 lines.
  • ServeHTTP itself becomes a thin dispatcher (< 80 lines).
  • All existing behavior (including edge cases around coalescing + errors + Ranges) still passes the test suite.
  • New unit tests become feasible for the extracted pieces.
• Dependencies: None (can be done in parallel with other P2 work)
• Effort: Large (8-16 hours). Best done as a series of small, reviewable refactors rather than one giant PR.

P2-02: Reduce lock contention during eviction

• Description: EvictLRU, EvictBySize, etc. take the global mu.Lock() on the entire MemoryFS/DiskFS for the duration of the scan + deletion loop.
• Impact: Under cache pressure (very common when the disk cache fills), all other operations (Open, Stat, Create) serialize behind the eviction. This can cause request latency spikes even for hot memory-tier hits.
• Affected Files:
  • vfs/memory/memory.go (eviction methods)
  • vfs/disk/disk.go (eviction methods)
• Approach options:
  1. Collect candidates under read lock, then do the actual deletes and size updates in a second phase or in small batches while briefly acquiring write locks.
  2. Move eviction into a dedicated background goroutine that the GC layer signals, using finer-grained coordination.
  3. Use a "generation" or "watermark" approach so readers can proceed while eviction cleans up.
• Acceptance Criteria:
  • Benchmark or load test shows improved tail latencies for Open/Stat while eviction is running.
  • No data races introduced (race detector clean).
  • Total size and LRU invariants remain correct after concurrent eviction.
• Dependencies: Good test coverage on the VFS layer (see P2-04)
• Effort: Medium-Large (4-8 hours + measurement)

P2-03: Dramatically improve test coverage on the VFS and storage layer

• Description: Most vfs/* packages currently have 0% coverage. The critical storage, eviction, GC, and tiering logic is almost untested in isolation.
• Impact:
  • Very low confidence when changing eviction, promotion, or GC behavior.
  • Hard to catch regressions in size accounting, LRU ordering, or sharded locking.
  • Blocks safe execution of P2-02 and future performance work.
• Affected Areas (need new or expanded tests):
  • vfs/memory/*
  • vfs/disk/*
  • vfs/gc/*
  • vfs/cache/*
  • vfs/eviction/*
  • vfs/locks/* and vfs/lru/* (at least basic)
• Approach:
  1. Write focused unit tests for each VFS implementation using t.TempDir for disk.
  2. Add property-style or table-driven tests that verify size never exceeds capacity after many Create + Evict cycles.
  3. Test concurrent Create/Open/Delete/Delete under load (with -race).
  4. Test promotion, tier fallback, and lazy discovery paths.
  5. Add benchmarks (BenchmarkMemoryFS_CreateOpen, BenchmarkEvictionUnderPressure, etc.).
• Acceptance Criteria:
  • Combined coverage for all vfs/* packages ≥ 70% (statement).
  • At least one benchmark per major component that can be run in CI or locally.
  • New tests catch at least one real bug during development (celebrated in commit message).
• Dependencies: None
• Effort: Large (12-20 hours spread over multiple sessions). High leverage.

P2-04: Clean up build, CI, linting, and repository hygiene

• Description:
  • Makefile run / run-debug targets are hardcoded to a Windows binary path.
  • dist/ artifacts are committed even though .gitignore lists /dist/.
  • No golangci-lint, no go vet in CI, no vulnerability scanning.
  • Test target exists but coverage reporting and gates are missing.
• Impact: Painful local development on non-Windows machines. Risk of shipping known-bad artifacts. Harder to maintain code quality over time.
• Affected Files:
  • Makefile
  • .gitea/workflows/test-pr.yaml (and release workflow)
  • .gitignore (verify dist is truly ignored)
  • Possibly add .golangci.yml
• Approach:
  1. Fix Makefile to use go run . or build a platform-appropriate binary.
  2. Add a make lint target and wire golangci-lint (with reasonable defaults + errcheck, gosec, etc.).
  3. Update Gitea workflows to run go vet, lint, and (optionally) govulncheck.
  4. Remove any committed files under dist/ (or add them to .gitignore more aggressively and git-rm them).
  5. Consider adding a coverage report step (even if not enforcing a hard gate yet).
• Acceptance Criteria:
  • make test and make run work cleanly on Linux and macOS.
  • CI runs lint + vet and fails the PR on new issues.
  • Repository no longer contains built binaries in its tree.
  • go mod tidy + build is reproducible.
• Dependencies: None
• Effort: Small-Medium (3-5 hours)

P2-05: Use the existing rich error types consistently and improve observability

• Description: A nice steamcache/errors package with context, unwrap, retry classification, and client/server error helpers exists but is almost unused. Metrics are still very basic.
• Impact:
  • Lost opportunity for better structured logging and error handling.
  • Harder to write generic retry / circuit-breaker logic later.
  • /metrics and logs give limited insight into what actually failed and why.
• Approach:
  1. Audit the top 10-15 error sites in ServeHTTP and VFS layers.
  2. Convert the most important ones to use NewSteamCacheError* helpers.
  3. Wire more structured fields into zerolog calls using the error types.
  4. Expand the metrics package (per-service error counts, upstream error breakdown, cache write failures, etc.).
  5. Consider exporting Prometheus-style metrics in addition to the current text format (optional).
• Acceptance Criteria:
  • At least the major error categories (upstream fetch, cache corruption, rate limit, validation) use the custom error types.
  • /metrics surface becomes more useful (new counters for categories of errors).
  • No behavior change for clients (still get the same HTTP status codes).
• Dependencies: P0-04 (basic error counting)
• Effort: Medium (4-6 hours)

Definition of Done (P2 Milestone)

• Major refactoring (P2-01) landed in reviewable chunks; ServeHTTP is no longer a monster function.
• Eviction lock contention measurably reduced (P2-02).
• VFS/storage layer has ≥70% test coverage + benchmarks (P2-03).
• Build/CI hygiene is excellent: cross-platform make targets, lint in CI, clean repo (P2-04).
• Error handling and metrics are noticeably better (P2-05).
• go test -race -shuffle=on ./... + go vet + linter are all green.
• A new contributor can run make help, make test, make run on Linux/macOS without friction.
• At least one performance or quality regression has been prevented by the new tests/benchmarks during the work.

Suggested Order of Execution (within P2)

1. P2-04 (CI hygiene) — cheap wins, improves confidence for everything else.
2. P2-03 (test coverage on VFS) — unblocks safe work on P2-02.
3. P2-02 (eviction locking).
4. P2-01 (big refactor) — do this when the test safety net is stronger.
5. P2-05 (errors + observability) — can run in parallel with others.

Notes for Implementers

• P2 work has the highest risk of "refactoring for its own sake." Every change should be justified by either a concrete performance win, a maintainability win that reduces future bug rate, or enabling future features.
• Keep changes reviewable. Large refactors should be broken into multiple PRs with clear "no behavior change" invariants.
• The custom LRU list and sharded locking are clever — make sure any refactoring preserves their performance characteristics.

References

• steamcache/steamcache.go:1724 (file size)
• vfs/disk/disk.go and vfs/memory/memory.go eviction methods (global lock held)
• .gitea/workflows/
• Makefile
• steamcache/errors/errors.go (under-used)

---

After P2: The project should feel like a mature, professional Go service rather than a sophisticated prototype.