148 lines
8.6 KiB
Markdown
148 lines
8.6 KiB
Markdown
# P1: Hardening, Correctness & Security Improvements
|
|
|
|
**Priority**: P1 — Important hardening and correctness work
|
|
**Theme**: Eliminate data integrity risks, resource exhaustion vectors, and incomplete security controls.
|
|
**Status**: Not started
|
|
**Depends on**: P0 (recommended — many P1 items are easier to verify once the service starts/stops cleanly)
|
|
|
|
## Goal
|
|
|
|
Make the cache **safe by default** against common failure modes, malicious or malformed input, and misconfiguration while preserving the high-performance characteristics required for Steam traffic.
|
|
|
|
## Overview
|
|
|
|
Even after P0 items are resolved, several classes of defects remain:
|
|
|
|
- Unbounded memory usage on large responses or cache promotion
|
|
- Incomplete / spoofable client identification used for rate limiting
|
|
- Overstated features (LFU, hybrid eviction) that do not actually work as documented
|
|
- Significant "smart caching" code (adaptive/predictive) that provides no actual benefit today
|
|
|
|
These items directly affect correctness, security posture, and user trust.
|
|
|
|
## Tasks
|
|
|
|
### P1-01: Implement bounded / streaming response handling (prevent OOM on large bodies)
|
|
|
|
- **Description**: `ServeHTTP` currently does `bodyData, err := io.ReadAll(resp.Body)` for every cache miss before deciding whether to serve or cache. Promotion paths do the same. There are no size limits.
|
|
- **Impact**:
|
|
- A single large (or malicious) response from upstream can exhaust RAM and crash the process.
|
|
- Steam chunks are usually small, but manifests, depots, and especially misconfigured upstreams can be very large.
|
|
- Coalesced request buffering also keeps full bodies in memory.
|
|
- **Affected Files**:
|
|
- `steamcache/steamcache.go` (ServeHTTP around lines 1505-1518, reconstruct, coalesced paths)
|
|
- `vfs/cache/cache.go` (promoteToFast)
|
|
- Possibly disk/memory write paths
|
|
- **Approach** (choose one or hybrid):
|
|
1. Preferred long-term: Stream to client with `io.TeeReader` (or custom tee) directly into the VFS `Create` writer while serving. Only buffer small responses.
|
|
2. Short-term mitigation: Add a hard per-request body cap (e.g. 64 MiB or configurable) and return 502/413 for anything larger without caching.
|
|
3. Make coalesced request buffering also respect a size limit or use a temp file for very large objects.
|
|
- **Acceptance Criteria**:
|
|
- No `io.ReadAll` of unbounded upstream responses in the hot path.
|
|
- Configurable or hard safety limit exists and is documented.
|
|
- Large responses are still served correctly (streaming) when they fit the limit.
|
|
- Existing Range + cache hit behavior is unaffected.
|
|
- New integration test that attempts a > limit response and verifies graceful handling.
|
|
- **Dependencies**: P0-04 (error metrics will help prove the new path works)
|
|
- **Effort**: Medium-Large (4-8 hours). Streaming tee writer is the cleanest but requires care with VFS `Create` semantics.
|
|
|
|
### P1-02: Make client IP extraction for rate limiting configurable and safe
|
|
|
|
- **Description**: `getClientIP` unconditionally trusts `X-Forwarded-For` and `X-Real-IP`.
|
|
- **Impact**:
|
|
- Any client can spoof its IP and bypass per-client `max_requests_per_client` limits.
|
|
- In environments with a real reverse proxy this is fine; in direct or partially proxied setups it is a DoS vector.
|
|
- **Affected Files**:
|
|
- `steamcache/steamcache.go` (getClientIP and getOrCreateClientLimiter)
|
|
- `config/config.go` (new settings)
|
|
- `cmd/root.go`
|
|
- **Approach**:
|
|
1. Add config options:
|
|
- `trusted_proxies: []string` (CIDR list) or `trust_x_forwarded_for: bool`
|
|
- Default should be conservative (`false` or empty list).
|
|
2. When not trusting forwarded headers, fall back strictly to `r.RemoteAddr`.
|
|
3. When trusting, implement proper "rightmost trusted proxy" logic (do not just take the first XFF entry blindly).
|
|
4. Document the security implications clearly in README.
|
|
- **Acceptance Criteria**:
|
|
- Default behavior is safe (does not trust arbitrary XFF).
|
|
- When `trusted_proxies` is configured, correct client IP is extracted.
|
|
- Spoofing tests exist (or at least negative tests).
|
|
- Per-client semaphore still works correctly.
|
|
- **Dependencies**: None
|
|
- **Effort**: Medium (3-5 hours including tests + docs)
|
|
|
|
### P1-03: Implement real LFU or remove the false claim; make "hybrid" meaningful
|
|
|
|
- **Description**:
|
|
- `EvictLFU` just calls `EvictBySizeAsc` (smallest first) with a TODO comment.
|
|
- `EvictHybrid` is literally just `EvictLRU`.
|
|
- Documentation in README and config examples heavily advertises these algorithms.
|
|
- **Impact**: Users who select `lfu` or `hybrid` get behavior they did not ask for. This is misleading and can produce worse cache hit rates than expected.
|
|
- **Affected Files**:
|
|
- `vfs/eviction/eviction.go`
|
|
- `vfs/memory/memory.go` (EvictLFU / EvictHybrid methods if they exist)
|
|
- `vfs/disk/disk.go`
|
|
- README.md (GC algorithm section)
|
|
- Possibly `config/config.go` comments
|
|
- **Approach** (two options — pick one):
|
|
**Option A (Recommended for P1)**: Implement a real (approximate) LFU using the existing `AccessCount` field already present in `FileInfo`.
|
|
**Option B**: Remove the non-functional choices from the public API and docs for now; keep only algorithms that actually do something different (LRU, FIFO, largest, smallest). Re-introduce LFU later under P2.
|
|
- **Acceptance Criteria**:
|
|
- Selecting `lfu` either does real LFU or is rejected at config validation time with a clear message.
|
|
- "hybrid" either has a documented size+recency policy or is removed.
|
|
- Unit tests exist that demonstrate different eviction behavior between the algorithms under controlled access patterns.
|
|
- **Dependencies**: P0-03 (so invalid algorithm names are caught early)
|
|
- **Effort**: Medium (if implementing real LFU: 4-6 hours; if removing: 1-2 hours)
|
|
|
|
### P1-04: Decide the fate of the adaptive/predictive caching subsystem
|
|
|
|
- **Description**: Large amounts of code (`vfs/adaptive/`, `vfs/predictive/`, plus fields and goroutines in `SteamCache`) collect access patterns but never actually change eviction strategy, promotion decisions, or GC algorithm at runtime.
|
|
- **Impact**:
|
|
- Wasted memory and CPU (multiple background analyzers + maps).
|
|
- Increased goroutine count and shutdown complexity.
|
|
- False advertising in README ("adaptive and predictive caching").
|
|
- Maintenance burden for code that provides zero user value today.
|
|
- **Affected Files**:
|
|
- `vfs/adaptive/adaptive.go`
|
|
- `vfs/predictive/predictive.go`
|
|
- `steamcache/steamcache.go` (record* methods, manager fields, New, Shutdown)
|
|
- `vfs/cache/cache.go` (promotion decisions)
|
|
- **Approach** (choose one):
|
|
1. **Prune (fast)**: Remove the unused subsystems, the recording calls, and all related goroutines/fields. Update docs. Keep the data structures in `types.FileInfo` if they are still useful for future work.
|
|
2. **Integrate (larger)**: Wire the analyzers into actual decisions (e.g., switch promotion aggressiveness, temporarily bias toward LFU-style scoring, pre-warm on predicted sequences). This is a P2-level project.
|
|
- **Acceptance Criteria** (for prune path):
|
|
- No more goroutines or memory overhead from these packages at runtime.
|
|
- `Shutdown` becomes simpler.
|
|
- README no longer claims adaptive/predictive behavior that does not exist.
|
|
- If kept for future, the packages are clearly marked "experimental / not yet active".
|
|
- **Dependencies**: None
|
|
- **Effort**: Prune = 2-4 hours. Full integration = multi-day project (defer to P2).
|
|
|
|
## Definition of Done (P1 Milestone)
|
|
|
|
- [ ] P1-01 (streaming/bounded bodies) implemented and load-tested.
|
|
- [ ] P1-02 (client IP trust) implemented with safe defaults + documentation.
|
|
- [ ] P1-03 (LFU/hybrid truthfulness) resolved (either real impl or removal + doc fixes).
|
|
- [ ] P1-04 (adaptive/predictive) decided and executed (prune is acceptable for P1).
|
|
- [ ] All changes have accompanying tests (unit + at least one integration test per major feature).
|
|
- [ ] `go test -race ./...` and manual long-running soak (with induced large responses and spoofed headers) pass.
|
|
- [ ] README and any user-facing docs are updated to reflect reality (no more over-claiming).
|
|
|
|
## Notes for Implementers
|
|
|
|
- P1-01 is the highest leverage item for stability under real-world (or adversarial) traffic.
|
|
- When implementing streaming writes, be careful with the current VFS `Create(key, declaredSize)` contract — it may need adjustment.
|
|
- Consider adding a `max_object_size` config knob as part of P1-01.
|
|
|
|
## References
|
|
|
|
- Original full code review
|
|
- `steamcache/steamcache.go:1506` (io.ReadAll)
|
|
- `vfs/cache/cache.go:206` (promotion ReadAll)
|
|
- `vfs/eviction/eviction.go:82` (LFU TODO)
|
|
- Large unused packages in `vfs/adaptive` and `vfs/predictive`
|
|
|
|
---
|
|
|
|
**After P1**: The service should be safe to expose to untrusted Steam clients on a LAN with reasonable resource protections.
|