steamcache2/README.md

# SteamCache2

SteamCache2 is a blazing fast download cache for Steam, designed to reduce bandwidth usage and speed up game downloads.

## Features

- High-speed caching for Steam downloads
- Tiered storage for getting the most out of your storage media
- Garbage Collected storage for limiting the size of RAM or Disk cache and will not go above what you choose or stop caching unlike others
- Reduces bandwidth usage
- Easy to set up and configure aside from dns stuff to trick Steam into using it
- Supports multiple clients
- **NEW:** YAML configuration system with automatic config generation
- **NEW:** Simple Makefile for development workflow
- Cross-platform builds (Linux, macOS, Windows)

## Quick Start

### First Time Setup

1. **Clone and build:**
   ```bash
   git clone <repository-url>
   cd steamcache2
   make  # This will run tests and build the application
   ```

2. **Run the application** (it will create a default config):
   ```bash
   ./steamcache2
   # or on Windows:
   steamcache2.exe
   ```

   The application will automatically create a `config.yaml` file with default settings and exit, allowing you to customize it.

3. **Edit the configuration** (`config.yaml`):
   ```yaml
   listen_address: :80
   cache:
       memory:
           size: 1GB
           gc_algorithm: lru
       disk:
           size: 10GB
           path: ./disk
           gc_algorithm: hybrid
   upstream: "https://steam.cdn.com"  # Set your upstream server
   ```

4. **Run the application again:**
   ```bash
   make run  # or ./steamcache2
   ```

### Development Workflow

Use `make` for the majority of common development tasks. The Makefile handles running tests, linting, hygiene checks, building, running the application, and other routine boilerplate work.

Run `make help` to see the full list of available commands.

This is the preferred approach for day-to-day development. Avoid running raw `go test`, `go run`, or `golangci-lint` commands directly for routine tasks.

**Important rule**: Do not leave temporary review labels (P2-05, T1, I3, R2, "per Issue 7", etc.) in source code or comments. See `plans/README.md` → "Review & Implementation Hygiene" for details. `make check-review-labels` (part of `make lint`) will catch violations.

### Command Line Flags

While most configuration is done via the YAML file, some runtime options are still available as command-line flags:

```bash
# Use a custom config file
./steamcache2 --config /path/to/my-config.yaml

# Set logging level
./steamcache2 --log-level debug --log-format json

# Set number of worker threads
./steamcache2 --threads 8

# Show help
./steamcache2 --help
```

### Configuration

SteamCache2 uses a YAML configuration file (`config.yaml`) for all settings. Here's a complete configuration example:

```yaml
# Server configuration
listen_address: :80

# P1 hardening (see Security Hardening section)
max_object_size: "0"   # 0=unlimited; set e.g. "256MB" for response size DoS protection
trusted_proxies: []    # empty = safe (ignore XFF for rate limit); set CIDRs for trusted proxies

# Cache configuration
cache:
  # Memory cache settings
  memory:
    # Size of memory cache (e.g., "512MB", "1GB", "0" to disable)
    size: 1GB
    # Garbage collection algorithm
    gc_algorithm: lru

  # Disk cache settings
  disk:
    # Size of disk cache (e.g., "10GB", "50GB", "0" to disable)
    size: 10GB
    # Path to disk cache directory
    path: ./disk
    # Garbage collection algorithm
    gc_algorithm: hybrid

# Upstream server configuration
# The upstream server to proxy requests to
upstream: "https://steam.cdn.com"
```

#### Startup Validation
As of P0, `steamcache2` performs strict validation on startup (after loading config + CLI overrides, before creating the cache). Invalid configs cause immediate clean failure (no default written, no panic):

- Negative `max_concurrent_requests` / `max_requests_per_client`: "negative concurrency not allowed"
- Invalid `gc_algorithm` (memory): "invalid memory gc algorithm: badvalue"
- Disk enabled (`size` non-zero/"") but no `path`: "disk cache enabled but no path specified"
- Invalid memory/disk `size` strings (via direct New): "invalid memory size: ..." / "invalid disk size: ..." (clean error return, no panic)

Example error on stderr + logs:
```
Error: Invalid configuration: invalid memory gc algorithm: foo. Please fix the config file and try again.
```

See `config.Validate()` and `steamcache.New` error paths. This ensures the LAN appliance fails fast on misconfig.

#### Security Hardening (P1)
- `max_object_size` (default "0" = unlimited): set e.g. "256MB" or "512MB" to reject oversized upstream responses with HTTP 413 before buffering/ReadAll. Prevents OOM DoS from large or malicious responses (P1-01). Large legitimate Steam files still served if under limit.
- `trusted_proxies`: CIDR list (default empty). When empty (safe default), X-Forwarded-For and client IP spoofing are ignored for rate limiting — always uses `r.RemoteAddr` only. When set (e.g. your reverse proxy CIDR), uses correct "rightmost untrusted" extraction. Prevents bypass of `max_requests_per_client` (P1-02). Documented for LAN proxy setups only.
- These + P0 validation make steamcache2 safe-by-default for LAN exposure.

#### Migration / Breaking Changes (P1)
- `New()` public signature gained 2 required trailing params (`maxObjectSize`, `trustedProxies`). Direct callers (rare; most use config or NewWithOptions) must update.
- Recommended: migrate to `NewWithOptions(Options{...})` (non-breaking) or rely on YAML config + cmd/root.go.
- No behavior change for existing configs (defaults preserve prior semantics).

#### Garbage Collection Algorithms

SteamCache2 supports different garbage collection algorithms for memory and disk caches, allowing you to optimize performance for each storage tier:

**Available GC Algorithms:**

- **`lru`** (default): Least Recently Used - evicts oldest accessed files
- **`lfu`**: Least Frequently Used (P1 real impl) - evicts by lowest AccessCount (tiebreak older ATime); uses existing FileInfo counters
- **`fifo`**: First In, First Out - evicts oldest created files (predictable)
- **`largest`**: Size-based - evicts largest files first (maximizes file count)
- **`smallest`**: Size-based - evicts smallest files first (maximizes cache hit rate)
- **`hybrid`**: Recency + frequency hybrid (P1 meaningful) - evicts by lowest time-decayed score (GetTimeDecayedScore combining ATime + AccessCount)

**Recommended Algorithms by Cache Type:**

**For Memory Cache (Fast, Limited Size):**
- **`lru`** - Best overall performance, good balance of speed and hit rate
- **`lfu`** - Excellent for gaming cafes where popular games stay cached
- **`hybrid`** - Optimal for mixed workloads with varying file sizes

**For Disk Cache (Slow, Large Size):**
- **`hybrid`** - Recommended for optimal performance, balances speed and storage efficiency
- **`largest`** - Good for maximizing number of cached files
- **`lru`** - Reliable default with good performance

**Use Cases:**
- **Gaming Cafes**: Use `lfu` for memory, `hybrid` for disk
- **LAN Events**: Use `lfu` for memory, `hybrid` for disk
- **Home Use**: Use `lru` for memory, `hybrid` for disk
- **Testing**: Use `fifo` for predictable behavior
- **Large File Storage**: Use `largest` for disk to maximize file count

### DNS Configuration

Configure your DNS to direct Steam traffic to your SteamCache2 server:

- If you're on Windows and don't want a whole network implementation, see the [Windows Hosts File Override](#windows-hosts-file-override) section below.

### Windows Hosts File Override

1. Open Notepad as Administrator:
    - Click on the Start menu, type `Notepad`, right-click on Notepad, and select `Run as administrator`.

2. Open the Hosts File:
    - In Notepad, go to `File` > `Open`.
    - Navigate to `C:\Windows\System32\drivers\etc`.
    - Select `All Files` from the dropdown menu to see the hosts file.
    - Open the `hosts` file.

3. Add the Override Entry:
    - At the end of the file, add a new line with the IP address of your SteamCache2 server followed by `lancache.steamcontent.com`. For example:
      ```plaintext
      192.168.1.100 lancache.steamcontent.com
      ```
      Replace `192.168.1.100` with the actual IP address of your SteamCache2 server.

4. Save the Hosts File:
    - Save the changes by going to `File` > `Save`.

5. Flush DNS Cache (optional but recommended):
    - Open Command Prompt as Administrator.
    - Run the following command to flush the DNS cache:
      ```sh
      ipconfig /flushdns
      ```

6. Restart
    - Restart Steam or Restart Your PC

This will direct any requests to `lancache.steamcontent.com` to your SteamCache2 server.

## Building from Source

### Prerequisites

- Go 1.19 or later
- Make (optional, but recommended)

### Build Commands

```bash
# Clone the repository
git clone <repository-url>
cd SteamCache2

# Download dependencies
make deps

# Run tests
make test

# Build for current platform
go build -o steamcache2 .

# Build for specific platforms
GOOS=linux GOARCH=amd64 go build -o steamcache2-linux-amd64 .
GOOS=windows GOARCH=amd64 go build -o steamcache2-windows-amd64.exe .
```

### Development

```bash
# Run in development mode with debug logging
make run-debug

# Run all tests and start the application
make
```

## Troubleshooting

### Common Issues

1. **"Config file not found" on first run**
   - This is expected! SteamCache2 will automatically create a default `config.yaml` file
   - Edit the generated config file with your desired settings
   - Run the application again

2. **Permission denied when creating config**
   - Make sure you have write permissions in the current directory
   - Try running with elevated privileges if necessary

3. **Port already in use**
   - Change the `listen_address` in `config.yaml` to a different port (e.g., `:8080`)
   - Or stop the service using the current port

4. **High memory usage**
   - Reduce the memory cache size in `config.yaml`
   - Consider using disk-only caching by setting `memory.size: "0"`

5. **Slow disk performance**
   - Use SSD storage for the disk cache
   - Consider using a different GC algorithm like `hybrid`
   - Adjust the disk cache size to match available storage

### Getting Help

- Check the logs for detailed error messages
- Run with `--log-level debug` for more verbose output
- Ensure your upstream server is accessible
- Verify DNS configuration is working correctly

## License

See the [LICENSE](LICENSE) file for details.
But just for clarity this covers all files in this project unless stated in the individual file.

## Acknowledgements

- Inspired by [Lancache.net](https://lancache.net/)