Commit Graph

148 Commits

Author SHA1 Message Date
eva 3411defd51 docs: Real DNS + empty upstream Quick Start
CI / vulncheck (pull_request) Successful in 15s
CI / check-and-test (pull_request) Successful in 40s
2026-09-07 14:58:06 +00:00
linus 71d5106777 docs: Add README operator cache check (#30)
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 41s
Release Tag / release (push) Successful in 16s
README Quick check + make validate-check using existing /metrics and heartbeat. Closes #29.
1.0.23
2026-09-03 00:19:53 -05:00
pike 8b1b229539 docs: Deduplicate validate-check helper
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
Keep a single Makefile validate-check target that prints the full
/metrics dump, highlights hit/miss fields, and curls /lancache-heartbeat.
Drop the duplicate README troubleshooting item and align the validation
chapter plus validate-config.yaml comments with that behavior.

Link: #29
2026-09-03 05:18:19 +00:00
pike ff1ab31327 docs: Add README operator cache check
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
The README buried hit/miss verification under the SteamPrefill validation
chapter, and advertised make validate-check without a Makefile target.
Operators need a short start → traffic → metrics path against the existing
/metrics and /lancache-heartbeat endpoints.

Link: #29
2026-09-03 05:16:14 +00:00
s1d3sw1ped_bot 85e14bc8af README: Replace --threads with real cobra concurrency flags
Release Tag / release (push) Successful in 17s
Docs sync: align README with current install/CLI behavior.

Co-authored-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
Co-committed-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
1.0.22
2026-09-02 10:43:52 -05:00
s1d3sw1ped_bot 3f5175b482 .gitea/workflows: Prefer job GITEA_TOKEN
CI / vulncheck (push) Successful in 15s
CI / check-and-test (push) Successful in 41s
Release Tag still used secrets.RELEASE_TOKEN. Prefer the job built-in token with contents: write, same path as vulture/teleport.

Link: #26
Co-authored-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
Co-committed-by: s1d3sw1ped_bot <s1d3sw1ped+giteabot@gmail.com>
2026-09-02 10:08:00 -05:00
s1d3sw1ped_bot 04d1c6c368 Merge pull request 'Promote develop: vfs/cache promoteToFast race fix' (#24) from develop into main
CI / vulncheck (push) Successful in 15s
CI / check-and-test (push) Successful in 41s
Release Tag / release (push) Successful in 30s
Promote vfs/cache Stat snapshot + promoteToFast size fix.
1.0.21
2026-09-01 15:36:49 -05:00
s1d3sw1ped_bot 523a9a4782 Merge pull request 'vfs/cache: Fix promoteToFast race with WriteCloser Close' (#23) from vfs/cache-promote-race into develop
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
Stat returns FileInfo snapshots; promoteToFast sizes from len(content).

#21
2026-09-01 15:35:20 -05:00
s1d3sw1ped_bot 8e09c89e24 vfs/cache: Fix promoteToFast race with WriteCloser Close
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 40s
promoteToFast read FileInfo.Size from Stat after the VFS lock was
released, while WriteCloser.Close updated Size on the same live
object. Concurrent Create/Open on overlapping keys tripped the race
detector.

Stat now returns a FileInfo snapshot, and promotion uses the
ReadAll length for the fast-tier Create size. Promotion stays
best-effort.

#21
2026-09-01 20:30:40 +00:00
s1d3sw1ped_bot a2ac13d317 vfs/disk: Fix EvictDiskVisibilityAndRecreateSafety flake
CI / vulncheck (push) Successful in 13s
CI / check-and-test (push) Failing after 40s
2026-09-01 15:23:10 -05:00
s1d3sw1ped_bot 5d006ac44f vfs/disk: Fix EvictDiskVisibilityAndRecreateSafety flake
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 41s
2026-09-01 15:21:05 -05:00
s1d3sw1ped_bot acd006d4a2 ci: Skip test workflow on markdown-only pushes
CI / vulncheck (pull_request) Successful in 14s
CI / check-and-test (pull_request) Successful in 40s
Docs-only pushes to main (**.md / CONTRIBUTING.md) do not need the
full check-and-test job. Pull requests are unchanged.
2026-09-01 20:19:19 +00:00
s1d3sw1ped_bot 30a695458e vfs/disk: Fix EvictDiskVisibilityAndRecreateSafety flake
New() launches background calculateSizeAndPopulateIndex which scans
disk and calls insertBatch. Create does not wait on initDone, so a
file can be written and indexed, discovered by the scan, then removed
from d.info and disk by EvictLRU/EvictBySize. insertBatch then
re-inserted the stale discoveredFile without checking the path still
existed, so Stat succeeded from the index while os.Stat failed.

Re-stat under the lock and skip gone files so evicted keys are not
resurrected.

Fixes #18.
2026-09-01 20:19:14 +00:00
s1d3sw1ped_bot e8bcf0ddbd docs: Add CONTRIBUTING.md
CI / vulncheck (push) Successful in 14s
CI / check-and-test (push) Successful in 41s
Add short contribution guide on main from develop tip content.
2026-09-01 14:11:02 -05:00
s1d3sw1ped_bot f497e71ef0 docs: Add CONTRIBUTING.md
CI / vulncheck (pull_request) Successful in 16s
CI / check-and-test (pull_request) Successful in 43s
Copy the develop tip contribution guide onto main so the default
branch carries the same short CONTRIBUTING.md.
2026-09-01 19:10:15 +00:00
s1d3sw1ped_bot 0198e8990b docs: Add CONTRIBUTING.md
CI / vulncheck (pull_request) Successful in 31s
CI / check-and-test (pull_request) Failing after 57s
Contributors need a short guide for develop-targeted PRs, commit
subject form, and Gitea issue-closing rules.
2026-09-01 13:37:56 -05:00
s1d3sw1ped_bot 2a2cd8d393 Bump Go to 1.27.0 (#14)
CI / vulncheck (push) Successful in 8s
CI / check-and-test (push) Successful in 35s
2026-08-31 20:04:49 -05:00
s1d3sw1ped_bot 81b3a7df53 Restrict Host-based fetches to Steam CDN names (#13)
CI / vulncheck (push) Successful in 7s
CI / check-and-test (push) Successful in 27s
Allowlist steamcontent/steampowered/steamstatic Hosts; do not follow upstream redirects.
2026-08-31 18:58:53 -05:00
s1d3sw1ped_bot 8e8e877533 Restrict Host-based origin fetches to Steam CDN names.
CI / vulncheck (pull_request) Successful in 7s
CI / check-and-test (pull_request) Successful in 28s
When upstream is empty the cache used the client Host as the fetch URL, so any LAN client with a spoofed Steam User-Agent could proxy to literal IPs or arbitrary names. Reject those hosts, stop following upstream redirects, and keep path-only cache keys so real Steam CDNs still share entries.
2026-08-31 23:56:03 +00:00
s1d3sw1ped d63d7b4d3c Merge pull request 'Run CI on push to main' (#11) from ci/test-on-push into main
CI / vulncheck (push) Successful in 8s
CI / check-and-test (push) Successful in 28s
Release Tag / release (push) Successful in 14s
Reviewed-on: #11
1.0.20
2026-08-31 15:47:07 -05:00
s1d3sw1ped_bot 35d698a232 Keep v1 lint rules under golangci-lint v2 (no ST/QF, no G704/G705).
CI / vulncheck (pull_request) Successful in 7s
CI / check-and-test (pull_request) Successful in 38s
2026-08-31 20:30:36 +00:00
s1d3sw1ped_bot 36b613b6cd Drop empty gosec settings so golangci-lint v2 config verifies.
CI / vulncheck (pull_request) Successful in 7s
CI / check-and-test (pull_request) Failing after 13s
2026-08-31 20:28:54 +00:00
s1d3sw1ped_bot 79f02d9868 Fix CI on Go 1.26.7: golangci-lint-action v8 and vulncheck on 1.27.
CI / vulncheck (pull_request) Successful in 19s
CI / check-and-test (pull_request) Failing after 27s
v4 installs golangci-lint v1.64.8 (built with Go 1.24) which cannot lint a
1.26 module; v8 pulls v2.12. govulncheck@latest still fails to install on
1.26.7, so run that job on Go 1.27. Keep tests on push to main.
2026-08-31 20:27:35 +00:00
s1d3sw1ped_bot 19497eba0c Pin CI to Go 1.26.7 and stable action tags.
CI / vulncheck (pull_request) Failing after 13s
CI / check-and-test (pull_request) Failing after 23s
setup-go@main plus go-version-file 1.26.0 was flaky (version: not found) and still scanned an unpatched 1.26.0 stdlib.
2026-08-31 15:18:59 -05:00
s1d3sw1ped_bot c7a2312994 Raise module Go version from EOL 1.23.0 to 1.26.0.
CI / check-and-test (pull_request) Failing after 11s
CI / vulncheck (pull_request) Failing after 21s
govulncheck fails on stdlib crypto/tls and crypto/x509 findings (including GO-2025-4008 / CVE-2025-58189) that were never patched on 1.23. 1.26 is still a supported release.
2026-08-31 15:15:48 -05:00
s1d3sw1ped_bot fbb084d824 Use latest Go 1.23 patch in CI and run tests even if govulncheck fails.
CI / vulncheck (pull_request) Failing after 18s
CI / check-and-test (pull_request) Successful in 37s
setup-go was installing go.mod's 1.23.0 exactly, so govulncheck reported stdlib x509 findings and skipped tests. check-latest gets the patched 1.23, and vulncheck is its own job.
2026-08-31 15:11:43 -05:00
s1d3sw1ped_bot e7d4a19c3f Let govulncheck@latest fetch a newer Go toolchain.
CI / check-and-test (pull_request) Failing after 22s
CI was dying on: golang.org/x/vuln@v1.7.0 requires go >= 1.25.0 (running go 1.23.0; GOTOOLCHAIN=local).
2026-08-31 15:11:18 -05:00
s1d3sw1ped_bot 0c54ef3404 Let govulncheck@latest fetch a newer Go toolchain.
CI / check-and-test (pull_request) Failing after 24s
CI was dying on: golang.org/x/vuln@v1.7.0 requires go >= 1.25.0 (running go 1.23.0; GOTOOLCHAIN=local).
2026-08-31 15:08:11 -05:00
s1d3sw1ped_bot 3d3c74fdb2 Run CI on push to main as well as pull requests.
CI / check-and-test (pull_request) Failing after 1m1s
Direct pushes to main currently skip tests because the workflow only listened for pull_request. Scratchbox already tests every push; do the same here.
2026-08-31 15:05:28 -05:00
s1d3sw1ped 04f55535a5 Refactor validation process and update configuration examples
- Replaced the `validate-with-prefill.sh` script with a streamlined `make validate` command for improved usability.
- Updated `validate-config.yaml` to clarify cache management instructions and garbage collection algorithms.
- Enhanced comments to provide better guidance on upstream configurations and their implications for caching setups.
2026-05-28 21:06:28 -05:00
s1d3sw1ped 05640bb549 Refine README for validation server instructions and configuration clarity
- Removed outdated quick start section to streamline the validation process.
- Updated the validation server description for better clarity and accessibility.
- Enhanced the explanation of the validation configuration file to emphasize its importance and usage.
2026-05-28 21:05:58 -05:00
s1d3sw1ped e4be82cddf Remove obsolete validate-check target from Makefile to streamline validation process. Updated help message to reflect this change, enhancing clarity in available commands. 2026-05-28 20:31:06 -05:00
s1d3sw1ped 60b2c3e514 Update Makefile to include linting in build, test, and test-race targets
- Added `lint` as a prerequisite for the `build`, `test`, and `test-race` targets to ensure code quality checks are performed before executing tests and builds.
- This enhancement promotes better code hygiene and consistency across the development workflow.
2026-05-28 20:29:36 -05:00
s1d3sw1ped 099e5347d5 Enhance Makefile and README for improved validation and cache management
- Updated the Makefile to include a new `clean-disk` target for removing disk cache, and modified the `run-validation` target to clean the disk cache before starting.
- Enhanced the `check-review-labels` target to include additional file types in the search for temporary review labels, improving code hygiene checks.
- Refined the README.md to clarify the hardening section and improve the description of the `prefill` command.
- Removed the obsolete `test_cache/.gitkeep` file to clean up the repository.
2026-05-28 20:26:23 -05:00
s1d3sw1ped b7e3a0da86 Update metrics tracking and enhance cache eviction strategies
Release Tag / release (push) Successful in 34s
- Added metrics for bytes saved from cache to improve performance insights.
- Updated cache eviction strategies in MemoryFS and DiskFS to include metrics tracking for hits and evictions.
- Enhanced README.md with updated garbage collection algorithm descriptions and recommendations for cache usage.
- Introduced new madviseSequential functionality for improved memory access hints on Unix systems.
- Adjusted validation configuration in examples to better reflect realistic usage scenarios.
1.0.19
2026-05-28 10:31:23 -05:00
s1d3sw1ped 3fd72705fc Enhance Makefile and documentation for validation workflow
- Added new targets in the Makefile for validation, including `run-validation`, `validate-check`, and `validate-kill`, to streamline the testing process with external tools like SteamPrefill.
- Introduced a `setcap` target to manage necessary capabilities for running the server on port 80 without root access.
- Updated README.md to include detailed instructions for validating functionality, including quick start guides and troubleshooting tips.
- Improved .gitignore to exclude validation artifacts and logs, ensuring a cleaner repository.
2026-05-28 04:15:24 -05:00
s1d3sw1ped c3464d692e Add core components for request coalescing and service management
- Introduced coalescing logic in `coalescing.go` to handle concurrent identical upstream fetches, including a state machine and response buffering for improved performance.
- Implemented a new cache file format in `format.go`, supporting serialization and deserialization of HTTP responses, along with range request handling.
- Developed an HTTP handler in `handler.go` to manage requests, including special endpoint handling and metrics reporting.
- Added rate limiting functionality in `ratelimit.go` to control per-client and global request rates, enhancing security and performance.
- Created service management capabilities in `service.go` to define and manage cacheable services, including user-agent detection.
- Updated tests in `steamcache_test.go` to cover new functionalities, ensuring robustness and reliability across the codebase.
2026-05-28 01:17:30 -05:00
s1d3sw1ped 843772e9f7 Refactor golangci-lint configuration and improve error handling
- Updated .golangci.yml to enable default linters and refine suppression rules, enhancing code quality visibility.
- Improved error handling in cmd/root.go by explicitly discarding low-value error messages during fatal exits for consistency with errcheck posture.
- Added best-effort error handling in various locations across the codebase, ensuring that non-critical errors are logged without affecting overall functionality.
- Introduced a new writeMetricsText function to streamline metrics output, improving code clarity and maintainability.
2026-05-27 18:51:33 -05:00
s1d3sw1ped feda55e225 Enhance DiskFS initialization and error handling
- Updated `disk.New` to support asynchronous initialization for large caches, improving responsiveness during startup.
- Introduced an eviction function parameter to `disk.New`, ensuring proper handling of over-capacity scenarios.
- Enhanced error handling in various components, including memory and disk tests, to ensure robustness and clarity.
- Refactored tests to validate new behaviors, including checks for delayed attachment and proper error propagation.
- Removed obsolete error handling code and tests related to the now-deleted errors package, streamlining the codebase.
2026-05-27 13:15:33 -05:00
s1d3sw1ped 4861f93e6f Update AGENTS.md and Makefile for review hygiene guidelines
- Added a new section in AGENTS.md outlining the importance of not leaving temporary review labels in source code or comments.
- Updated the error message in the Makefile's check-review-labels target to reference AGENTS.md for review hygiene rules instead of plans/README.md, ensuring consistency in documentation.
2026-05-27 03:07:25 -05:00
s1d3sw1ped ffa9aa04f7 Refactor Makefile and enhance disk/memory eviction tests
- Updated the 'bench' target in the Makefile to run all benchmarks for MemoryFS and DiskFS, improving clarity and coverage.
- Added explicit post-eviction consistency checks in DiskFS tests to ensure on-disk files are removed after eviction.
- Introduced new benchmarks for memory eviction strategies under pressure, enhancing test coverage for memory management.
- Improved error handling in benchmark tests for both disk and memory file systems, ensuring robustness during performance evaluations.
- Refactored key generation in tests for consistency and clarity.
2026-05-27 03:02:34 -05:00
s1d3sw1ped 6f28362790 Add /plans/ to .gitignore 2026-05-27 02:13:48 -05:00
s1d3sw1ped 0dbb2e02ed Remove plans/ directory (P0/P1/P2 work complete) 2026-05-27 02:12:21 -05:00
s1d3sw1ped 0c1840d223 chore: capture post-P0/P1 state for clean P2 start (working tree was dirty at task begin) 2026-05-27 00:53:49 -05:00
s1d3sw1ped 9cb38a9a18 Enhance SteamCache shutdown and coalesced request handling
- Implemented a more robust shutdown mechanism using sync.Once to prevent multiple shutdown calls and ensure all background managers are stopped properly.
- Refactored coalesced request handling to utilize atomic operations for waiting counts, improving thread safety and performance.
- Introduced a done channel for coalesced requests to signal completion, enhancing the handling of concurrent requests and reducing potential deadlocks.
- Updated logging to provide better insights into cache request processing and error handling.
2026-05-26 23:14:47 -05:00
s1d3sw1ped 41777cd9a4 Refactor Makefile to streamline build and test processes
- Consolidated build commands by replacing 'build-snapshot-single' with a generic 'build' target.
- Enhanced test commands to include shuffling and a race detector for improved reliability.
- Updated help output to reflect new build and test targets, improving usability for developers.
2026-05-26 22:47:56 -05:00
s1d3sw1ped 8a4a7728ed Remove production hardening review document
The implementation work based on the review is complete and the
resulting code changes are already merged. The review document
itself (with all the round-by-round notes) is no longer needed in
the repository.
2026-05-26 22:45:14 -05:00
s1d3sw1ped 953ac4d9d8 Finalize .gitignore after review document inclusion
Removed broad docs/ ignore now that the final production hardening
review is tracked.
2026-05-26 22:41:29 -05:00
s1d3sw1ped 928a5d74cf Add final production hardening review document
Include the completed review (with all Status/Response updates and
Completion Note) as the authoritative record of the work done.

This closes out the post-implement merge cleanup.
2026-05-26 22:41:23 -05:00
s1d3sw1ped cfa65c423c Remove temporary config/config_test.go
This test file was added back temporarily during the post-implement
cleanup so that its removal could be recorded explicitly as part of
the production hardening merge.

It was originally scaffolding from the implement session and is no
longer needed (the minimal Validate + GetDefaultConfig support was
added directly to config.go instead).
2026-05-26 22:41:10 -05:00