Commit Graph

41 Commits

Author SHA1 Message Date
bd123bc63a Refactor module naming and update references to steamcache2
All checks were successful
Release Tag / release (push) Successful in 9s
- Changed module name from `s1d3sw1ped/SteamCache2` to `s1d3sw1ped/steamcache2` for consistency.
- Updated all import paths and references throughout the codebase to reflect the new module name.
- Adjusted README and Makefile to use the updated module name, ensuring clarity in usage instructions.
2025-09-21 23:10:21 -05:00
46495dc3aa Refactor caching functions and simplify response serialization
All checks were successful
Release Tag / release (push) Successful in 27s
- Updated the `downloadThroughCache` function to remove the upstream URL parameter, streamlining the caching process.
- Modified the `serializeRawResponse` function to eliminate unnecessary parameters, enhancing clarity and usability.
- Adjusted integration tests to align with the new function signatures, ensuring consistent testing of caching behavior.
2025-09-21 22:55:49 -05:00
45ae234694 Enhance caching mechanisms and introduce adaptive features
- Updated caching logic to support size-based promotion filtering, ensuring that not all files may be promoted based on size constraints.
- Implemented adaptive caching strategies with a new AdaptiveCacheManager to analyze access patterns and adjust caching strategies dynamically.
- Introduced predictive caching features with a PredictiveCacheManager to prefetch content based on access patterns.
- Added a CacheWarmer to preload popular content into the cache, improving access times for frequently requested files.
- Refactored memory management with a DynamicCacheManager to adjust cache sizes based on system memory usage.
- Enhanced VFS interface and file metadata handling to support new features and improve performance.
- Updated tests to validate new caching behaviors and ensure reliability of the caching system.
2025-09-21 22:47:13 -05:00
694c223b00 Add integration tests and service management for SteamCache
- Introduced integration tests for SteamCache to validate caching behavior with real Steam URLs.
- Implemented a ServiceManager to manage service configurations, allowing for dynamic detection of services based on User-Agent.
- Updated cache key generation to include service prefixes, enhancing cache organization and retrieval.
- Enhanced the caching logic to support multiple services, starting with Steam and Epic Games.
- Improved .gitignore to exclude test cache files while retaining necessary structure.
2025-09-21 20:07:18 -05:00
9ca8fa4a5e Add concurrency limits and configuration options for SteamCache
- Introduced maxConcurrentRequests and maxRequestsPerClient fields in the Config struct to manage request limits.
- Updated the SteamCache implementation to utilize these new configuration options for controlling concurrent requests.
- Enhanced the ServeHTTP method to enforce global and per-client rate limiting using semaphores.
- Modified the root command to accept new flags for configuring concurrency limits via command-line arguments.
- Updated tests to reflect changes in the SteamCache initialization and request handling logic.
2025-09-02 06:50:42 -05:00
ee6fc32a1a Update content type validation in ServeHTTP method for Steam files
- Changed expected Content-Type from "application/octet-stream" to "application/x-steam-chunk" to align with Steam's file specifications.
- Enhanced warning message for unexpected content types to provide clearer context for debugging.
2025-09-02 05:48:24 -05:00
4a4579b0f3 Refactor caching logic and enhance hash generation in steamcache
- Replaced SHA1 hash calculations with SHA256 for improved security and consistency in cache key generation.
- Introduced a new TestURLHashing function to validate the new cache key generation logic.
- Removed outdated hash calculation tests and streamlined the caching process to focus on URL-based hashing.
- Implemented lightweight validation methods in ServeHTTP to enhance performance and reliability of cached responses.
- Added batched time updates in VFS implementations for better performance during access time tracking.
2025-09-02 05:45:44 -05:00
b9358a0e8d Refactor steamcache.go to simplify code and improve readability
- Removed the min function and the verifyResponseHash function to streamline the codebase.
- Updated extractHashFromSteamPath to use strings.TrimPrefix for cleaner path handling.
- Retained comments regarding removed Prometheus metrics for future reference.
2025-09-02 05:03:15 -05:00
c197841960 Refactor configuration management and enhance build process
- Introduced a YAML-based configuration system, allowing for automatic generation of a default `config.yaml` file.
- Updated the application to load configuration settings from the YAML file, improving flexibility and ease of use.
- Added a Makefile to streamline development tasks, including running the application, testing, and managing dependencies.
- Enhanced `.gitignore` to include build artifacts and configuration files.
- Removed unused Prometheus metrics and related code to simplify the codebase.
- Updated dependencies in `go.mod` and `go.sum` for improved functionality and performance.
2025-09-02 05:01:42 -05:00
56bb1ddc12 Add hop-by-hop header handling in ServeHTTP method
All checks were successful
Release Tag / release (push) Successful in 12s
- Introduced a map for hop-by-hop headers to be removed from responses.
- Enhanced cache serving logic to read and filter HTTP responses, ensuring only relevant headers are forwarded.
- Updated cache writing to handle full HTTP responses, improving cache integrity and performance.
2025-07-19 05:07:36 -05:00
9c65cdb156 Fix HTTP status code for root path in ServeHTTP method to ensure correct response for upstream verification
All checks were successful
Release Tag / release (push) Successful in 12s
2025-07-19 04:42:20 -05:00
ae013f9a3b Enhance SteamCache configuration and HTTP client settings
All checks were successful
Release Tag / release (push) Successful in 14s
- Added upstream server configuration to launch.json for improved connectivity.
- Increased HTTP client timeout from 60s to 120s for better handling of slow responses.
- Updated server timeouts in steamcache.go: increased ReadTimeout to 30s and WriteTimeout to 60s.
- Introduced ReadHeaderTimeout to mitigate header attacks and set MaxHeaderBytes to 1MB.
- Improved error logging in the Run method to include HTTP status codes for better debugging.
- Adjusted ServeHTTP method to handle root path and metrics endpoint correctly.
2025-07-19 04:40:05 -05:00
847931ed43 Update .goreleaser.yaml and enhance HTTP client settings in steamcache.go
All checks were successful
PR Check / check-and-test (pull_request) Successful in 18s
- Removed copyright footer from .goreleaser.yaml.
- Increased HTTP client connection settings in steamcache.go for improved performance:
  - MaxIdleConns from 100 to 200
  - MaxIdleConnsPerHost from 10 to 50
  - IdleConnTimeout from 90s to 120s
  - TLSHandshakeTimeout from 10s to 15s
  - ResponseHeaderTimeout from 10s to 30s
  - ExpectContinueTimeout from 1s to 5s
  - Added DisableCompression and ForceAttemptHTTP2 options.
- Removed debug logging for manifest files in ServeHTTP method.
2025-07-19 04:12:56 -05:00
163e64790c Enhance garbage collection and caching functionality
All checks were successful
PR Check / check-and-test (pull_request) Successful in 21s
- Updated .gitignore to include all .exe files and ensure .smashignore is tracked.
- Expanded README.md with advanced configuration options for garbage collection algorithms, detailing available algorithms and use cases.
- Modified launch.json to include memory and disk garbage collection flags for better configuration.
- Refactored root.go to introduce memoryGC and diskGC flags for garbage collection algorithms.
- Implemented hash extraction and verification in steamcache.go to ensure data integrity during caching.
- Added new tests in steamcache_test.go for hash extraction and verification, ensuring correctness of caching behavior.
- Enhanced garbage collection strategies in gc.go, introducing LFU, FIFO, Largest, Smallest, and Hybrid algorithms with corresponding metrics.
- Updated caching logic to conditionally cache responses based on hash verification results.
2025-07-19 02:27:04 -05:00
3427b8f5bc fix: gc was being stupid allowing another thread to take the space it made before it could not anymore
All checks were successful
PR Check / check-and-test (pull_request) Successful in 12s
2025-07-13 07:50:22 -05:00
6c98d03ae7 fix: trim query parameters from URL path in ServeHTTP to ensure cache key correctness
All checks were successful
PR Check / check-and-test (pull_request) Successful in 16s
2025-07-13 05:42:07 -05:00
539f14e8ec refactor: moved the GC stuff around and corrected all tests
All checks were successful
PR Check / check-and-test (pull_request) Successful in 30s
2025-07-13 04:20:12 -05:00
1673e9554a Refactor VFS implementation to use Create and Open methods
Some checks failed
PR Check / check-and-test (pull_request) Failing after 11m4s
- Updated disk_test.go to replace Set and Get with Create and Open methods for better clarity and functionality.
- Modified fileinfo.go to include package comment.
- Refactored gc.go to streamline garbage collection handling and removed unused statistics.
- Updated gc_test.go to comment out large random tests for future implementation.
- Enhanced memory.go to implement LRU caching and metrics for memory usage.
- Updated memory_test.go to replace Set and Get with Create and Open methods.
- Removed sync.go as it was redundant and not utilized.
- Updated vfs.go to reflect changes in the VFS interface, replacing Set and Get with Create and Open.
- Added package comments to vfserror.go for consistency.
2025-07-13 03:17:22 -05:00
b83836f914 fix: update log message for server startup and improve request handling in ServeHTTP
All checks were successful
PR Check / check-and-test (pull_request) Successful in 1m6s
2025-07-12 09:48:06 -05:00
b4d2b1305e fix: add logging for unsupported methods and error handling in ServeHTTP
All checks were successful
PR Check / check-and-test (pull_request) Successful in 1m6s
2025-07-12 08:50:34 -05:00
f378d0e81f feat: update dependencies and improve caching mechanism
Some checks failed
PR Check / check-and-test (pull_request) Failing after 2m11s
- Added Prometheus client library for metrics collection.
- Refactored garbage collection strategy from random deletion to LRU (Least Recently Used) deletion.
- Introduced per-key locking in cache to prevent race conditions.
- Enhanced logging with structured log messages for cache hits and misses.
- Implemented a retry mechanism for upstream requests with exponential backoff.
- Updated Go modules and indirect dependencies for better compatibility and performance.
- Removed unused sync filesystem implementation.
- Added version initialization to ensure a default version string.
2025-07-12 06:43:00 -05:00
8c1bb695b8 fix: enhance logging to handle empty upstream values
All checks were successful
Release Tag / release (push) Successful in 10s
2025-01-23 11:31:28 -06:00
70786da8c6 fix: improve logging readability and remove configuration messages
All checks were successful
Release Tag / release (push) Successful in 10s
2025-01-23 11:25:18 -06:00
e24af47697 feat: add upstream and verbose flags to command line interface
All checks were successful
Release Tag / release (push) Successful in 13s
feat: add upstream support allowing to chain cache servers if needed
fix: tweaked garbage collection to be better
2025-01-23 11:14:39 -06:00
6fe80c82ad fix: reduce cache hits averaging size for improved performance
All checks were successful
Release Tag / release (push) Successful in 9s
2025-01-22 20:59:24 -06:00
4a69c4ba66 fix: add empty line in logging for improved readability
All checks were successful
Release Tag / release (push) Successful in 9s
2025-01-22 20:51:32 -06:00
2d0fe6571d feat: enhance garbage collection logging with total GC time and improved stat formatting
All checks were successful
Release Tag / release (push) Successful in 9s
2025-01-22 20:48:05 -06:00
550948951e feat: implement enhanced garbage collection statistics logging
All checks were successful
Release Tag / release (push) Successful in 12s
2025-01-22 20:27:12 -06:00
4a23eecae0 fix: reduce log interval to 1 second for more frequent statistics updates
All checks were successful
Release Tag / release (push) Successful in 28s
2025-01-22 19:37:30 -06:00
7401c040dc feat: add configurations for memory only, disk only, and memory & disk modes
All checks were successful
Release Tag / release (push) Successful in 14s
2025-01-22 19:28:45 -06:00
3e8a92b865 fix: log memory statistics only if memory is enabled
All checks were successful
Release Tag / release (push) Successful in 9s
2025-01-22 18:29:19 -06:00
08b8d0ce3d fix: enhance logging for memory and disk statistics in cache 2025-01-22 18:23:53 -06:00
6b0a556690 feat: include application version in logging output 2025-01-22 17:49:58 -06:00
d49c4e1799 fix: increase hits buffer size in AvgCacheState initialization 2025-01-22 17:49:58 -06:00
0ca2a9eeed feat: integrate zerolog for structured logging and replace fmt with logger 2025-01-22 17:49:58 -06:00
0d8e8acf3a refactor: replace log statements with fmt for console output 2025-01-22 17:49:58 -06:00
837960e41b fix: initialize the avgcachestatus 2025-01-22 17:49:58 -06:00
86754b21aa refactor: comment out logging for request duration in ServeHTTP method 2025-01-22 17:49:58 -06:00
5682c1e9c8 feat: add AvgCacheState to track average cache hits and misses 2025-01-22 17:49:58 -06:00
bc7744b5a7 fix: handle error from http.ListenAndServe in SteamCache 2025-01-22 17:49:58 -06:00
f54150c3d2 initial commit 2025-01-22 17:49:22 -06:00