initial commit
This commit is contained in:
@@ -0,0 +1,5 @@
|
||||
# Build artifacts
|
||||
/bin/
|
||||
|
||||
# Coverage outputs
|
||||
/*.out
|
||||
@@ -0,0 +1,40 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Purpose
|
||||
|
||||
This repository hosts a small self-hosted scratch service (pastebin-like) written in Go.
|
||||
|
||||
Key product constraints:
|
||||
- All routes are unauthenticated by design.
|
||||
- One upload creates one scratch.
|
||||
- For multiple files, users must upload a single archive file.
|
||||
- Scratch data is stored gzip-compressed on disk transparently.
|
||||
|
||||
## Project layout
|
||||
|
||||
- `cmd/pastebin`: service entrypoint
|
||||
- `internal/config`: config loading and validation
|
||||
- `internal/http`: HTTP handlers and middleware
|
||||
- `internal/storage`: filesystem storage and metadata index
|
||||
- `internal/cleanup`: expiration cleanup worker
|
||||
- `web/templates`, `web/static`: UI assets
|
||||
|
||||
## API routes
|
||||
|
||||
- `POST /api/scratch`
|
||||
- `GET /api/scratch/{id}`
|
||||
- `GET /s/{id}`
|
||||
- `GET /raw/{id}`
|
||||
|
||||
## Build and test
|
||||
|
||||
Use `make help` as the source of truth for available build/test targets and descriptions.
|
||||
|
||||
## Coding conventions
|
||||
|
||||
- Keep changes minimal and focused.
|
||||
- Preserve unauthenticated behavior unless explicitly requested.
|
||||
- Keep limits/operator controls configurable via config.
|
||||
- Maintain write-route protections (allowlist and rate limiting).
|
||||
- Keep storage behavior transparent for clients (compression is internal).
|
||||
- Add or update tests for behavior changes.
|
||||
@@ -0,0 +1,31 @@
|
||||
APP_NAME := scratchbox
|
||||
CMD_DIR := ./cmd/pastebin
|
||||
BIN_DIR := ./bin
|
||||
BIN := $(BIN_DIR)/$(APP_NAME)
|
||||
CONFIG ?= config.yaml
|
||||
|
||||
.PHONY: help run build test fmt clean
|
||||
|
||||
help:
|
||||
@echo "Targets:"
|
||||
@echo " make run - Run the server (CONFIG=$(CONFIG))"
|
||||
@echo " make build - Build binary to $(BIN)"
|
||||
@echo " make test - Run all Go tests"
|
||||
@echo " make fmt - Format Go code"
|
||||
@echo " make clean - Remove built artifacts"
|
||||
|
||||
run:
|
||||
go run $(CMD_DIR) -config $(CONFIG)
|
||||
|
||||
build:
|
||||
mkdir -p $(BIN_DIR)
|
||||
go build -o $(BIN) $(CMD_DIR)
|
||||
|
||||
test:
|
||||
go test ./...
|
||||
|
||||
fmt:
|
||||
go fmt ./...
|
||||
|
||||
clean:
|
||||
rm -rf $(BIN_DIR)
|
||||
@@ -0,0 +1,90 @@
|
||||
# Scratchbox Pastebin
|
||||
|
||||
Scratchbox is a minimal self-hosted pastebin service written in Go. It is intentionally unauthenticated and optimized for temporary sharing of text or small files with an expiration time.
|
||||
|
||||
Each upload creates exactly one scratch. For multi-file sharing, bundle files into an archive (for example `.zip` or `.tar.gz`) and upload that single archive file.
|
||||
Scratch content is stored on disk gzip-compressed transparently to reduce filesystem usage; API/UI reads return original uncompressed content.
|
||||
|
||||
## Quick Start
|
||||
|
||||
1. Copy the sample config:
|
||||
- `cp config.example.yaml config.yaml`
|
||||
2. Adjust settings for your environment.
|
||||
3. Start the server:
|
||||
- `go run ./cmd/pastebin -config config.yaml`
|
||||
4. Open:
|
||||
- `http://127.0.0.1:8080/`
|
||||
|
||||
Without `-config`, built-in defaults are used.
|
||||
|
||||
## Configuration
|
||||
|
||||
All configuration is YAML.
|
||||
|
||||
### `server`
|
||||
|
||||
- `listen_addr`: address for the HTTP server (required).
|
||||
- Default: `:8080`
|
||||
|
||||
### `limits`
|
||||
|
||||
- `max_upload_size`: max request body size for uploads.
|
||||
- Examples: `5MB`, `100MB`, `1GiB`
|
||||
- Default: `100MB`
|
||||
- `default_ttl`: expiration applied to new scratches.
|
||||
- Must be `> 0` and `<= 24h`
|
||||
- Default: `1h`
|
||||
|
||||
### `storage`
|
||||
|
||||
- `data_dir`: directory for scratch files and metadata index.
|
||||
- Default: `./data`
|
||||
- `cleanup_interval`: background interval for deleting expired content.
|
||||
- Must be at least `10s`
|
||||
- Default: `1m`
|
||||
|
||||
### `security`
|
||||
|
||||
- `allowed_ips`: optional list of IPs and CIDRs allowed to create scratches.
|
||||
- Applies to write route only: `POST /api/scratch`
|
||||
- Empty list means no allowlist restriction
|
||||
- `trust_proxy_headers`: if `true`, client IP extraction honors `X-Forwarded-For` then `X-Real-IP`.
|
||||
- Keep `false` unless behind a trusted reverse proxy that sets these headers.
|
||||
- `rate_limit.enabled`: enable per-IP token-bucket rate limiting on write route.
|
||||
- Default: `true`
|
||||
- `rate_limit.requests_per_minute`: steady refill rate per client IP.
|
||||
- Must be `> 0`
|
||||
- Default: `30`
|
||||
- `rate_limit.burst`: immediate burst capacity per client IP.
|
||||
- Must be `> 0`
|
||||
- Default: `10`
|
||||
|
||||
## Security Posture
|
||||
|
||||
This service is intentionally unauthenticated. Anyone with network access can read scratches, and (unless restricted) create scratches.
|
||||
|
||||
- No authentication, user accounts, or moderation controls.
|
||||
- No malware scanning.
|
||||
- Abuse controls are limited to upload size limits, TTL expiration, optional IP allowlisting, and write-route rate limiting.
|
||||
- Treat this as a convenience utility, not a hardened internet-facing platform.
|
||||
|
||||
## LAN Deployment Notes
|
||||
|
||||
For home/lab/LAN-only use:
|
||||
|
||||
- Bind to a private address, for example `127.0.0.1:8080` (single host) or `192.168.x.x:8080`.
|
||||
- Use `allowed_ips` to restrict write access to trusted subnets.
|
||||
- Keep `max_upload_size` and `default_ttl` small.
|
||||
- Keep `trust_proxy_headers` disabled unless using a trusted reverse proxy.
|
||||
|
||||
## Public Deployment Notes
|
||||
|
||||
If exposed beyond a trusted LAN, place a reverse proxy in front (Nginx, Caddy, Traefik, etc.) and add external controls:
|
||||
|
||||
- TLS termination and strict transport settings.
|
||||
- Additional request filtering/WAF controls.
|
||||
- Stronger rate limits at the edge.
|
||||
- Optional network-level restrictions to write route.
|
||||
- Monitoring and log retention suitable for abuse triage.
|
||||
|
||||
Recommended baseline: run behind a reverse proxy, keep low TTL and size limits, and use `allowed_ips` for write access whenever possible.
|
||||
@@ -0,0 +1,65 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"flag"
|
||||
"log"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"scratchbox/internal/cleanup"
|
||||
"scratchbox/internal/config"
|
||||
httpapi "scratchbox/internal/http"
|
||||
"scratchbox/internal/storage"
|
||||
)
|
||||
|
||||
func main() {
|
||||
configPath := flag.String("config", "", "path to config file")
|
||||
flag.Parse()
|
||||
|
||||
cfg, err := config.Load(*configPath)
|
||||
if err != nil {
|
||||
log.Fatalf("config load failed: %v", err)
|
||||
}
|
||||
|
||||
logger := log.New(os.Stdout, "[pastebin] ", log.LstdFlags|log.LUTC)
|
||||
|
||||
store, err := storage.NewFilesystemStore(cfg.Storage.DataDir)
|
||||
if err != nil {
|
||||
logger.Fatalf("storage init failed: %v", err)
|
||||
}
|
||||
|
||||
server, err := httpapi.NewServer(cfg, store, logger)
|
||||
if err != nil {
|
||||
logger.Fatalf("http init failed: %v", err)
|
||||
}
|
||||
|
||||
cleanupWorker := cleanup.NewWorker(store, cfg.Storage.CleanupIntervalParsed, logger)
|
||||
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer cancel()
|
||||
|
||||
go cleanupWorker.Start(ctx)
|
||||
|
||||
httpServer := &http.Server{
|
||||
Addr: cfg.Server.ListenAddr,
|
||||
Handler: server.Routes(),
|
||||
}
|
||||
|
||||
go func() {
|
||||
<-ctx.Done()
|
||||
shutdownCtx, shutdownCancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer shutdownCancel()
|
||||
if err := httpServer.Shutdown(shutdownCtx); err != nil {
|
||||
logger.Printf("http shutdown failed: %v", err)
|
||||
}
|
||||
}()
|
||||
|
||||
logger.Printf("listening on %s", cfg.Server.ListenAddr)
|
||||
if err := httpServer.ListenAndServe(); err != nil && err != http.ErrServerClosed {
|
||||
logger.Fatalf("http server exited with error: %v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,336 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"flag"
|
||||
"io"
|
||||
"log"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"scratchbox/internal/config"
|
||||
httpapi "scratchbox/internal/http"
|
||||
"scratchbox/internal/storage"
|
||||
)
|
||||
|
||||
func repoRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("runtime.Caller failed")
|
||||
}
|
||||
return filepath.Clean(filepath.Join(filepath.Dir(file), "..", ".."))
|
||||
}
|
||||
|
||||
func TestMainProcessSuccess(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tmp := t.TempDir()
|
||||
cfgPath := filepath.Join(tmp, "config.yaml")
|
||||
cfg := `
|
||||
server:
|
||||
listen_addr: "127.0.0.1:0"
|
||||
limits:
|
||||
max_upload_size: "1MB"
|
||||
default_ttl: "1h"
|
||||
storage:
|
||||
data_dir: "` + filepath.Join(tmp, "data") + `"
|
||||
cleanup_interval: "10s"
|
||||
security:
|
||||
allowed_ips: []
|
||||
trust_proxy_headers: false
|
||||
rate_limit:
|
||||
enabled: true
|
||||
requests_per_minute: 10
|
||||
burst: 1
|
||||
`
|
||||
if err := os.WriteFile(cfgPath, []byte(cfg), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
cmd := exec.Command(os.Args[0], "-test.run=TestMainHelperProcess")
|
||||
cmd.Dir = repoRoot(t)
|
||||
cmd.Env = append(os.Environ(),
|
||||
"GO_WANT_MAIN_HELPER=1",
|
||||
"MAIN_HELPER_MODE=success",
|
||||
"MAIN_HELPER_CONFIG="+cfgPath,
|
||||
)
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("helper process failed: %v, output=%s", err, string(out))
|
||||
}
|
||||
}
|
||||
|
||||
func TestMainProcessBadConfigFails(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cmd := exec.Command(os.Args[0], "-test.run=TestMainHelperProcess")
|
||||
cmd.Dir = repoRoot(t)
|
||||
cmd.Env = append(os.Environ(),
|
||||
"GO_WANT_MAIN_HELPER=1",
|
||||
"MAIN_HELPER_MODE=bad-config",
|
||||
)
|
||||
if err := cmd.Run(); err == nil {
|
||||
t.Fatalf("expected helper process failure for bad config")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMainProcessStorageInitFails(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tmp := t.TempDir()
|
||||
cfgPath := filepath.Join(tmp, "config.yaml")
|
||||
dataFile := filepath.Join(tmp, "data-file")
|
||||
if err := os.WriteFile(dataFile, []byte("x"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
cfg := `
|
||||
server:
|
||||
listen_addr: "127.0.0.1:0"
|
||||
limits:
|
||||
max_upload_size: "1MB"
|
||||
default_ttl: "1h"
|
||||
storage:
|
||||
data_dir: "` + dataFile + `"
|
||||
cleanup_interval: "10s"
|
||||
security:
|
||||
allowed_ips: []
|
||||
trust_proxy_headers: false
|
||||
rate_limit:
|
||||
enabled: true
|
||||
requests_per_minute: 10
|
||||
burst: 1
|
||||
`
|
||||
if err := os.WriteFile(cfgPath, []byte(cfg), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
cmd := exec.Command(os.Args[0], "-test.run=TestMainHelperProcess")
|
||||
cmd.Dir = repoRoot(t)
|
||||
cmd.Env = append(os.Environ(),
|
||||
"GO_WANT_MAIN_HELPER=1",
|
||||
"MAIN_HELPER_MODE=storage-fail",
|
||||
"MAIN_HELPER_CONFIG="+cfgPath,
|
||||
)
|
||||
if err := cmd.Run(); err == nil {
|
||||
t.Fatalf("expected helper process failure for storage init")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMainProcessHTTPInitFails(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tmp := t.TempDir()
|
||||
cfgPath := filepath.Join(tmp, "config.yaml")
|
||||
cfg := `
|
||||
server:
|
||||
listen_addr: "127.0.0.1:0"
|
||||
limits:
|
||||
max_upload_size: "1MB"
|
||||
default_ttl: "1h"
|
||||
storage:
|
||||
data_dir: "` + filepath.Join(tmp, "data") + `"
|
||||
cleanup_interval: "10s"
|
||||
security:
|
||||
allowed_ips: []
|
||||
trust_proxy_headers: false
|
||||
rate_limit:
|
||||
enabled: true
|
||||
requests_per_minute: 10
|
||||
burst: 1
|
||||
`
|
||||
if err := os.WriteFile(cfgPath, []byte(cfg), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
cmd := exec.Command(os.Args[0], "-test.run=TestMainHelperProcess")
|
||||
cmd.Dir = repoRoot(t)
|
||||
cmd.Env = append(os.Environ(),
|
||||
"GO_WANT_MAIN_HELPER=1",
|
||||
"MAIN_HELPER_MODE=http-fail",
|
||||
"MAIN_HELPER_CONFIG="+cfgPath,
|
||||
)
|
||||
if err := cmd.Run(); err == nil {
|
||||
t.Fatalf("expected helper process failure for http init")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMainHelperProcess(t *testing.T) {
|
||||
if os.Getenv("GO_WANT_MAIN_HELPER") != "1" {
|
||||
return
|
||||
}
|
||||
|
||||
flag.CommandLine = flag.NewFlagSet(os.Args[0], flag.ExitOnError)
|
||||
mode := os.Getenv("MAIN_HELPER_MODE")
|
||||
switch mode {
|
||||
case "success":
|
||||
cfgPath := os.Getenv("MAIN_HELPER_CONFIG")
|
||||
if cfgPath == "" {
|
||||
os.Exit(2)
|
||||
}
|
||||
go func() {
|
||||
time.Sleep(120 * time.Millisecond)
|
||||
proc, err := os.FindProcess(os.Getpid())
|
||||
if err == nil {
|
||||
_ = proc.Signal(os.Interrupt)
|
||||
}
|
||||
}()
|
||||
os.Args = []string{"pastebin", "-config", cfgPath}
|
||||
main()
|
||||
os.Exit(0)
|
||||
case "bad-config":
|
||||
os.Args = []string{"pastebin", "-config", "/no/such/config.yaml"}
|
||||
main()
|
||||
os.Exit(0)
|
||||
case "storage-fail":
|
||||
cfgPath := os.Getenv("MAIN_HELPER_CONFIG")
|
||||
os.Args = []string{"pastebin", "-config", cfgPath}
|
||||
main()
|
||||
os.Exit(0)
|
||||
case "http-fail":
|
||||
cfgPath := os.Getenv("MAIN_HELPER_CONFIG")
|
||||
if err := os.Chdir(t.TempDir()); err != nil {
|
||||
os.Exit(2)
|
||||
}
|
||||
os.Args = []string{"pastebin", "-config", cfgPath}
|
||||
main()
|
||||
os.Exit(0)
|
||||
default:
|
||||
os.Exit(2)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLiveServerEndToEndHTTP(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
cfgPath := filepath.Join(tmp, "config.yaml")
|
||||
cfgRaw := `
|
||||
server:
|
||||
listen_addr: "127.0.0.1:0"
|
||||
limits:
|
||||
max_upload_size: "1MB"
|
||||
default_ttl: "1h"
|
||||
storage:
|
||||
data_dir: "` + filepath.Join(tmp, "data") + `"
|
||||
cleanup_interval: "10s"
|
||||
security:
|
||||
allowed_ips: []
|
||||
trust_proxy_headers: false
|
||||
rate_limit:
|
||||
enabled: true
|
||||
requests_per_minute: 30
|
||||
burst: 10
|
||||
`
|
||||
if err := os.WriteFile(cfgPath, []byte(cfgRaw), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
cfg, err := config.Load(cfgPath)
|
||||
if err != nil {
|
||||
t.Fatalf("config.Load() error = %v", err)
|
||||
}
|
||||
logger := log.New(io.Discard, "", 0)
|
||||
store, err := storage.NewFilesystemStore(cfg.Storage.DataDir)
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
|
||||
wd, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatalf("Getwd() error = %v", err)
|
||||
}
|
||||
if err := os.Chdir(repoRoot(t)); err != nil {
|
||||
t.Fatalf("Chdir() error = %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = os.Chdir(wd)
|
||||
}()
|
||||
|
||||
srv, err := httpapi.NewServer(cfg, store, logger)
|
||||
if err != nil {
|
||||
t.Fatalf("httpapi.NewServer() error = %v", err)
|
||||
}
|
||||
httpServer := &http.Server{
|
||||
Addr: cfg.Server.ListenAddr,
|
||||
Handler: srv.Routes(),
|
||||
}
|
||||
ln, err := net.Listen("tcp", cfg.Server.ListenAddr)
|
||||
if err != nil {
|
||||
t.Fatalf("net.Listen() error = %v", err)
|
||||
}
|
||||
defer ln.Close()
|
||||
go func() {
|
||||
_ = httpServer.Serve(ln)
|
||||
}()
|
||||
t.Cleanup(func() {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||
defer cancel()
|
||||
_ = httpServer.Shutdown(ctx)
|
||||
})
|
||||
|
||||
baseURL := "http://" + ln.Addr().String()
|
||||
client := &http.Client{Timeout: 3 * time.Second}
|
||||
|
||||
createReq, err := http.NewRequest(http.MethodPost, baseURL+"/api/scratch", bytes.NewBufferString("hello from live server"))
|
||||
if err != nil {
|
||||
t.Fatalf("http.NewRequest(create) error = %v", err)
|
||||
}
|
||||
createReq.Header.Set("Content-Type", "text/plain; charset=utf-8")
|
||||
createResp, err := client.Do(createReq)
|
||||
if err != nil {
|
||||
t.Fatalf("client.Do(create) error = %v", err)
|
||||
}
|
||||
defer createResp.Body.Close()
|
||||
if createResp.StatusCode != http.StatusCreated {
|
||||
b, _ := io.ReadAll(createResp.Body)
|
||||
t.Fatalf("create status = %d, want %d, body=%q", createResp.StatusCode, http.StatusCreated, string(b))
|
||||
}
|
||||
|
||||
var payload map[string]any
|
||||
if err := json.NewDecoder(createResp.Body).Decode(&payload); err != nil {
|
||||
t.Fatalf("decode create payload error = %v", err)
|
||||
}
|
||||
id, _ := payload["id"].(string)
|
||||
if id == "" {
|
||||
t.Fatalf("create payload missing id")
|
||||
}
|
||||
|
||||
rawResp, err := client.Get(baseURL + "/raw/" + id)
|
||||
if err != nil {
|
||||
t.Fatalf("client.Get(raw) error = %v", err)
|
||||
}
|
||||
defer rawResp.Body.Close()
|
||||
if rawResp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("raw status = %d, want %d", rawResp.StatusCode, http.StatusOK)
|
||||
}
|
||||
rawBody, err := io.ReadAll(rawResp.Body)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadAll(raw) error = %v", err)
|
||||
}
|
||||
if got := string(rawBody); got != "hello from live server" {
|
||||
t.Fatalf("raw body = %q, want %q", got, "hello from live server")
|
||||
}
|
||||
|
||||
viewResp, err := client.Get(baseURL + "/s/" + id)
|
||||
if err != nil {
|
||||
t.Fatalf("client.Get(view) error = %v", err)
|
||||
}
|
||||
defer viewResp.Body.Close()
|
||||
if viewResp.StatusCode != http.StatusOK {
|
||||
t.Fatalf("view status = %d, want %d", viewResp.StatusCode, http.StatusOK)
|
||||
}
|
||||
viewBody, err := io.ReadAll(viewResp.Body)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadAll(view) error = %v", err)
|
||||
}
|
||||
if !strings.Contains(string(viewBody), "Scratch "+id) {
|
||||
t.Fatalf("view body missing scratch id")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,24 @@
|
||||
server:
|
||||
# Bind locally by default; change as needed for LAN/public access.
|
||||
listen_addr: "127.0.0.1:8080"
|
||||
|
||||
limits:
|
||||
# Conservative default limits suitable for small scratch usage.
|
||||
max_upload_size: "5MB"
|
||||
default_ttl: "30m"
|
||||
|
||||
storage:
|
||||
# Directory used for scratch files + metadata index.
|
||||
data_dir: "./data"
|
||||
cleanup_interval: "30s"
|
||||
|
||||
security:
|
||||
# Empty means all source IPs can write.
|
||||
# Add explicit IPs/CIDRs to lock down POST /api/scratch.
|
||||
allowed_ips: []
|
||||
trust_proxy_headers: false
|
||||
|
||||
rate_limit:
|
||||
enabled: true
|
||||
requests_per_minute: 20
|
||||
burst: 5
|
||||
@@ -0,0 +1,8 @@
|
||||
module scratchbox
|
||||
|
||||
go 1.25.4
|
||||
|
||||
require (
|
||||
golang.org/x/time v0.15.0 // indirect
|
||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||
)
|
||||
@@ -0,0 +1,5 @@
|
||||
golang.org/x/time v0.15.0 h1:bbrp8t3bGUeFOx08pvsMYRTCVSMk89u4tKbNOZbp88U=
|
||||
golang.org/x/time v0.15.0/go.mod h1:Y4YMaQmXwGQZoFaVFk4YpCt4FLQMYKZe9oeV/f4MSno=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
@@ -0,0 +1,44 @@
|
||||
package cleanup
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"scratchbox/internal/storage"
|
||||
)
|
||||
|
||||
type Worker struct {
|
||||
store *storage.FilesystemStore
|
||||
interval time.Duration
|
||||
logger *log.Logger
|
||||
}
|
||||
|
||||
func NewWorker(store *storage.FilesystemStore, interval time.Duration, logger *log.Logger) *Worker {
|
||||
return &Worker{
|
||||
store: store,
|
||||
interval: interval,
|
||||
logger: logger,
|
||||
}
|
||||
}
|
||||
|
||||
func (w *Worker) Start(ctx context.Context) {
|
||||
ticker := time.NewTicker(w.interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return
|
||||
case now := <-ticker.C:
|
||||
deleted, err := w.store.DeleteExpired(now)
|
||||
if err != nil {
|
||||
w.logger.Printf("cleanup worker failed: %v", err)
|
||||
continue
|
||||
}
|
||||
if deleted > 0 {
|
||||
w.logger.Printf("cleanup worker removed %d expired scratches", deleted)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,74 @@
|
||||
package cleanup
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"log"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"scratchbox/internal/storage"
|
||||
)
|
||||
|
||||
func TestNewWorker(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := storage.NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
logger := log.New(&bytes.Buffer{}, "", 0)
|
||||
w := NewWorker(store, 20*time.Millisecond, logger)
|
||||
if w == nil {
|
||||
t.Fatalf("NewWorker() = nil")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerStartRemovesExpiredAndLogs(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := storage.NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
if _, err := store.Create(context.Background(), strings.NewReader("expired"), "text/plain", -time.Second); err != nil {
|
||||
t.Fatalf("Create() error = %v", err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
logger := log.New(&buf, "", 0)
|
||||
w := NewWorker(store, 10*time.Millisecond, logger)
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 80*time.Millisecond)
|
||||
defer cancel()
|
||||
w.Start(ctx)
|
||||
|
||||
if !strings.Contains(buf.String(), "cleanup worker removed") {
|
||||
t.Fatalf("expected cleanup removal log, got %q", buf.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkerStartWithNoExpiredEntries(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := storage.NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
if _, err := store.Create(context.Background(), strings.NewReader("active"), "text/plain", time.Hour); err != nil {
|
||||
t.Fatalf("Create() error = %v", err)
|
||||
}
|
||||
|
||||
var buf bytes.Buffer
|
||||
logger := log.New(&buf, "", 0)
|
||||
w := NewWorker(store, 10*time.Millisecond, logger)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 35*time.Millisecond)
|
||||
defer cancel()
|
||||
w.Start(ctx)
|
||||
|
||||
if strings.Contains(buf.String(), "removed") {
|
||||
t.Fatalf("expected no removal log, got %q", buf.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,251 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/netip"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
const (
|
||||
defaultListenAddr = ":8080"
|
||||
defaultMaxUploadSize = "100MB"
|
||||
defaultTTL = "1h"
|
||||
defaultDataDir = "./data"
|
||||
defaultCleanupInterval = "1m"
|
||||
defaultRateLimitEnabled = true
|
||||
defaultRequestsPerMinute = 30
|
||||
defaultRateLimitBurst = 10
|
||||
maxDefaultTTLLimit = 24 * time.Hour
|
||||
minCleanupInterval = 10 * time.Second
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Server ServerConfig `yaml:"server"`
|
||||
Limits LimitsConfig `yaml:"limits"`
|
||||
Storage StorageConfig `yaml:"storage"`
|
||||
Security SecurityConfig `yaml:"security"`
|
||||
}
|
||||
|
||||
type ServerConfig struct {
|
||||
ListenAddr string `yaml:"listen_addr"`
|
||||
}
|
||||
|
||||
type LimitsConfig struct {
|
||||
MaxUploadSize string `yaml:"max_upload_size"`
|
||||
DefaultTTL string `yaml:"default_ttl"`
|
||||
MaxUploadSizeBytes int64 `yaml:"-"`
|
||||
DefaultTTLDuration time.Duration `yaml:"-"`
|
||||
}
|
||||
|
||||
type StorageConfig struct {
|
||||
DataDir string `yaml:"data_dir"`
|
||||
CleanupInterval string `yaml:"cleanup_interval"`
|
||||
CleanupIntervalParsed time.Duration `yaml:"-"`
|
||||
}
|
||||
|
||||
type SecurityConfig struct {
|
||||
AllowedIPs []string `yaml:"allowed_ips"`
|
||||
TrustProxyHeaders bool `yaml:"trust_proxy_headers"`
|
||||
RateLimit RateLimitConfig `yaml:"rate_limit"`
|
||||
AllowedPrefixes []netip.Prefix `yaml:"-"`
|
||||
}
|
||||
|
||||
type RateLimitConfig struct {
|
||||
Enabled bool `yaml:"enabled"`
|
||||
RequestsPerMinute int `yaml:"requests_per_minute"`
|
||||
Burst int `yaml:"burst"`
|
||||
}
|
||||
|
||||
func Load(path string) (Config, error) {
|
||||
cfg := defaults()
|
||||
if path != "" {
|
||||
b, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return Config{}, fmt.Errorf("read config: %w", err)
|
||||
}
|
||||
if err := yaml.Unmarshal(b, &cfg); err != nil {
|
||||
return Config{}, fmt.Errorf("parse config yaml: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
if err := cfg.Validate(); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
func defaults() Config {
|
||||
return Config{
|
||||
Server: ServerConfig{
|
||||
ListenAddr: defaultListenAddr,
|
||||
},
|
||||
Limits: LimitsConfig{
|
||||
MaxUploadSize: defaultMaxUploadSize,
|
||||
DefaultTTL: defaultTTL,
|
||||
},
|
||||
Storage: StorageConfig{
|
||||
DataDir: defaultDataDir,
|
||||
CleanupInterval: defaultCleanupInterval,
|
||||
},
|
||||
Security: SecurityConfig{
|
||||
RateLimit: RateLimitConfig{
|
||||
Enabled: defaultRateLimitEnabled,
|
||||
RequestsPerMinute: defaultRequestsPerMinute,
|
||||
Burst: defaultRateLimitBurst,
|
||||
},
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Config) Validate() error {
|
||||
if strings.TrimSpace(c.Server.ListenAddr) == "" {
|
||||
return errors.New("server.listen_addr is required")
|
||||
}
|
||||
|
||||
sizeBytes, err := parseHumanSize(c.Limits.MaxUploadSize)
|
||||
if err != nil {
|
||||
return fmt.Errorf("limits.max_upload_size invalid: %w", err)
|
||||
}
|
||||
c.Limits.MaxUploadSizeBytes = sizeBytes
|
||||
|
||||
ttl, err := time.ParseDuration(strings.TrimSpace(c.Limits.DefaultTTL))
|
||||
if err != nil {
|
||||
return fmt.Errorf("limits.default_ttl invalid: %w", err)
|
||||
}
|
||||
if ttl <= 0 || ttl > maxDefaultTTLLimit {
|
||||
return fmt.Errorf("limits.default_ttl must be in (0, %s]", maxDefaultTTLLimit)
|
||||
}
|
||||
c.Limits.DefaultTTLDuration = ttl
|
||||
|
||||
cleanupInterval, err := time.ParseDuration(strings.TrimSpace(c.Storage.CleanupInterval))
|
||||
if err != nil {
|
||||
return fmt.Errorf("storage.cleanup_interval invalid: %w", err)
|
||||
}
|
||||
if cleanupInterval < minCleanupInterval {
|
||||
return fmt.Errorf("storage.cleanup_interval must be at least %s", minCleanupInterval)
|
||||
}
|
||||
c.Storage.CleanupIntervalParsed = cleanupInterval
|
||||
|
||||
if strings.TrimSpace(c.Storage.DataDir) == "" {
|
||||
return errors.New("storage.data_dir is required")
|
||||
}
|
||||
if err := ensureWritableDir(c.Storage.DataDir); err != nil {
|
||||
return fmt.Errorf("storage.data_dir not writable: %w", err)
|
||||
}
|
||||
|
||||
prefixes, err := parseAllowedPrefixes(c.Security.AllowedIPs)
|
||||
if err != nil {
|
||||
return fmt.Errorf("security.allowed_ips invalid: %w", err)
|
||||
}
|
||||
c.Security.AllowedPrefixes = prefixes
|
||||
|
||||
if c.Security.RateLimit.RequestsPerMinute <= 0 {
|
||||
return errors.New("security.rate_limit.requests_per_minute must be > 0")
|
||||
}
|
||||
if c.Security.RateLimit.Burst <= 0 {
|
||||
return errors.New("security.rate_limit.burst must be > 0")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func ensureWritableDir(dir string) error {
|
||||
if err := os.MkdirAll(dir, 0o755); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
probe := filepath.Join(dir, ".write_probe")
|
||||
if err := os.WriteFile(probe, []byte("ok"), 0o644); err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Remove(probe)
|
||||
}
|
||||
|
||||
func parseAllowedPrefixes(values []string) ([]netip.Prefix, error) {
|
||||
if len(values) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
out := make([]netip.Prefix, 0, len(values))
|
||||
for _, value := range values {
|
||||
item := strings.TrimSpace(value)
|
||||
if item == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
if strings.Contains(item, "/") {
|
||||
prefix, err := netip.ParsePrefix(item)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse cidr %q: %w", item, err)
|
||||
}
|
||||
out = append(out, prefix)
|
||||
continue
|
||||
}
|
||||
|
||||
ip, err := netip.ParseAddr(item)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse ip %q: %w", item, err)
|
||||
}
|
||||
out = append(out, netip.PrefixFrom(ip, ip.BitLen()))
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func parseHumanSize(raw string) (int64, error) {
|
||||
value := strings.TrimSpace(strings.ToUpper(raw))
|
||||
if value == "" {
|
||||
return 0, errors.New("size is empty")
|
||||
}
|
||||
|
||||
idx := 0
|
||||
for idx < len(value) && (value[idx] == '.' || (value[idx] >= '0' && value[idx] <= '9')) {
|
||||
idx++
|
||||
}
|
||||
if idx == 0 {
|
||||
return 0, fmt.Errorf("size %q does not start with a number", raw)
|
||||
}
|
||||
|
||||
numPart := value[:idx]
|
||||
unitPart := strings.TrimSpace(value[idx:])
|
||||
if unitPart == "" {
|
||||
unitPart = "B"
|
||||
}
|
||||
|
||||
number, err := strconv.ParseFloat(numPart, 64)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("parse number %q: %w", numPart, err)
|
||||
}
|
||||
if number <= 0 {
|
||||
return 0, errors.New("size must be > 0")
|
||||
}
|
||||
|
||||
multipliers := map[string]float64{
|
||||
"B": 1,
|
||||
"KB": 1_000,
|
||||
"MB": 1_000_000,
|
||||
"GB": 1_000_000_000,
|
||||
"TB": 1_000_000_000_000,
|
||||
"KIB": 1024,
|
||||
"MIB": 1024 * 1024,
|
||||
"GIB": 1024 * 1024 * 1024,
|
||||
"TIB": 1024 * 1024 * 1024 * 1024,
|
||||
}
|
||||
|
||||
multiplier, ok := multipliers[unitPart]
|
||||
if !ok {
|
||||
return 0, fmt.Errorf("unsupported unit %q", unitPart)
|
||||
}
|
||||
|
||||
size := int64(number * multiplier)
|
||||
if size <= 0 {
|
||||
return 0, errors.New("calculated size must be > 0")
|
||||
}
|
||||
return size, nil
|
||||
}
|
||||
@@ -0,0 +1,264 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"net/netip"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestValidateAppliesDefaultsAndParsedFields(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := defaults()
|
||||
cfg.Storage.DataDir = t.TempDir()
|
||||
|
||||
if err := cfg.Validate(); err != nil {
|
||||
t.Fatalf("Validate() returned error: %v", err)
|
||||
}
|
||||
|
||||
if cfg.Limits.MaxUploadSizeBytes != 100_000_000 {
|
||||
t.Fatalf("MaxUploadSizeBytes = %d, want %d", cfg.Limits.MaxUploadSizeBytes, int64(100_000_000))
|
||||
}
|
||||
if cfg.Limits.DefaultTTLDuration != time.Hour {
|
||||
t.Fatalf("DefaultTTLDuration = %s, want 1h", cfg.Limits.DefaultTTLDuration)
|
||||
}
|
||||
if cfg.Storage.CleanupIntervalParsed != time.Minute {
|
||||
t.Fatalf("CleanupIntervalParsed = %s, want 1m", cfg.Storage.CleanupIntervalParsed)
|
||||
}
|
||||
if len(cfg.Security.AllowedPrefixes) != 0 {
|
||||
t.Fatalf("AllowedPrefixes len = %d, want 0", len(cfg.Security.AllowedPrefixes))
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadParsesConfiguredFields(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tmp := t.TempDir()
|
||||
dataDir := filepath.Join(tmp, "store")
|
||||
configPath := filepath.Join(tmp, "config.yaml")
|
||||
content := `
|
||||
server:
|
||||
listen_addr: "127.0.0.1:9090"
|
||||
limits:
|
||||
max_upload_size: "2MiB"
|
||||
default_ttl: "2h"
|
||||
storage:
|
||||
data_dir: "` + dataDir + `"
|
||||
cleanup_interval: "45s"
|
||||
security:
|
||||
allowed_ips:
|
||||
- "127.0.0.1"
|
||||
- "10.0.0.0/8"
|
||||
trust_proxy_headers: true
|
||||
rate_limit:
|
||||
enabled: true
|
||||
requests_per_minute: 12
|
||||
burst: 4
|
||||
`
|
||||
if err := os.WriteFile(configPath, []byte(content), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
|
||||
cfg, err := Load(configPath)
|
||||
if err != nil {
|
||||
t.Fatalf("Load() error = %v", err)
|
||||
}
|
||||
|
||||
if cfg.Server.ListenAddr != "127.0.0.1:9090" {
|
||||
t.Fatalf("ListenAddr = %q, want %q", cfg.Server.ListenAddr, "127.0.0.1:9090")
|
||||
}
|
||||
if cfg.Limits.MaxUploadSizeBytes != 2*1024*1024 {
|
||||
t.Fatalf("MaxUploadSizeBytes = %d, want %d", cfg.Limits.MaxUploadSizeBytes, int64(2*1024*1024))
|
||||
}
|
||||
if cfg.Limits.DefaultTTLDuration != 2*time.Hour {
|
||||
t.Fatalf("DefaultTTLDuration = %s, want 2h", cfg.Limits.DefaultTTLDuration)
|
||||
}
|
||||
if cfg.Storage.CleanupIntervalParsed != 45*time.Second {
|
||||
t.Fatalf("CleanupIntervalParsed = %s, want 45s", cfg.Storage.CleanupIntervalParsed)
|
||||
}
|
||||
if !cfg.Security.TrustProxyHeaders {
|
||||
t.Fatalf("TrustProxyHeaders = false, want true")
|
||||
}
|
||||
if cfg.Security.RateLimit.RequestsPerMinute != 12 || cfg.Security.RateLimit.Burst != 4 {
|
||||
t.Fatalf("unexpected rate_limit values: %+v", cfg.Security.RateLimit)
|
||||
}
|
||||
if len(cfg.Security.AllowedPrefixes) != 2 {
|
||||
t.Fatalf("AllowedPrefixes len = %d, want 2", len(cfg.Security.AllowedPrefixes))
|
||||
}
|
||||
}
|
||||
|
||||
func TestValidateRejectsInvalidValues(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tests := []struct {
|
||||
name string
|
||||
mutate func(*Config)
|
||||
wantErr string
|
||||
}{
|
||||
{
|
||||
name: "listen addr required",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Server.ListenAddr = " "
|
||||
},
|
||||
wantErr: "server.listen_addr",
|
||||
},
|
||||
{
|
||||
name: "invalid max upload size",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Limits.MaxUploadSize = "oops"
|
||||
},
|
||||
wantErr: "limits.max_upload_size",
|
||||
},
|
||||
{
|
||||
name: "invalid ttl parse",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Limits.DefaultTTL = "not-a-duration"
|
||||
},
|
||||
wantErr: "limits.default_ttl",
|
||||
},
|
||||
{
|
||||
name: "ttl above max",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Limits.DefaultTTL = "25h"
|
||||
},
|
||||
wantErr: "limits.default_ttl",
|
||||
},
|
||||
{
|
||||
name: "invalid allowlist entry",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Security.AllowedIPs = []string{"not-an-ip"}
|
||||
},
|
||||
wantErr: "security.allowed_ips",
|
||||
},
|
||||
{
|
||||
name: "invalid rate limit burst",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Security.RateLimit.Burst = 0
|
||||
},
|
||||
wantErr: "security.rate_limit.burst",
|
||||
},
|
||||
{
|
||||
name: "data dir required",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Storage.DataDir = " "
|
||||
},
|
||||
wantErr: "storage.data_dir",
|
||||
},
|
||||
{
|
||||
name: "cleanup parse failure",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Storage.CleanupInterval = "not-a-duration"
|
||||
},
|
||||
wantErr: "storage.cleanup_interval",
|
||||
},
|
||||
{
|
||||
name: "cleanup too small",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Storage.CleanupInterval = "5s"
|
||||
},
|
||||
wantErr: "storage.cleanup_interval",
|
||||
},
|
||||
{
|
||||
name: "invalid rate limit rpm",
|
||||
mutate: func(cfg *Config) {
|
||||
cfg.Security.RateLimit.RequestsPerMinute = 0
|
||||
},
|
||||
wantErr: "security.rate_limit.requests_per_minute",
|
||||
},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
tc := tc
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
cfg := defaults()
|
||||
cfg.Storage.DataDir = t.TempDir()
|
||||
tc.mutate(&cfg)
|
||||
|
||||
err := cfg.Validate()
|
||||
if err == nil {
|
||||
t.Fatalf("Validate() error = nil, want non-nil")
|
||||
}
|
||||
if !strings.Contains(err.Error(), tc.wantErr) {
|
||||
t.Fatalf("Validate() error = %q, want substring %q", err.Error(), tc.wantErr)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadErrors(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, err := Load(filepath.Join(t.TempDir(), "missing.yaml"))
|
||||
if err == nil || !strings.Contains(err.Error(), "read config") {
|
||||
t.Fatalf("Load() error = %v, want read config error", err)
|
||||
}
|
||||
|
||||
tmp := t.TempDir()
|
||||
configPath := filepath.Join(tmp, "bad.yaml")
|
||||
if err := os.WriteFile(configPath, []byte("limits: [bad"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
_, err = Load(configPath)
|
||||
if err == nil || !strings.Contains(err.Error(), "parse config yaml") {
|
||||
t.Fatalf("Load() error = %v, want yaml parse error", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseAllowedPrefixesTrimsAndParses(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
prefixes, err := parseAllowedPrefixes([]string{" 192.0.2.4 ", " ", "2001:db8::/32"})
|
||||
if err != nil {
|
||||
t.Fatalf("parseAllowedPrefixes() error = %v", err)
|
||||
}
|
||||
if len(prefixes) != 2 {
|
||||
t.Fatalf("prefixes len = %d, want 2", len(prefixes))
|
||||
}
|
||||
wantIP := netip.MustParsePrefix("192.0.2.4/32")
|
||||
if prefixes[0] != wantIP {
|
||||
t.Fatalf("prefixes[0] = %v, want %v", prefixes[0], wantIP)
|
||||
}
|
||||
}
|
||||
|
||||
func TestParseHumanSizeVariants(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tests := []struct {
|
||||
in string
|
||||
want int64
|
||||
wantErr string
|
||||
}{
|
||||
{in: "10", want: 10},
|
||||
{in: "1.5KB", want: 1500},
|
||||
{in: "2MiB", want: 2 * 1024 * 1024},
|
||||
{in: "0", wantErr: "size must be > 0"},
|
||||
{in: "abc", wantErr: "does not start with a number"},
|
||||
{in: "5XB", wantErr: "unsupported unit"},
|
||||
{in: "", wantErr: "size is empty"},
|
||||
}
|
||||
|
||||
for _, tc := range tests {
|
||||
tc := tc
|
||||
t.Run(tc.in, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
got, err := parseHumanSize(tc.in)
|
||||
if tc.wantErr != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), tc.wantErr) {
|
||||
t.Fatalf("parseHumanSize(%q) err=%v, want contains %q", tc.in, err, tc.wantErr)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatalf("parseHumanSize(%q) error = %v", tc.in, err)
|
||||
}
|
||||
if got != tc.want {
|
||||
t.Fatalf("parseHumanSize(%q) = %d, want %d", tc.in, got, tc.want)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,294 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"html/template"
|
||||
"io"
|
||||
"log"
|
||||
"mime"
|
||||
"net/http"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"scratchbox/internal/config"
|
||||
"scratchbox/internal/storage"
|
||||
)
|
||||
|
||||
const maxViewPreviewBytes int64 = 2 * 1024 * 1024
|
||||
|
||||
type Server struct {
|
||||
cfg config.Config
|
||||
store *storage.FilesystemStore
|
||||
logger *log.Logger
|
||||
templates *template.Template
|
||||
}
|
||||
|
||||
type templateViewData struct {
|
||||
ID string
|
||||
Content string
|
||||
ContentType string
|
||||
RawURL string
|
||||
ExpiresAt string
|
||||
Truncated bool
|
||||
Binary bool
|
||||
}
|
||||
|
||||
func NewServer(cfg config.Config, store *storage.FilesystemStore, logger *log.Logger) (*Server, error) {
|
||||
tmpl, err := template.ParseFiles(
|
||||
filepath.Join("web", "templates", "index.html"),
|
||||
filepath.Join("web", "templates", "view.html"),
|
||||
)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse templates: %w", err)
|
||||
}
|
||||
|
||||
return &Server{
|
||||
cfg: cfg,
|
||||
store: store,
|
||||
logger: logger,
|
||||
templates: tmpl,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func (s *Server) Routes() http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.Handle("GET /", http.HandlerFunc(s.index))
|
||||
mux.Handle("GET /api/scratch/{id}", http.HandlerFunc(s.getScratch))
|
||||
mux.Handle("GET /s/{id}", http.HandlerFunc(s.viewScratch))
|
||||
mux.Handle("GET /raw/{id}", http.HandlerFunc(s.rawScratch))
|
||||
|
||||
writeHandler := http.HandlerFunc(s.createScratch)
|
||||
middlewares := make([]func(http.Handler) http.Handler, 0, 2)
|
||||
middlewares = append(middlewares, AllowlistMiddleware(s.cfg.Security.AllowedPrefixes, s.cfg.Security.TrustProxyHeaders, s.logger))
|
||||
if s.cfg.Security.RateLimit.Enabled {
|
||||
limiter := NewRateLimiter(s.cfg.Security.RateLimit.RequestsPerMinute, s.cfg.Security.RateLimit.Burst)
|
||||
middlewares = append(middlewares, RateLimitMiddleware(limiter, s.cfg.Security.TrustProxyHeaders))
|
||||
}
|
||||
mux.Handle("POST /api/scratch", Chain(writeHandler, middlewares...))
|
||||
|
||||
mux.Handle("GET /static/", http.StripPrefix("/static/", http.FileServer(http.Dir(filepath.Join("web", "static")))))
|
||||
return mux
|
||||
}
|
||||
|
||||
func (s *Server) index(w http.ResponseWriter, r *http.Request) {
|
||||
if err := s.templates.ExecuteTemplate(w, "index.html", nil); err != nil {
|
||||
http.Error(w, "failed to render page", http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) createScratch(w http.ResponseWriter, r *http.Request) {
|
||||
r.Body = http.MaxBytesReader(w, r.Body, s.cfg.Limits.MaxUploadSizeBytes)
|
||||
|
||||
var reader io.Reader
|
||||
contentType := strings.TrimSpace(r.Header.Get("Content-Type"))
|
||||
|
||||
if strings.HasPrefix(contentType, "multipart/form-data") {
|
||||
if err := r.ParseMultipartForm(s.cfg.Limits.MaxUploadSizeBytes); err != nil {
|
||||
s.writeCreateError(w, err)
|
||||
return
|
||||
}
|
||||
if fileCount := multipartFileCount(r); fileCount > 1 {
|
||||
http.Error(w, "only one file is allowed per scratch upload", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
file, header, err := r.FormFile("file")
|
||||
if err == nil {
|
||||
defer file.Close()
|
||||
if strings.TrimSpace(r.FormValue("content")) != "" {
|
||||
http.Error(w, "multipart request must include either file or content, not both", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
reader = file
|
||||
contentType = header.Header.Get("Content-Type")
|
||||
if contentType == "" {
|
||||
contentType = "application/octet-stream"
|
||||
}
|
||||
} else {
|
||||
body := strings.TrimSpace(r.FormValue("content"))
|
||||
if body == "" {
|
||||
http.Error(w, "multipart request requires file or content field", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
reader = strings.NewReader(body)
|
||||
contentType = "text/plain; charset=utf-8"
|
||||
}
|
||||
} else {
|
||||
reader = r.Body
|
||||
if contentType == "" {
|
||||
contentType = "text/plain; charset=utf-8"
|
||||
}
|
||||
}
|
||||
|
||||
meta, err := s.store.Create(r.Context(), reader, contentType, s.cfg.Limits.DefaultTTLDuration)
|
||||
if err != nil {
|
||||
if strings.Contains(err.Error(), "request body too large") {
|
||||
http.Error(w, "upload exceeds limits.max_upload_size", http.StatusRequestEntityTooLarge)
|
||||
return
|
||||
}
|
||||
http.Error(w, "failed to save scratch", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
payload := map[string]any{
|
||||
"id": meta.ID,
|
||||
"created_at": meta.CreatedAt,
|
||||
"expires_at": meta.ExpiresAt,
|
||||
"size": meta.Size,
|
||||
"content_type": meta.ContentType,
|
||||
"view_url": fmt.Sprintf("/s/%s", meta.ID),
|
||||
"raw_url": fmt.Sprintf("/raw/%s", meta.ID),
|
||||
"api_url": fmt.Sprintf("/api/scratch/%s", meta.ID),
|
||||
}
|
||||
writeJSON(w, http.StatusCreated, payload)
|
||||
}
|
||||
|
||||
func multipartFileCount(r *http.Request) int {
|
||||
if r.MultipartForm == nil || len(r.MultipartForm.File) == 0 {
|
||||
return 0
|
||||
}
|
||||
count := 0
|
||||
for _, headers := range r.MultipartForm.File {
|
||||
count += len(headers)
|
||||
}
|
||||
return count
|
||||
}
|
||||
|
||||
func (s *Server) getScratch(w http.ResponseWriter, r *http.Request) {
|
||||
id := strings.TrimSpace(r.PathValue("id"))
|
||||
if id == "" {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
meta, err := s.store.Get(id)
|
||||
if err != nil {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
if s.isExpired(meta) {
|
||||
_ = s.store.Delete(id)
|
||||
http.Error(w, "scratch expired", http.StatusGone)
|
||||
return
|
||||
}
|
||||
|
||||
payload := map[string]any{
|
||||
"id": meta.ID,
|
||||
"created_at": meta.CreatedAt,
|
||||
"expires_at": meta.ExpiresAt,
|
||||
"size": meta.Size,
|
||||
"content_type": meta.ContentType,
|
||||
"view_url": fmt.Sprintf("/s/%s", meta.ID),
|
||||
"raw_url": fmt.Sprintf("/raw/%s", meta.ID),
|
||||
}
|
||||
writeJSON(w, http.StatusOK, payload)
|
||||
}
|
||||
|
||||
func (s *Server) viewScratch(w http.ResponseWriter, r *http.Request) {
|
||||
id := strings.TrimSpace(r.PathValue("id"))
|
||||
file, meta, err := s.store.Open(id)
|
||||
if err != nil {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
if s.isExpired(meta) {
|
||||
_ = s.store.Delete(id)
|
||||
http.Error(w, "scratch expired", http.StatusGone)
|
||||
return
|
||||
}
|
||||
|
||||
limited := io.LimitReader(file, maxViewPreviewBytes+1)
|
||||
b, err := io.ReadAll(limited)
|
||||
if err != nil {
|
||||
http.Error(w, "failed to read scratch", http.StatusInternalServerError)
|
||||
return
|
||||
}
|
||||
|
||||
truncated := int64(len(b)) > maxViewPreviewBytes
|
||||
if truncated {
|
||||
b = b[:maxViewPreviewBytes]
|
||||
}
|
||||
isBinary := !isLikelyText(meta.ContentType, b)
|
||||
|
||||
data := templateViewData{
|
||||
ID: meta.ID,
|
||||
ContentType: meta.ContentType,
|
||||
RawURL: fmt.Sprintf("/raw/%s", meta.ID),
|
||||
ExpiresAt: meta.ExpiresAt.Format("2006-01-02 15:04:05 MST"),
|
||||
Truncated: truncated,
|
||||
Binary: isBinary,
|
||||
}
|
||||
if !isBinary {
|
||||
data.Content = string(b)
|
||||
}
|
||||
|
||||
if err := s.templates.ExecuteTemplate(w, "view.html", data); err != nil {
|
||||
http.Error(w, "failed to render scratch view", http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) rawScratch(w http.ResponseWriter, r *http.Request) {
|
||||
id := strings.TrimSpace(r.PathValue("id"))
|
||||
file, meta, err := s.store.Open(id)
|
||||
if err != nil {
|
||||
http.NotFound(w, r)
|
||||
return
|
||||
}
|
||||
defer file.Close()
|
||||
|
||||
if s.isExpired(meta) {
|
||||
_ = s.store.Delete(id)
|
||||
http.Error(w, "scratch expired", http.StatusGone)
|
||||
return
|
||||
}
|
||||
|
||||
if meta.ContentType == "" {
|
||||
meta.ContentType = "application/octet-stream"
|
||||
}
|
||||
w.Header().Set("Content-Type", meta.ContentType)
|
||||
w.Header().Set("Content-Length", fmt.Sprintf("%d", meta.Size))
|
||||
if _, err := io.Copy(w, file); err != nil {
|
||||
s.logger.Printf("failed to stream raw scratch id=%s err=%v", id, err)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Server) writeCreateError(w http.ResponseWriter, err error) {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) || strings.Contains(err.Error(), "request body too large") {
|
||||
http.Error(w, "upload exceeds limits.max_upload_size", http.StatusRequestEntityTooLarge)
|
||||
return
|
||||
}
|
||||
http.Error(w, "invalid upload payload", http.StatusBadRequest)
|
||||
}
|
||||
|
||||
func (s *Server) isExpired(meta storage.Metadata) bool {
|
||||
return !meta.ExpiresAt.After(nowUTC())
|
||||
}
|
||||
|
||||
func nowUTC() time.Time {
|
||||
return time.Now().UTC()
|
||||
}
|
||||
|
||||
func isLikelyText(contentType string, content []byte) bool {
|
||||
if ct := strings.TrimSpace(contentType); ct != "" {
|
||||
mediaType, _, err := mime.ParseMediaType(ct)
|
||||
if err == nil {
|
||||
if strings.HasPrefix(mediaType, "text/") {
|
||||
return true
|
||||
}
|
||||
switch mediaType {
|
||||
case "application/json", "application/xml", "application/javascript", "application/x-www-form-urlencoded":
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
if len(content) == 0 {
|
||||
return true
|
||||
}
|
||||
return utf8.Valid(content)
|
||||
}
|
||||
@@ -0,0 +1,454 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"html/template"
|
||||
"io"
|
||||
"log"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"scratchbox/internal/config"
|
||||
"scratchbox/internal/storage"
|
||||
)
|
||||
|
||||
func TestCreateReadThenExpireScratch(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: 120 * time.Millisecond,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
createReq := httptest.NewRequest(http.MethodPost, "/api/scratch", bytes.NewBufferString("hello scratch"))
|
||||
createReq.Header.Set("Content-Type", "text/plain; charset=utf-8")
|
||||
createReq.RemoteAddr = "127.0.0.1:9999"
|
||||
createRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(createRec, createReq)
|
||||
if createRec.Code != http.StatusCreated {
|
||||
t.Fatalf("create status = %d, want %d", createRec.Code, http.StatusCreated)
|
||||
}
|
||||
|
||||
var payload map[string]any
|
||||
if err := json.Unmarshal(createRec.Body.Bytes(), &payload); err != nil {
|
||||
t.Fatalf("decode create payload error = %v", err)
|
||||
}
|
||||
id, _ := payload["id"].(string)
|
||||
if id == "" {
|
||||
t.Fatalf("create payload missing id")
|
||||
}
|
||||
|
||||
metaReq := httptest.NewRequest(http.MethodGet, "/api/scratch/"+id, nil)
|
||||
metaReq.RemoteAddr = "127.0.0.1:9999"
|
||||
metaRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(metaRec, metaReq)
|
||||
if metaRec.Code != http.StatusOK {
|
||||
t.Fatalf("read metadata status = %d, want %d", metaRec.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
rawReq := httptest.NewRequest(http.MethodGet, "/raw/"+id, nil)
|
||||
rawReq.RemoteAddr = "127.0.0.1:9999"
|
||||
rawRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rawRec, rawReq)
|
||||
if rawRec.Code != http.StatusOK {
|
||||
t.Fatalf("read raw status = %d, want %d", rawRec.Code, http.StatusOK)
|
||||
}
|
||||
if got := rawRec.Body.String(); got != "hello scratch" {
|
||||
t.Fatalf("raw body = %q, want %q", got, "hello scratch")
|
||||
}
|
||||
|
||||
time.Sleep(180 * time.Millisecond)
|
||||
|
||||
expiredReq := httptest.NewRequest(http.MethodGet, "/api/scratch/"+id, nil)
|
||||
expiredReq.RemoteAddr = "127.0.0.1:9999"
|
||||
expiredRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(expiredRec, expiredReq)
|
||||
if expiredRec.Code != http.StatusGone {
|
||||
t.Fatalf("expired status = %d, want %d", expiredRec.Code, http.StatusGone)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateScratchRejectsBodyOverLimit(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 8,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", bytes.NewBufferString("this is too large"))
|
||||
req.Header.Set("Content-Type", "text/plain; charset=utf-8")
|
||||
req.RemoteAddr = "127.0.0.1:5050"
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusRequestEntityTooLarge {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusRequestEntityTooLarge)
|
||||
}
|
||||
if !bytes.Contains(rec.Body.Bytes(), []byte("upload exceeds limits.max_upload_size")) {
|
||||
t.Fatalf("unexpected body: %q", rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateScratchRejectsMultipleMultipartFiles(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
file1, err := writer.CreateFormFile("file", "first.txt")
|
||||
if err != nil {
|
||||
t.Fatalf("CreateFormFile first error = %v", err)
|
||||
}
|
||||
if _, err := file1.Write([]byte("first")); err != nil {
|
||||
t.Fatalf("write first file error = %v", err)
|
||||
}
|
||||
file2, err := writer.CreateFormFile("file", "second.txt")
|
||||
if err != nil {
|
||||
t.Fatalf("CreateFormFile second error = %v", err)
|
||||
}
|
||||
if _, err := file2.Write([]byte("second")); err != nil {
|
||||
t.Fatalf("write second file error = %v", err)
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatalf("multipart close error = %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", &body)
|
||||
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
req.RemoteAddr = "127.0.0.1:5051"
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
|
||||
}
|
||||
if !bytes.Contains(rec.Body.Bytes(), []byte("only one file is allowed per scratch upload")) {
|
||||
t.Fatalf("unexpected body: %q", rec.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateScratchMultipartContentOnly(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
if err := writer.WriteField("content", "hello from content field"); err != nil {
|
||||
t.Fatalf("WriteField() error = %v", err)
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatalf("multipart close error = %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", &body)
|
||||
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
req.RemoteAddr = "127.0.0.1:5052"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusCreated {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusCreated)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateScratchMultipartRejectsFileAndContent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
file, err := writer.CreateFormFile("file", "one.txt")
|
||||
if err != nil {
|
||||
t.Fatalf("CreateFormFile() error = %v", err)
|
||||
}
|
||||
if _, err := file.Write([]byte("file content")); err != nil {
|
||||
t.Fatalf("write file error = %v", err)
|
||||
}
|
||||
if err := writer.WriteField("content", "extra content"); err != nil {
|
||||
t.Fatalf("WriteField() error = %v", err)
|
||||
}
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatalf("multipart close error = %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", &body)
|
||||
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
req.RemoteAddr = "127.0.0.1:5053"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateScratchMultipartRequiresFileOrContent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
var body bytes.Buffer
|
||||
writer := multipart.NewWriter(&body)
|
||||
if err := writer.Close(); err != nil {
|
||||
t.Fatalf("multipart close error = %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", &body)
|
||||
req.Header.Set("Content-Type", writer.FormDataContentType())
|
||||
req.RemoteAddr = "127.0.0.1:5054"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetScratchNotFound(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/api/scratch/does-not-exist", nil)
|
||||
req.RemoteAddr = "127.0.0.1:5055"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusNotFound {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusNotFound)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRawScratchDefaultsContentType(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler, store := newTestHandlerAndStore(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
meta, err := store.Create(t.Context(), strings.NewReader("no content type"), "", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("store.Create() error = %v", err)
|
||||
}
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/raw/"+meta.ID, nil)
|
||||
req.RemoteAddr = "127.0.0.1:5056"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusOK {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusOK)
|
||||
}
|
||||
if got := rec.Header().Get("Content-Type"); got != "application/octet-stream" {
|
||||
t.Fatalf("Content-Type = %q, want %q", got, "application/octet-stream")
|
||||
}
|
||||
}
|
||||
|
||||
func TestViewScratchAndIndexRender(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler, store := newTemplateHandlerAndStore(t, testServerOptions{
|
||||
maxUploadBytes: 10 * 1024 * 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
meta, err := store.Create(t.Context(), strings.NewReader("hello view"), "text/plain; charset=utf-8", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("store.Create() error = %v", err)
|
||||
}
|
||||
|
||||
indexReq := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
indexRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(indexRec, indexReq)
|
||||
if indexRec.Code != http.StatusOK {
|
||||
t.Fatalf("index status = %d, want %d", indexRec.Code, http.StatusOK)
|
||||
}
|
||||
|
||||
viewReq := httptest.NewRequest(http.MethodGet, "/s/"+meta.ID, nil)
|
||||
viewRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(viewRec, viewReq)
|
||||
if viewRec.Code != http.StatusOK {
|
||||
t.Fatalf("view status = %d, want %d", viewRec.Code, http.StatusOK)
|
||||
}
|
||||
if !strings.Contains(viewRec.Body.String(), "hello view") {
|
||||
t.Fatalf("expected view body to include scratch content")
|
||||
}
|
||||
}
|
||||
|
||||
func TestViewAndRawScratchExpired(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler, store := newTemplateHandlerAndStore(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
meta, err := store.Create(t.Context(), strings.NewReader("expired body"), "text/plain", -time.Second)
|
||||
if err != nil {
|
||||
t.Fatalf("store.Create() error = %v", err)
|
||||
}
|
||||
|
||||
viewReq := httptest.NewRequest(http.MethodGet, "/s/"+meta.ID, nil)
|
||||
viewRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(viewRec, viewReq)
|
||||
if viewRec.Code != http.StatusGone {
|
||||
t.Fatalf("view status = %d, want %d", viewRec.Code, http.StatusGone)
|
||||
}
|
||||
|
||||
rawReq := httptest.NewRequest(http.MethodGet, "/raw/"+meta.ID, nil)
|
||||
rawRec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rawRec, rawReq)
|
||||
if rawRec.Code != http.StatusNotFound {
|
||||
t.Fatalf("raw status after lazy delete = %d, want %d", rawRec.Code, http.StatusNotFound)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateScratchRejectsMalformedMultipart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := newTestHandler(t, testServerOptions{
|
||||
maxUploadBytes: 1024,
|
||||
defaultTTL: time.Hour,
|
||||
rateLimitEnable: false,
|
||||
})
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", strings.NewReader("not-multipart"))
|
||||
req.Header.Set("Content-Type", "multipart/form-data; boundary=missing")
|
||||
req.RemoteAddr = "127.0.0.1:5057"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusBadRequest {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
type testServerOptions struct {
|
||||
maxUploadBytes int64
|
||||
defaultTTL time.Duration
|
||||
rateLimitEnable bool
|
||||
}
|
||||
|
||||
func newTestHandler(t *testing.T, opts testServerOptions) http.Handler {
|
||||
t.Helper()
|
||||
|
||||
handler, _ := newTestHandlerAndStore(t, opts)
|
||||
return handler
|
||||
}
|
||||
|
||||
func newTestHandlerAndStore(t *testing.T, opts testServerOptions) (http.Handler, *storage.FilesystemStore) {
|
||||
t.Helper()
|
||||
|
||||
dataDir := filepath.Join(t.TempDir(), "data")
|
||||
store, err := storage.NewFilesystemStore(dataDir)
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
|
||||
cfg := config.Config{
|
||||
Server: config.ServerConfig{
|
||||
ListenAddr: ":0",
|
||||
},
|
||||
Limits: config.LimitsConfig{
|
||||
MaxUploadSizeBytes: opts.maxUploadBytes,
|
||||
DefaultTTLDuration: opts.defaultTTL,
|
||||
},
|
||||
Storage: config.StorageConfig{
|
||||
DataDir: dataDir,
|
||||
},
|
||||
Security: config.SecurityConfig{
|
||||
RateLimit: config.RateLimitConfig{
|
||||
Enabled: opts.rateLimitEnable,
|
||||
RequestsPerMinute: 120,
|
||||
Burst: 1,
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
srv := &Server{
|
||||
cfg: cfg,
|
||||
store: store,
|
||||
logger: log.New(io.Discard, "", 0),
|
||||
templates: nil,
|
||||
}
|
||||
return srv.Routes(), store
|
||||
}
|
||||
|
||||
func newTemplateHandlerAndStore(t *testing.T, opts testServerOptions) (http.Handler, *storage.FilesystemStore) {
|
||||
t.Helper()
|
||||
|
||||
handler, store := newTestHandlerAndStore(t, opts)
|
||||
_ = handler // keep shared setup path
|
||||
|
||||
_, filePath, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("runtime.Caller failed")
|
||||
}
|
||||
repoRoot := filepath.Clean(filepath.Join(filepath.Dir(filePath), "..", ".."))
|
||||
tmpl, err := template.ParseFiles(
|
||||
filepath.Join(repoRoot, "web", "templates", "index.html"),
|
||||
filepath.Join(repoRoot, "web", "templates", "view.html"),
|
||||
)
|
||||
if err != nil {
|
||||
t.Fatalf("ParseFiles() error = %v", err)
|
||||
}
|
||||
|
||||
cfg := config.Config{
|
||||
Server: config.ServerConfig{
|
||||
ListenAddr: ":0",
|
||||
},
|
||||
Limits: config.LimitsConfig{
|
||||
MaxUploadSizeBytes: opts.maxUploadBytes,
|
||||
DefaultTTLDuration: opts.defaultTTL,
|
||||
},
|
||||
Storage: config.StorageConfig{
|
||||
DataDir: filepath.Join(t.TempDir(), "unused"),
|
||||
},
|
||||
Security: config.SecurityConfig{
|
||||
RateLimit: config.RateLimitConfig{
|
||||
Enabled: opts.rateLimitEnable,
|
||||
RequestsPerMinute: 120,
|
||||
Burst: 1,
|
||||
},
|
||||
},
|
||||
}
|
||||
srv := &Server{
|
||||
cfg: cfg,
|
||||
store: store,
|
||||
logger: log.New(io.Discard, "", 0),
|
||||
templates: tmpl,
|
||||
}
|
||||
return srv.Routes(), store
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"html/template"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime"
|
||||
"testing"
|
||||
|
||||
"scratchbox/internal/config"
|
||||
"scratchbox/internal/storage"
|
||||
)
|
||||
|
||||
func TestMultipartFileCount(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
if got := multipartFileCount(req); got != 0 {
|
||||
t.Fatalf("multipartFileCount() = %d, want 0", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWriteCreateErrorBranches(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := &Server{}
|
||||
rec1 := httptest.NewRecorder()
|
||||
s.writeCreateError(rec1, &http.MaxBytesError{Limit: 5})
|
||||
if rec1.Code != http.StatusRequestEntityTooLarge {
|
||||
t.Fatalf("max bytes error status = %d, want %d", rec1.Code, http.StatusRequestEntityTooLarge)
|
||||
}
|
||||
|
||||
rec2 := httptest.NewRecorder()
|
||||
s.writeCreateError(rec2, errors.New("request body too large"))
|
||||
if rec2.Code != http.StatusRequestEntityTooLarge {
|
||||
t.Fatalf("body too large string status = %d, want %d", rec2.Code, http.StatusRequestEntityTooLarge)
|
||||
}
|
||||
|
||||
rec3 := httptest.NewRecorder()
|
||||
s.writeCreateError(rec3, errors.New("other parse failure"))
|
||||
if rec3.Code != http.StatusBadRequest {
|
||||
t.Fatalf("generic error status = %d, want %d", rec3.Code, http.StatusBadRequest)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIsLikelyText(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
if !isLikelyText("text/plain; charset=utf-8", []byte{0x00}) {
|
||||
t.Fatalf("text/plain should be treated as text")
|
||||
}
|
||||
if !isLikelyText("application/json", []byte{0x00}) {
|
||||
t.Fatalf("application/json should be treated as text")
|
||||
}
|
||||
if isLikelyText("application/octet-stream", []byte{0xff, 0xfe, 0xfd}) {
|
||||
t.Fatalf("binary octet-stream should not be treated as text")
|
||||
}
|
||||
if !isLikelyText("", []byte("hello")) {
|
||||
t.Fatalf("utf8 payload should be treated as text")
|
||||
}
|
||||
if !isLikelyText("", nil) {
|
||||
t.Fatalf("empty payload should be treated as text")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewServerLoadsTemplates(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
store, err := storage.NewFilesystemStore(filepath.Join(tmp, "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
cfg := config.Config{
|
||||
Limits: config.LimitsConfig{MaxUploadSizeBytes: 1024},
|
||||
Security: config.SecurityConfig{
|
||||
RateLimit: config.RateLimitConfig{Enabled: false},
|
||||
},
|
||||
}
|
||||
|
||||
_, file, _, ok := runtime.Caller(0)
|
||||
if !ok {
|
||||
t.Fatal("runtime.Caller failed")
|
||||
}
|
||||
repoRoot := filepath.Clean(filepath.Join(filepath.Dir(file), "..", ".."))
|
||||
wd, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatalf("Getwd() error = %v", err)
|
||||
}
|
||||
if err := os.Chdir(repoRoot); err != nil {
|
||||
t.Fatalf("Chdir() error = %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = os.Chdir(wd)
|
||||
}()
|
||||
|
||||
srv, err := NewServer(cfg, store, log.New(io.Discard, "", 0))
|
||||
if err != nil {
|
||||
t.Fatalf("NewServer() error = %v", err)
|
||||
}
|
||||
if srv == nil || srv.templates == nil {
|
||||
t.Fatalf("expected initialized server with templates")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewServerTemplateErrorAndIndexError(t *testing.T) {
|
||||
tmp := t.TempDir()
|
||||
store, err := storage.NewFilesystemStore(filepath.Join(tmp, "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
cfg := config.Config{
|
||||
Limits: config.LimitsConfig{MaxUploadSizeBytes: 1024},
|
||||
Security: config.SecurityConfig{
|
||||
RateLimit: config.RateLimitConfig{Enabled: false},
|
||||
},
|
||||
}
|
||||
wd, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatalf("Getwd() error = %v", err)
|
||||
}
|
||||
if err := os.Chdir(tmp); err != nil {
|
||||
t.Fatalf("Chdir() error = %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_ = os.Chdir(wd)
|
||||
}()
|
||||
|
||||
if _, err := NewServer(cfg, store, log.New(io.Discard, "", 0)); err == nil {
|
||||
t.Fatalf("expected NewServer() template parse error")
|
||||
}
|
||||
|
||||
s := &Server{}
|
||||
func() {
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Fatalf("expected panic when templates are nil")
|
||||
}
|
||||
}()
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
s.index(rec, req)
|
||||
}()
|
||||
}
|
||||
|
||||
func TestIndexExecuteTemplateError(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s := &Server{
|
||||
templates: template.New("empty"),
|
||||
}
|
||||
rec := httptest.NewRecorder()
|
||||
req := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
s.index(rec, req)
|
||||
if rec.Code != http.StatusInternalServerError {
|
||||
t.Fatalf("index status = %d, want %d", rec.Code, http.StatusInternalServerError)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,153 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"log"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/time/rate"
|
||||
)
|
||||
|
||||
type visitor struct {
|
||||
limiter *rate.Limiter
|
||||
lastSeen time.Time
|
||||
}
|
||||
|
||||
type RateLimiter struct {
|
||||
mu sync.Mutex
|
||||
visitors map[string]*visitor
|
||||
rate rate.Limit
|
||||
burst int
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func NewRateLimiter(requestsPerMinute int, burst int) *RateLimiter {
|
||||
return &RateLimiter{
|
||||
visitors: map[string]*visitor{},
|
||||
rate: rate.Limit(float64(requestsPerMinute) / 60.0),
|
||||
burst: burst,
|
||||
ttl: 10 * time.Minute,
|
||||
}
|
||||
}
|
||||
|
||||
func (r *RateLimiter) Allow(ip string) bool {
|
||||
now := time.Now()
|
||||
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
|
||||
for key, v := range r.visitors {
|
||||
if now.Sub(v.lastSeen) > r.ttl {
|
||||
delete(r.visitors, key)
|
||||
}
|
||||
}
|
||||
|
||||
v, ok := r.visitors[ip]
|
||||
if !ok {
|
||||
v = &visitor{
|
||||
limiter: rate.NewLimiter(r.rate, r.burst),
|
||||
}
|
||||
r.visitors[ip] = v
|
||||
}
|
||||
|
||||
v.lastSeen = now
|
||||
return v.limiter.Allow()
|
||||
}
|
||||
|
||||
func AllowlistMiddleware(allowed []netip.Prefix, trustProxyHeaders bool, logger *log.Logger) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
if len(allowed) == 0 {
|
||||
return next
|
||||
}
|
||||
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
clientIP, ok := extractClientIP(r, trustProxyHeaders)
|
||||
if !ok {
|
||||
http.Error(w, "unable to determine client ip", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
|
||||
for _, prefix := range allowed {
|
||||
if prefix.Contains(clientIP) {
|
||||
next.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
logger.Printf("allowlist denied request method=%s path=%s ip=%s", r.Method, r.URL.Path, clientIP.String())
|
||||
http.Error(w, "ip not allowed", http.StatusForbidden)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func RateLimitMiddleware(limiter *RateLimiter, trustProxyHeaders bool) func(http.Handler) http.Handler {
|
||||
return func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
clientIP, ok := extractClientIP(r, trustProxyHeaders)
|
||||
if !ok {
|
||||
http.Error(w, "unable to determine client ip", http.StatusTooManyRequests)
|
||||
return
|
||||
}
|
||||
|
||||
if limiter.Allow(clientIP.String()) {
|
||||
next.ServeHTTP(w, r)
|
||||
return
|
||||
}
|
||||
|
||||
w.Header().Set("Retry-After", "60")
|
||||
writeJSON(w, http.StatusTooManyRequests, map[string]string{
|
||||
"error": "too many requests from this IP, retry later",
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func Chain(handler http.Handler, middlewares ...func(http.Handler) http.Handler) http.Handler {
|
||||
wrapped := handler
|
||||
for i := len(middlewares) - 1; i >= 0; i-- {
|
||||
wrapped = middlewares[i](wrapped)
|
||||
}
|
||||
return wrapped
|
||||
}
|
||||
|
||||
func extractClientIP(r *http.Request, trustProxyHeaders bool) (netip.Addr, bool) {
|
||||
if trustProxyHeaders {
|
||||
if xff := strings.TrimSpace(r.Header.Get("X-Forwarded-For")); xff != "" {
|
||||
parts := strings.Split(xff, ",")
|
||||
if len(parts) > 0 {
|
||||
if ip, err := netip.ParseAddr(strings.TrimSpace(parts[0])); err == nil {
|
||||
return ip, true
|
||||
}
|
||||
}
|
||||
}
|
||||
if xrip := strings.TrimSpace(r.Header.Get("X-Real-IP")); xrip != "" {
|
||||
if ip, err := netip.ParseAddr(xrip); err == nil {
|
||||
return ip, true
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
host, _, err := net.SplitHostPort(strings.TrimSpace(r.RemoteAddr))
|
||||
if err != nil {
|
||||
if ip, parseErr := netip.ParseAddr(strings.TrimSpace(r.RemoteAddr)); parseErr == nil {
|
||||
return ip, true
|
||||
}
|
||||
return netip.Addr{}, false
|
||||
}
|
||||
ip, err := netip.ParseAddr(host)
|
||||
if err != nil {
|
||||
return netip.Addr{}, false
|
||||
}
|
||||
return ip, true
|
||||
}
|
||||
|
||||
func writeJSON(w http.ResponseWriter, status int, payload any) {
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
w.WriteHeader(status)
|
||||
_ = json.NewEncoder(w).Encode(payload)
|
||||
}
|
||||
@@ -0,0 +1,217 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestAllowlistMiddlewareDeniesAndAllowsByIP(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
logger := log.New(io.Discard, "", 0)
|
||||
allowed := []netip.Prefix{netip.MustParsePrefix("10.0.0.0/8")}
|
||||
mw := AllowlistMiddleware(allowed, false, logger)
|
||||
|
||||
next := http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
})
|
||||
handler := mw(next)
|
||||
|
||||
reqAllowed := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
reqAllowed.RemoteAddr = "10.1.2.3:1234"
|
||||
recAllowed := httptest.NewRecorder()
|
||||
handler.ServeHTTP(recAllowed, reqAllowed)
|
||||
if recAllowed.Code != http.StatusNoContent {
|
||||
t.Fatalf("allowed request status = %d, want %d", recAllowed.Code, http.StatusNoContent)
|
||||
}
|
||||
|
||||
reqDenied := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
reqDenied.RemoteAddr = "192.168.10.5:1234"
|
||||
recDenied := httptest.NewRecorder()
|
||||
handler.ServeHTTP(recDenied, reqDenied)
|
||||
if recDenied.Code != http.StatusForbidden {
|
||||
t.Fatalf("denied request status = %d, want %d", recDenied.Code, http.StatusForbidden)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllowlistMiddlewareHonorsProxyHeaderWhenTrusted(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
logger := log.New(io.Discard, "", 0)
|
||||
allowed := []netip.Prefix{netip.MustParsePrefix("203.0.113.5/32")}
|
||||
mw := AllowlistMiddleware(allowed, true, logger)
|
||||
handler := mw(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req.RemoteAddr = "10.0.0.10:9999"
|
||||
req.Header.Set("X-Forwarded-For", "203.0.113.5, 198.51.100.2")
|
||||
rec := httptest.NewRecorder()
|
||||
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusNoContent {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusNoContent)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimitMiddlewareKeysByClientIP(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
limiter := NewRateLimiter(60, 1)
|
||||
handler := RateLimitMiddleware(limiter, false)(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
|
||||
req1 := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req1.RemoteAddr = "198.51.100.7:1111"
|
||||
rec1 := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec1, req1)
|
||||
if rec1.Code != http.StatusNoContent {
|
||||
t.Fatalf("first request status = %d, want %d", rec1.Code, http.StatusNoContent)
|
||||
}
|
||||
|
||||
req2 := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req2.RemoteAddr = "198.51.100.7:2222"
|
||||
rec2 := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec2, req2)
|
||||
if rec2.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("second same-ip request status = %d, want %d", rec2.Code, http.StatusTooManyRequests)
|
||||
}
|
||||
|
||||
req3 := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req3.RemoteAddr = "198.51.100.8:3333"
|
||||
rec3 := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec3, req3)
|
||||
if rec3.Code != http.StatusNoContent {
|
||||
t.Fatalf("different-ip request status = %d, want %d", rec3.Code, http.StatusNoContent)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllowlistMiddlewareNoRestrictionsReturnsNext(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
handler := AllowlistMiddleware(nil, false, log.New(io.Discard, "", 0))(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusAccepted)
|
||||
}))
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req.RemoteAddr = "not-a-valid-remote-addr"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusAccepted {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusAccepted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllowlistMiddlewareRejectsUnknownClientIP(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
logger := log.New(io.Discard, "", 0)
|
||||
allowed := []netip.Prefix{netip.MustParsePrefix("10.0.0.0/8")}
|
||||
handler := AllowlistMiddleware(allowed, false, logger)(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req.RemoteAddr = "garbage"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusForbidden {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusForbidden)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimitMiddlewareUnknownIP(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
limiter := NewRateLimiter(60, 1)
|
||||
handler := RateLimitMiddleware(limiter, false)(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
req := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req.RemoteAddr = "bad-addr"
|
||||
rec := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec, req)
|
||||
if rec.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("status = %d, want %d", rec.Code, http.StatusTooManyRequests)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimitMiddleware429PayloadAndRetryHeader(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
limiter := NewRateLimiter(1, 1)
|
||||
handler := RateLimitMiddleware(limiter, false)(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}))
|
||||
req1 := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req1.RemoteAddr = "198.18.0.1:1111"
|
||||
handler.ServeHTTP(httptest.NewRecorder(), req1)
|
||||
|
||||
req2 := httptest.NewRequest(http.MethodPost, "/api/scratch", nil)
|
||||
req2.RemoteAddr = "198.18.0.1:1112"
|
||||
rec2 := httptest.NewRecorder()
|
||||
handler.ServeHTTP(rec2, req2)
|
||||
if rec2.Code != http.StatusTooManyRequests {
|
||||
t.Fatalf("status = %d, want %d", rec2.Code, http.StatusTooManyRequests)
|
||||
}
|
||||
if got := rec2.Header().Get("Retry-After"); got != "60" {
|
||||
t.Fatalf("Retry-After = %q, want %q", got, "60")
|
||||
}
|
||||
if ct := rec2.Header().Get("Content-Type"); !strings.Contains(ct, "application/json") {
|
||||
t.Fatalf("Content-Type = %q, want application/json", ct)
|
||||
}
|
||||
var payload map[string]string
|
||||
if err := json.Unmarshal(rec2.Body.Bytes(), &payload); err != nil {
|
||||
t.Fatalf("json unmarshal error = %v", err)
|
||||
}
|
||||
if payload["error"] == "" {
|
||||
t.Fatalf("expected error payload")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimiterPrunesStaleVisitors(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
limiter := NewRateLimiter(60, 1)
|
||||
limiter.ttl = time.Nanosecond
|
||||
limiter.visitors["old"] = &visitor{lastSeen: time.Now().Add(-time.Hour)}
|
||||
if !limiter.Allow("new") {
|
||||
t.Fatalf("Allow(new) = false, want true")
|
||||
}
|
||||
if _, ok := limiter.visitors["old"]; ok {
|
||||
t.Fatalf("stale visitor should be removed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestExtractClientIPFallbacks(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
req := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
req.RemoteAddr = "203.0.113.9"
|
||||
ip, ok := extractClientIP(req, false)
|
||||
if !ok || ip.String() != "203.0.113.9" {
|
||||
t.Fatalf("extract direct ip = (%v,%v), want (203.0.113.9,true)", ip, ok)
|
||||
}
|
||||
|
||||
req2 := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
req2.RemoteAddr = "192.0.2.50:8080"
|
||||
req2.Header.Set("X-Forwarded-For", "bad,198.51.100.1")
|
||||
req2.Header.Set("X-Real-IP", "198.51.100.5")
|
||||
ip2, ok2 := extractClientIP(req2, true)
|
||||
if !ok2 || ip2.String() != "198.51.100.5" {
|
||||
t.Fatalf("extract x-real-ip = (%v,%v), want (198.51.100.5,true)", ip2, ok2)
|
||||
}
|
||||
|
||||
req3 := httptest.NewRequest(http.MethodGet, "/", nil)
|
||||
req3.RemoteAddr = "still-not-an-ip"
|
||||
if _, ok3 := extractClientIP(req3, false); ok3 {
|
||||
t.Fatalf("extractClientIP() ok = true, want false")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,293 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"compress/gzip"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
indexFilename = "metadata.json"
|
||||
filesDirName = "scratches"
|
||||
idBytes = 12
|
||||
)
|
||||
|
||||
var ErrNotFound = errors.New("scratch not found")
|
||||
|
||||
type Metadata struct {
|
||||
ID string `json:"id"`
|
||||
FilePath string `json:"file_path"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
ExpiresAt time.Time `json:"expires_at"`
|
||||
ContentType string `json:"content_type"`
|
||||
Size int64 `json:"size"`
|
||||
}
|
||||
|
||||
type FilesystemStore struct {
|
||||
dataDir string
|
||||
filesDir string
|
||||
index string
|
||||
|
||||
mu sync.RWMutex
|
||||
metadata map[string]Metadata
|
||||
}
|
||||
|
||||
func NewFilesystemStore(dataDir string) (*FilesystemStore, error) {
|
||||
filesDir := filepath.Join(dataDir, filesDirName)
|
||||
if err := os.MkdirAll(filesDir, 0o755); err != nil {
|
||||
return nil, fmt.Errorf("create files dir: %w", err)
|
||||
}
|
||||
|
||||
store := &FilesystemStore{
|
||||
dataDir: dataDir,
|
||||
filesDir: filesDir,
|
||||
index: filepath.Join(dataDir, indexFilename),
|
||||
metadata: map[string]Metadata{},
|
||||
}
|
||||
if err := store.loadIndex(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := store.reconcileOnStartup(time.Now()); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return store, nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) Create(ctx context.Context, body io.Reader, contentType string, ttl time.Duration) (Metadata, error) {
|
||||
id, err := generateID()
|
||||
if err != nil {
|
||||
return Metadata{}, fmt.Errorf("generate id: %w", err)
|
||||
}
|
||||
|
||||
tempFile, err := os.CreateTemp(s.filesDir, id+".tmp-*")
|
||||
if err != nil {
|
||||
return Metadata{}, fmt.Errorf("create temp file: %w", err)
|
||||
}
|
||||
|
||||
gzipWriter := gzip.NewWriter(tempFile)
|
||||
size, copyErr := io.Copy(gzipWriter, body)
|
||||
gzipCloseErr := gzipWriter.Close()
|
||||
closeErr := tempFile.Close()
|
||||
if copyErr != nil {
|
||||
_ = os.Remove(tempFile.Name())
|
||||
return Metadata{}, fmt.Errorf("write scratch content: %w", copyErr)
|
||||
}
|
||||
if gzipCloseErr != nil {
|
||||
_ = os.Remove(tempFile.Name())
|
||||
return Metadata{}, fmt.Errorf("close gzip writer: %w", gzipCloseErr)
|
||||
}
|
||||
if closeErr != nil {
|
||||
_ = os.Remove(tempFile.Name())
|
||||
return Metadata{}, fmt.Errorf("close temp file: %w", closeErr)
|
||||
}
|
||||
|
||||
finalPath := filepath.Join(s.filesDir, id)
|
||||
if err := os.Rename(tempFile.Name(), finalPath); err != nil {
|
||||
_ = os.Remove(tempFile.Name())
|
||||
return Metadata{}, fmt.Errorf("atomically move scratch file: %w", err)
|
||||
}
|
||||
|
||||
now := time.Now().UTC()
|
||||
meta := Metadata{
|
||||
ID: id,
|
||||
FilePath: finalPath,
|
||||
CreatedAt: now,
|
||||
ExpiresAt: now.Add(ttl),
|
||||
ContentType: contentType,
|
||||
Size: size,
|
||||
}
|
||||
|
||||
s.mu.Lock()
|
||||
s.metadata[id] = meta
|
||||
if err := s.persistLocked(); err != nil {
|
||||
delete(s.metadata, id)
|
||||
s.mu.Unlock()
|
||||
_ = os.Remove(finalPath)
|
||||
return Metadata{}, err
|
||||
}
|
||||
s.mu.Unlock()
|
||||
return meta, nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) Get(id string) (Metadata, error) {
|
||||
s.mu.RLock()
|
||||
meta, ok := s.metadata[id]
|
||||
s.mu.RUnlock()
|
||||
if !ok {
|
||||
return Metadata{}, ErrNotFound
|
||||
}
|
||||
return meta, nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) Open(id string) (io.ReadCloser, Metadata, error) {
|
||||
meta, err := s.Get(id)
|
||||
if err != nil {
|
||||
return nil, Metadata{}, err
|
||||
}
|
||||
file, err := os.Open(meta.FilePath)
|
||||
if err != nil {
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil, Metadata{}, ErrNotFound
|
||||
}
|
||||
return nil, Metadata{}, fmt.Errorf("open scratch file: %w", err)
|
||||
}
|
||||
gzipReader, err := gzip.NewReader(file)
|
||||
if err != nil {
|
||||
_ = file.Close()
|
||||
return nil, Metadata{}, fmt.Errorf("create gzip reader: %w", err)
|
||||
}
|
||||
return &combinedReadCloser{
|
||||
reader: gzipReader,
|
||||
closers: []io.Closer{
|
||||
gzipReader,
|
||||
file,
|
||||
},
|
||||
}, meta, nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) Delete(id string) error {
|
||||
s.mu.Lock()
|
||||
meta, ok := s.metadata[id]
|
||||
if !ok {
|
||||
s.mu.Unlock()
|
||||
return ErrNotFound
|
||||
}
|
||||
|
||||
delete(s.metadata, id)
|
||||
if err := s.persistLocked(); err != nil {
|
||||
s.metadata[id] = meta
|
||||
s.mu.Unlock()
|
||||
return err
|
||||
}
|
||||
s.mu.Unlock()
|
||||
|
||||
if err := os.Remove(meta.FilePath); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return fmt.Errorf("remove scratch file: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) DeleteExpired(now time.Time) (int, error) {
|
||||
s.mu.Lock()
|
||||
expired := make([]Metadata, 0)
|
||||
for id, meta := range s.metadata {
|
||||
if !meta.ExpiresAt.After(now) {
|
||||
expired = append(expired, meta)
|
||||
delete(s.metadata, id)
|
||||
}
|
||||
}
|
||||
if len(expired) == 0 {
|
||||
s.mu.Unlock()
|
||||
return 0, nil
|
||||
}
|
||||
if err := s.persistLocked(); err != nil {
|
||||
for _, meta := range expired {
|
||||
s.metadata[meta.ID] = meta
|
||||
}
|
||||
s.mu.Unlock()
|
||||
return 0, err
|
||||
}
|
||||
s.mu.Unlock()
|
||||
|
||||
for _, meta := range expired {
|
||||
if err := os.Remove(meta.FilePath); err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return 0, fmt.Errorf("remove expired file %s: %w", meta.ID, err)
|
||||
}
|
||||
}
|
||||
return len(expired), nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) loadIndex() error {
|
||||
b, err := os.ReadFile(s.index)
|
||||
if err != nil {
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("read metadata index: %w", err)
|
||||
}
|
||||
|
||||
loaded := map[string]Metadata{}
|
||||
if err := json.Unmarshal(b, &loaded); err != nil {
|
||||
return fmt.Errorf("decode metadata index: %w", err)
|
||||
}
|
||||
s.metadata = loaded
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) persistLocked() error {
|
||||
tmp := s.index + ".tmp"
|
||||
payload, err := json.MarshalIndent(s.metadata, "", " ")
|
||||
if err != nil {
|
||||
return fmt.Errorf("encode metadata index: %w", err)
|
||||
}
|
||||
if err := os.WriteFile(tmp, payload, 0o644); err != nil {
|
||||
return fmt.Errorf("write metadata temp file: %w", err)
|
||||
}
|
||||
if err := os.Rename(tmp, s.index); err != nil {
|
||||
return fmt.Errorf("replace metadata index: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *FilesystemStore) reconcileOnStartup(now time.Time) error {
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
|
||||
changed := false
|
||||
for id, meta := range s.metadata {
|
||||
if !meta.ExpiresAt.After(now) {
|
||||
delete(s.metadata, id)
|
||||
_ = os.Remove(meta.FilePath)
|
||||
changed = true
|
||||
continue
|
||||
}
|
||||
if _, err := os.Stat(meta.FilePath); err != nil {
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
delete(s.metadata, id)
|
||||
changed = true
|
||||
continue
|
||||
}
|
||||
return fmt.Errorf("stat scratch %s: %w", id, err)
|
||||
}
|
||||
}
|
||||
if !changed {
|
||||
return nil
|
||||
}
|
||||
return s.persistLocked()
|
||||
}
|
||||
|
||||
func generateID() (string, error) {
|
||||
b := make([]byte, idBytes)
|
||||
if _, err := rand.Read(b); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return hex.EncodeToString(b), nil
|
||||
}
|
||||
|
||||
type combinedReadCloser struct {
|
||||
reader io.Reader
|
||||
closers []io.Closer
|
||||
}
|
||||
|
||||
func (c *combinedReadCloser) Read(p []byte) (int, error) {
|
||||
return c.reader.Read(p)
|
||||
}
|
||||
|
||||
func (c *combinedReadCloser) Close() error {
|
||||
var firstErr error
|
||||
for _, closer := range c.closers {
|
||||
if err := closer.Close(); err != nil && firstErr == nil {
|
||||
firstErr = err
|
||||
}
|
||||
}
|
||||
return firstErr
|
||||
}
|
||||
@@ -0,0 +1,402 @@
|
||||
package storage
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestCreateStoresGzipAndOpenIsTransparent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
|
||||
original := bytes.Repeat([]byte("hello scratch "), 1024)
|
||||
meta, err := store.Create(context.Background(), bytes.NewReader(original), "text/plain; charset=utf-8", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("Create() error = %v", err)
|
||||
}
|
||||
if meta.Size != int64(len(original)) {
|
||||
t.Fatalf("meta.Size = %d, want %d", meta.Size, len(original))
|
||||
}
|
||||
|
||||
onDisk, err := os.ReadFile(meta.FilePath)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadFile() error = %v", err)
|
||||
}
|
||||
if len(onDisk) < 2 || onDisk[0] != 0x1f || onDisk[1] != 0x8b {
|
||||
t.Fatalf("stored file does not have gzip header")
|
||||
}
|
||||
|
||||
reader, _, err := store.Open(meta.ID)
|
||||
if err != nil {
|
||||
t.Fatalf("Open() error = %v", err)
|
||||
}
|
||||
defer reader.Close()
|
||||
got, err := io.ReadAll(reader)
|
||||
if err != nil {
|
||||
t.Fatalf("ReadAll() error = %v", err)
|
||||
}
|
||||
if !bytes.Equal(got, original) {
|
||||
t.Fatalf("open content mismatch: got %d bytes, want %d", len(got), len(original))
|
||||
}
|
||||
}
|
||||
|
||||
func TestGetDeleteAndNotFound(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
meta, err := store.Create(context.Background(), bytes.NewReader([]byte("hello")), "text/plain", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("Create() error = %v", err)
|
||||
}
|
||||
if _, err := store.Get(meta.ID); err != nil {
|
||||
t.Fatalf("Get() error = %v", err)
|
||||
}
|
||||
if err := store.Delete(meta.ID); err != nil {
|
||||
t.Fatalf("Delete() error = %v", err)
|
||||
}
|
||||
if _, err := store.Get(meta.ID); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatalf("Get() error = %v, want ErrNotFound", err)
|
||||
}
|
||||
if err := store.Delete(meta.ID); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatalf("Delete() error = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteExpired(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
expired, err := store.Create(context.Background(), bytes.NewReader([]byte("expired")), "text/plain", -time.Second)
|
||||
if err != nil {
|
||||
t.Fatalf("Create expired() error = %v", err)
|
||||
}
|
||||
active, err := store.Create(context.Background(), bytes.NewReader([]byte("active")), "text/plain", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("Create active() error = %v", err)
|
||||
}
|
||||
deleted, err := store.DeleteExpired(time.Now().UTC())
|
||||
if err != nil {
|
||||
t.Fatalf("DeleteExpired() error = %v", err)
|
||||
}
|
||||
if deleted != 1 {
|
||||
t.Fatalf("DeleteExpired() = %d, want 1", deleted)
|
||||
}
|
||||
if _, err := store.Get(expired.ID); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatalf("expired Get() error = %v, want ErrNotFound", err)
|
||||
}
|
||||
if _, err := store.Get(active.ID); err != nil {
|
||||
t.Fatalf("active Get() error = %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestOpenInvalidGzipFails(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
id := "deadbeefdeadbeefdeadbeef"
|
||||
badPath := filepath.Join(store.filesDir, id)
|
||||
if err := os.WriteFile(badPath, []byte("not-gzip"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
store.metadata[id] = Metadata{
|
||||
ID: id,
|
||||
FilePath: badPath,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
ExpiresAt: time.Now().UTC().Add(time.Hour),
|
||||
ContentType: "text/plain",
|
||||
Size: int64(len("not-gzip")),
|
||||
}
|
||||
if err := store.persistLocked(); err != nil {
|
||||
t.Fatalf("persistLocked() error = %v", err)
|
||||
}
|
||||
if _, _, err := store.Open(id); err == nil {
|
||||
t.Fatalf("Open() error = nil, want gzip reader error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewFilesystemStoreInvalidIndex(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
root := t.TempDir()
|
||||
scratches := filepath.Join(root, filesDirName)
|
||||
if err := os.MkdirAll(scratches, 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, indexFilename), []byte("{bad"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
if _, err := NewFilesystemStore(root); err == nil {
|
||||
t.Fatalf("NewFilesystemStore() error = nil, want invalid index error")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewFilesystemStoreReconcileOnStartup(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
root := t.TempDir()
|
||||
scratches := filepath.Join(root, filesDirName)
|
||||
if err := os.MkdirAll(scratches, 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
keepID := "aaaaaaaaaaaaaaaaaaaaaaaa"
|
||||
missingID := "bbbbbbbbbbbbbbbbbbbbbbbb"
|
||||
expiredID := "cccccccccccccccccccccccc"
|
||||
keepPath := filepath.Join(scratches, keepID)
|
||||
expiredPath := filepath.Join(scratches, expiredID)
|
||||
if err := os.WriteFile(keepPath, []byte{0x1f, 0x8b, 0x08, 0x00}, 0o644); err != nil {
|
||||
t.Fatalf("WriteFile keep error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(expiredPath, []byte{0x1f, 0x8b, 0x08, 0x00}, 0o644); err != nil {
|
||||
t.Fatalf("WriteFile expired error = %v", err)
|
||||
}
|
||||
|
||||
meta := map[string]Metadata{
|
||||
keepID: {
|
||||
ID: keepID,
|
||||
FilePath: keepPath,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
ExpiresAt: time.Now().UTC().Add(time.Hour),
|
||||
ContentType: "text/plain",
|
||||
Size: 1,
|
||||
},
|
||||
missingID: {
|
||||
ID: missingID,
|
||||
FilePath: filepath.Join(scratches, missingID),
|
||||
CreatedAt: time.Now().UTC(),
|
||||
ExpiresAt: time.Now().UTC().Add(time.Hour),
|
||||
ContentType: "text/plain",
|
||||
Size: 1,
|
||||
},
|
||||
expiredID: {
|
||||
ID: expiredID,
|
||||
FilePath: expiredPath,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
ExpiresAt: time.Now().UTC().Add(-time.Minute),
|
||||
ContentType: "text/plain",
|
||||
Size: 1,
|
||||
},
|
||||
}
|
||||
payload, err := json.Marshal(meta)
|
||||
if err != nil {
|
||||
t.Fatalf("json.Marshal() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, indexFilename), payload, 0o644); err != nil {
|
||||
t.Fatalf("WriteFile index error = %v", err)
|
||||
}
|
||||
|
||||
store, err := NewFilesystemStore(root)
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
if _, err := store.Get(keepID); err != nil {
|
||||
t.Fatalf("keep entry missing: %v", err)
|
||||
}
|
||||
if _, err := store.Get(missingID); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatalf("missing entry error = %v, want ErrNotFound", err)
|
||||
}
|
||||
if _, err := store.Get(expiredID); !errors.Is(err, ErrNotFound) {
|
||||
t.Fatalf("expired entry error = %v, want ErrNotFound", err)
|
||||
}
|
||||
}
|
||||
|
||||
type errCloser struct{}
|
||||
|
||||
func (errCloser) Close() error { return errors.New("close failed") }
|
||||
|
||||
func TestCombinedReadCloserClosePropagatesFirstError(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
c := &combinedReadCloser{
|
||||
reader: bytes.NewReader([]byte("ok")),
|
||||
closers: []io.Closer{errCloser{}, errCloser{}},
|
||||
}
|
||||
if err := c.Close(); err == nil {
|
||||
t.Fatalf("Close() error = nil, want non-nil")
|
||||
}
|
||||
}
|
||||
|
||||
type failingReader struct{}
|
||||
|
||||
func (failingReader) Read(_ []byte) (int, error) { return 0, errors.New("boom") }
|
||||
|
||||
func TestCreateRollbackOnPersistFailure(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
|
||||
badIndexDir := filepath.Join(t.TempDir(), "index-dir")
|
||||
if err := os.MkdirAll(badIndexDir, 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
store.index = badIndexDir
|
||||
|
||||
if _, err := store.Create(context.Background(), bytes.NewReader([]byte("x")), "text/plain", time.Hour); err == nil {
|
||||
t.Fatalf("Create() error = nil, want persist failure")
|
||||
}
|
||||
if len(store.metadata) != 0 {
|
||||
t.Fatalf("metadata should be rolled back on persist failure")
|
||||
}
|
||||
}
|
||||
|
||||
func TestCreateReaderFailure(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
if _, err := store.Create(context.Background(), failingReader{}, "text/plain", time.Hour); err == nil {
|
||||
t.Fatalf("Create() error = nil, want reader failure")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeletePersistFailureRestoresEntry(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
meta, err := store.Create(context.Background(), bytes.NewReader([]byte("hello")), "text/plain", time.Hour)
|
||||
if err != nil {
|
||||
t.Fatalf("Create() error = %v", err)
|
||||
}
|
||||
|
||||
badIndexDir := filepath.Join(t.TempDir(), "index-dir")
|
||||
if err := os.MkdirAll(badIndexDir, 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
store.index = badIndexDir
|
||||
|
||||
if err := store.Delete(meta.ID); err == nil {
|
||||
t.Fatalf("Delete() error = nil, want persist failure")
|
||||
}
|
||||
if _, err := store.Get(meta.ID); err != nil {
|
||||
t.Fatalf("entry should be restored after delete persist failure: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteRemoveFailure(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
|
||||
nonEmptyDir := filepath.Join(t.TempDir(), "non-empty")
|
||||
if err := os.MkdirAll(nonEmptyDir, 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(nonEmptyDir, "child"), []byte("x"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
id := "dddddddddddddddddddddddd"
|
||||
store.metadata[id] = Metadata{
|
||||
ID: id,
|
||||
FilePath: nonEmptyDir,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
ExpiresAt: time.Now().UTC().Add(time.Hour),
|
||||
ContentType: "text/plain",
|
||||
Size: 1,
|
||||
}
|
||||
if err := store.persistLocked(); err != nil {
|
||||
t.Fatalf("persistLocked() error = %v", err)
|
||||
}
|
||||
if err := store.Delete(id); err == nil {
|
||||
t.Fatalf("Delete() error = nil, want remove failure")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteExpiredPersistFailureRestoresEntries(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
meta, err := store.Create(context.Background(), bytes.NewReader([]byte("expired")), "text/plain", -time.Second)
|
||||
if err != nil {
|
||||
t.Fatalf("Create() error = %v", err)
|
||||
}
|
||||
|
||||
badIndexDir := filepath.Join(t.TempDir(), "index-dir")
|
||||
if err := os.MkdirAll(badIndexDir, 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
store.index = badIndexDir
|
||||
|
||||
if _, err := store.DeleteExpired(time.Now().UTC()); err == nil {
|
||||
t.Fatalf("DeleteExpired() error = nil, want persist failure")
|
||||
}
|
||||
if _, err := store.Get(meta.ID); err != nil {
|
||||
t.Fatalf("entry should be restored after DeleteExpired persist failure: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteExpiredRemoveFailure(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
store, err := NewFilesystemStore(filepath.Join(t.TempDir(), "data"))
|
||||
if err != nil {
|
||||
t.Fatalf("NewFilesystemStore() error = %v", err)
|
||||
}
|
||||
nonEmptyDir := filepath.Join(t.TempDir(), "expired-non-empty")
|
||||
if err := os.MkdirAll(nonEmptyDir, 0o755); err != nil {
|
||||
t.Fatalf("MkdirAll() error = %v", err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(nonEmptyDir, "child"), []byte("x"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
id := "eeeeeeeeeeeeeeeeeeeeeeee"
|
||||
store.metadata[id] = Metadata{
|
||||
ID: id,
|
||||
FilePath: nonEmptyDir,
|
||||
CreatedAt: time.Now().UTC(),
|
||||
ExpiresAt: time.Now().UTC().Add(-time.Second),
|
||||
ContentType: "text/plain",
|
||||
Size: 1,
|
||||
}
|
||||
if err := store.persistLocked(); err != nil {
|
||||
t.Fatalf("persistLocked() error = %v", err)
|
||||
}
|
||||
if _, err := store.DeleteExpired(time.Now().UTC()); err == nil {
|
||||
t.Fatalf("DeleteExpired() error = nil, want remove failure")
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewFilesystemStoreFailsWhenDataDirIsFile(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
tmp := t.TempDir()
|
||||
filePath := filepath.Join(tmp, "not-dir")
|
||||
if err := os.WriteFile(filePath, []byte("x"), 0o644); err != nil {
|
||||
t.Fatalf("WriteFile() error = %v", err)
|
||||
}
|
||||
if _, err := NewFilesystemStore(filePath); err == nil {
|
||||
t.Fatalf("NewFilesystemStore() error = nil, want mkdir failure")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
body {
|
||||
margin: 0;
|
||||
font-family: Inter, system-ui, sans-serif;
|
||||
background: #10141a;
|
||||
color: #e5e7eb;
|
||||
}
|
||||
|
||||
.container {
|
||||
max-width: 900px;
|
||||
margin: 2rem auto;
|
||||
padding: 0 1rem 2rem;
|
||||
}
|
||||
|
||||
textarea {
|
||||
width: 100%;
|
||||
box-sizing: border-box;
|
||||
border: 1px solid #334155;
|
||||
border-radius: 6px;
|
||||
background: #0f172a;
|
||||
color: #e2e8f0;
|
||||
padding: 0.75rem;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
button {
|
||||
background: #3b82f6;
|
||||
border: 0;
|
||||
color: white;
|
||||
border-radius: 6px;
|
||||
padding: 0.6rem 1rem;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.error {
|
||||
margin-top: 1rem;
|
||||
color: #fecaca;
|
||||
}
|
||||
|
||||
.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
pre {
|
||||
background: #0b1220;
|
||||
border: 1px solid #334155;
|
||||
border-radius: 6px;
|
||||
padding: 1rem;
|
||||
overflow: auto;
|
||||
white-space: pre-wrap;
|
||||
}
|
||||
|
||||
a {
|
||||
color: #93c5fd;
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Scratchbox</title>
|
||||
<link rel="stylesheet" href="/static/app.css">
|
||||
</head>
|
||||
<body>
|
||||
<main class="container">
|
||||
<h1>Scratchbox</h1>
|
||||
<p>Create an unauthenticated scratch. It expires automatically.</p>
|
||||
|
||||
<form id="scratch-form">
|
||||
<label for="content">Text content</label>
|
||||
<textarea id="content" name="content" rows="14" placeholder="Paste text here"></textarea>
|
||||
<button type="submit">Create scratch</button>
|
||||
</form>
|
||||
|
||||
<section id="result" class="hidden">
|
||||
<h2>Created</h2>
|
||||
<p><strong>View:</strong> <a id="view-link" href="#" target="_blank" rel="noreferrer"></a></p>
|
||||
<p><strong>Raw:</strong> <a id="raw-link" href="#" target="_blank" rel="noreferrer"></a></p>
|
||||
<p id="expire-text"></p>
|
||||
</section>
|
||||
|
||||
<section id="error" class="error hidden"></section>
|
||||
</main>
|
||||
|
||||
<script>
|
||||
const form = document.getElementById("scratch-form");
|
||||
const result = document.getElementById("result");
|
||||
const error = document.getElementById("error");
|
||||
const viewLink = document.getElementById("view-link");
|
||||
const rawLink = document.getElementById("raw-link");
|
||||
const expireText = document.getElementById("expire-text");
|
||||
|
||||
form.addEventListener("submit", async (event) => {
|
||||
event.preventDefault();
|
||||
result.classList.add("hidden");
|
||||
error.classList.add("hidden");
|
||||
|
||||
const content = document.getElementById("content").value;
|
||||
try {
|
||||
const response = await fetch("/api/scratch", {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "text/plain; charset=utf-8" },
|
||||
body: content
|
||||
});
|
||||
|
||||
if (!response.ok) {
|
||||
const message = await response.text();
|
||||
throw new Error(message || `request failed (${response.status})`);
|
||||
}
|
||||
|
||||
const payload = await response.json();
|
||||
viewLink.href = payload.view_url;
|
||||
viewLink.textContent = payload.view_url;
|
||||
rawLink.href = payload.raw_url;
|
||||
rawLink.textContent = payload.raw_url;
|
||||
expireText.textContent = `Expires at: ${payload.expires_at}`;
|
||||
result.classList.remove("hidden");
|
||||
} catch (err) {
|
||||
error.textContent = err.message;
|
||||
error.classList.remove("hidden");
|
||||
}
|
||||
});
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,26 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>Scratch {{ .ID }}</title>
|
||||
<link rel="stylesheet" href="/static/app.css">
|
||||
</head>
|
||||
<body>
|
||||
<main class="container">
|
||||
<h1>Scratch {{ .ID }}</h1>
|
||||
<p>Content type: <code>{{ .ContentType }}</code></p>
|
||||
<p>Expires at: {{ .ExpiresAt }}</p>
|
||||
<p><a href="{{ .RawURL }}">Open raw</a></p>
|
||||
|
||||
{{ if .Binary }}
|
||||
<p>This scratch looks binary and is not rendered in-page. Use the raw link.</p>
|
||||
{{ else }}
|
||||
<pre>{{ .Content }}</pre>
|
||||
{{ if .Truncated }}
|
||||
<p>Preview truncated to 2MB. Open raw for full content.</p>
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user