# JiggaBlend - Blender Render Farm A distributed Blender render farm system built with Go. The system consists of a manager server that handles job submission, file storage, and runner coordination, and runner clients that execute Blender renders on Linux amd64 systems. ## Architecture - **Manager**: Central server with REST API, embedded web UI, SQLite database, and local file storage - **Runner**: Linux amd64 client that connects to manager, receives jobs, executes Blender renders, and reports back Both manager and runner are part of a single binary (`jiggablend`) with subcommands. ## Features - **Authentication**: OAuth (Google and Discord) and local authentication with user management - **Web UI**: Server-rendered Go templates with HTMX fragments for job submission and monitoring - **Distributed Rendering**: Scale across multiple runners with automatic job distribution - **Real-time Updates**: Polling-based UI updates with lightweight HTMX refreshes - **Video Encoding**: Automatic video encoding from EXR sequences only. EXR→video always uses HDR (HLG, 10-bit); no option to disable. Codecs: - H.264 (MP4) - HDR (HLG) - AV1 (MP4) - Alpha channel support, HDR - VP9 (WebM) - Alpha channel and HDR - **Output Formats**: EXR frame sequence only, or EXR + video (H.264, AV1, VP9). Blender always renders EXR. - **Blender Version Management**: Support for multiple Blender versions with automatic detection - **Metadata Extraction**: Automatic extraction of scene metadata from Blender files - **Admin Panel**: User and runner management interface - **Runner Management**: API key-based authentication for runners with health monitoring - **HDR**: EXR→video is always encoded as HDR (HLG, 10-bit). There is no option to turn it off; for SDR-only output, download the EXR frames and encode locally. - **Alpha**: Alpha is always preserved in EXR frames. In video, alpha is preserved when present in the EXR for AV1 and VP9; H.264 MP4 does not support alpha. ## Prerequisites ### Manager - Go 1.25.4 or later - SQLite (via Go driver) - Blender installed and in PATH (for metadata extraction) - ImageMagick installed (for EXR preview conversion) ### Runner - Linux amd64 - Blender installed (can use bundled versions from storage) - FFmpeg installed (required for video encoding) ## Installation 1. Clone the repository: ```bash git clone cd jiggablend ``` 2. Install dependencies: ```bash go mod download ``` ## Configuration ### Manager Configuration is managed through the CLI using `jiggablend manager config` commands. The configuration is stored in the SQLite database. #### Initial Setup For testing, use the Makefile helper: ```bash make init-test ``` This will: - Enable local authentication - Set a fixed API key for testing - Create a test admin user (test@example.com / testpassword) #### Manual Configuration ```bash # Enable local authentication jiggablend manager config enable localauth # Add a user jiggablend manager config add user --admin # Generate an API key for runners jiggablend manager config add apikey --scope manager # Set OAuth credentials jiggablend manager config set google-oauth --redirect-url jiggablend manager config set discord-oauth --redirect-url # View current configuration jiggablend manager config show # List users and API keys jiggablend manager config list users jiggablend manager config list apikeys ``` #### Environment Variables You can also use environment variables with the `JIGGABLEND_` prefix: - `JIGGABLEND_PORT` - Server port (default: 8080) - `JIGGABLEND_DB` - Database path (default: jiggablend.db) - `JIGGABLEND_STORAGE` - Storage path (default: ./jiggablend-storage) - `JIGGABLEND_LOG_FILE` - Log file path - `JIGGABLEND_LOG_LEVEL` - Log level (debug, info, warn, error) - `JIGGABLEND_VERBOSE` - Enable verbose logging ### Runner The runner requires an API key to connect to the manager. The runner will auto-detect hostname and IP. ## Usage ### Building ```bash # Build the unified binary (includes embedded web UI) make build # Or build directly go build -o bin/jiggablend ./cmd/jiggablend # Build web UI separately make build-web ``` ### Running the Manager ```bash # Using make (includes test setup) make run-manager # Or directly bin/jiggablend manager # With custom options bin/jiggablend manager --port 8080 --db jiggablend.db --storage ./jiggablend-storage --log-file manager.log # Using environment variables JIGGABLEND_PORT=8080 JIGGABLEND_DB=jiggablend.db bin/jiggablend manager ``` The manager will start on `http://localhost:8080` by default. ### Running a Runner ```bash # Using make (uses test API key) make run-runner # Or directly (requires API key) bin/jiggablend runner --api-key # With custom options bin/jiggablend runner --manager http://localhost:8080 --name my-runner --api-key --log-file runner.log # Using environment variables JIGGABLEND_MANAGER=http://localhost:8080 JIGGABLEND_API_KEY= bin/jiggablend runner ``` ### Render Chunk Size Note For one heavy production scene/profile, chunked rendering (`frames 800-804` in one Blender process) was much slower than one-frame tasks: - Chunked task (`800-804`): `27m49s` end-to-end (`Task assigned` -> last `Saved`) - Single-frame tasks (`800`, `801`, `802`, `803`, `804`): `15m04s` wall clock total In that test, any chunk size greater than `1` caused a major slowdown after the first frame. Fresh installs should already have it set to `1`, but if you see similar performance degradation, try forcing one frame per task (hard reset Blender each frame): `jiggablend manager config set frames-per-render-task 1`. If `1` is worse on your scene/hardware, benchmark and use a higher chunk size instead. ### Running Both (for Testing) ```bash # Run manager and runner in parallel make run ``` This will start both the manager and a test runner with a fixed API key. ## OAuth Setup ### Google OAuth 1. Go to [Google Cloud Console](https://console.cloud.google.com/) 2. Create a new project or select existing 3. Enable Google+ API 4. Create OAuth 2.0 credentials 5. Add authorized redirect URI: `http://localhost:8080/api/auth/google/callback` 6. Configure using CLI: ```bash jiggablend manager config set google-oauth --redirect-url http://localhost:8080/api/auth/google/callback ``` ### Discord OAuth 1. Go to [Discord Developer Portal](https://discord.com/developers/applications) 2. Create a new application 3. Go to OAuth2 section 4. Add redirect URI: `http://localhost:8080/api/auth/discord/callback` 5. Configure using CLI: ```bash jiggablend manager config set discord-oauth --redirect-url http://localhost:8080/api/auth/discord/callback ``` ## Project Structure ``` jiggablend/ ├── cmd/ │ └── jiggablend/ # Unified CLI application │ ├── cmd/ # Cobra command definitions │ └── main.go # Entry point ├── internal/ │ ├── auth/ # Authentication (OAuth, local, sessions) │ ├── config/ # Configuration management │ ├── database/ # SQLite database models and migrations │ ├── logger/ # Logging utilities │ ├── manager/ # Manager server logic │ ├── runner/ # Runner client logic │ │ ├── api/ # Manager API client │ │ ├── blender/ # Blender version detection │ │ ├── encoding/ # Video encoding (H.264, AV1, VP9) │ │ ├── tasks/ # Task execution (render, encode, process) │ │ └── workspace/ # Workspace management │ └── storage/ # File storage operations ├── pkg/ │ ├── executils/ # Execution utilities │ ├── scripts/ # Python scripts for Blender │ └── types/ # Shared types and models ├── web/ # Embedded templates + static assets │ ├── templates/ # Go HTML templates and partials │ └── static/ # CSS/JS assets ├── go.mod └── Makefile ``` ## API Endpoints ### Authentication - `GET /api/auth/google/login` - Initiate Google OAuth - `GET /api/auth/google/callback` - Google OAuth callback - `GET /api/auth/discord/login` - Initiate Discord OAuth - `GET /api/auth/discord/callback` - Discord OAuth callback - `POST /api/auth/login` - Local authentication login - `POST /api/auth/register` - User registration (if enabled) - `POST /api/auth/logout` - Logout - `GET /api/auth/me` - Get current user - `POST /api/auth/password/change` - Change password ### Jobs - `POST /api/jobs` - Create a new job - `GET /api/jobs` - List user's jobs - `GET /api/jobs/{id}` - Get job details - `DELETE /api/jobs/{id}` - Cancel a job - `POST /api/jobs/{id}/upload` - Upload job file (Blender file) - `GET /api/jobs/{id}/files` - List job files - `GET /api/jobs/{id}/files/{fileId}/download` - Download job file - `GET /api/jobs/{id}/metadata` - Extract metadata from uploaded file - `GET /api/jobs/{id}/outputs` - List job output files ### Blender - `GET /api/blender/versions` - List available Blender versions ### Runners (Internal API) - `POST /api/runner/register` - Register a runner (uses API key) - `POST /api/runner/heartbeat` - Update runner heartbeat - `GET /api/runner/tasks` - Get pending tasks for runner - `POST /api/runner/tasks/{id}/complete` - Mark task as complete - `GET /api/runner/files/{jobId}/{fileName}` - Download file for runner - `POST /api/runner/files/{jobId}/upload` - Upload file from runner ### Admin (Admin Only) - `GET /api/admin/runners` - List all runners - `GET /api/admin/jobs` - List all jobs - `GET /api/admin/users` - List all users - `GET /api/admin/stats` - System statistics ### WebSocket - `WS /api/jobs/ws` - Optional API channel for advanced clients - The default web UI uses polling + HTMX for status updates and task views. ## Output Formats The system supports the following output formats. Blender always renders EXR (linear); the chosen format is the deliverable (frames only or frames + video). ### Deliverable Formats - **EXR** - EXR frame sequence only (no video) - **EXR_264_MP4** - EXR frames + H.264 MP4 (always HDR, HLG) - **EXR_AV1_MP4** - EXR frames + AV1 MP4 (alpha support, always HDR) - **EXR_VP9_WEBM** - EXR frames + VP9 WebM (alpha and HDR) Video encoding (EXR→video) is always HDR (HLG, 10-bit); there is no option to output SDR video. For SDR-only, download the EXR frames and encode locally. Video encoding features: - 2-pass encoding for optimal quality - EXR→video only (no PNG source); always HLG (HDR), 10-bit, full range - Alpha channel preservation (AV1 and VP9 only) - Software encoding (libx264, libaom-av1, libvpx-vp9) ## Storage Structure The manager uses a local storage directory (default: `./jiggablend-storage`) with the following structure: ``` jiggablend-storage/ ├── blender-versions/ # Bundled Blender versions │ └── / ├── jobs/ # Job context files │ └── / │ └── context.tar ├── outputs/ # Rendered outputs │ └── / ├── temp/ # Temporary files └── uploads/ # Uploaded files ``` ## Development ### Running Tests ```bash make test # Or directly go test ./... -timeout 30s ``` ### Web UI Development The web UI is server-rendered from embedded templates and static assets in `web/templates` and `web/static`. No Node/Vite build step is required. ## License MIT