jiggablend/README.md

JiggaBlend - Blender Render Farm

A distributed Blender render farm system built with Go. The system consists of a manager server that handles job submission, file storage, and runner coordination, and runner clients that execute Blender renders on Linux amd64 systems.

Architecture

• Manager: Central server with REST API, embedded web UI, SQLite database, and local file storage
• Runner: Linux amd64 client that connects to manager, receives jobs, executes Blender renders, and reports back

Both manager and runner are part of a single binary (jiggablend) with subcommands.

Features

• Authentication: OAuth (Google and Discord) and local authentication with user management
• Web UI: Server-rendered Go templates with HTMX fragments for job submission and monitoring
• Distributed Rendering: Scale across multiple runners with automatic job distribution
• Real-time Updates: Polling-based UI updates with lightweight HTMX refreshes
• Video Encoding: Automatic video encoding from EXR sequences only. EXR→video always uses HDR (HLG, 10-bit); no option to disable. Codecs:
  • H.264 (MP4) - HDR (HLG)
  • AV1 (MP4) - Alpha channel support, HDR
  • VP9 (WebM) - Alpha channel and HDR
• Output Formats: EXR frame sequence only, or EXR + video (H.264, AV1, VP9). Blender always renders EXR.
• Blender Version Management: Support for multiple Blender versions with automatic detection
• Metadata Extraction: Automatic extraction of scene metadata from Blender files
• Admin Panel: User and runner management interface
• Runner Management: API key-based authentication for runners with health monitoring
• HDR: EXR→video is always encoded as HDR (HLG, 10-bit). There is no option to turn it off; for SDR-only output, download the EXR frames and encode locally.
• Alpha: Alpha is always preserved in EXR frames. In video, alpha is preserved when present in the EXR for AV1 and VP9; H.264 MP4 does not support alpha.

Prerequisites

Manager

• Go 1.25.4 or later
• SQLite (via Go driver)
• Blender installed and in PATH (for metadata extraction)
• ImageMagick installed (for EXR preview conversion)

Runner

• Linux amd64
• FFmpeg installed (required for video encoding)
• Able to run Blender (the runner gets the job’s required Blender version from the manager; it does not need Blender pre-installed)

Installation

1. Clone the repository:

git clone <repository-url>
cd jiggablend

2. Install dependencies:

go mod download

Configuration

Manager

Configuration is managed through the CLI using jiggablend manager config commands. The configuration is stored in the SQLite database.

Initial Setup

For testing, use the Makefile helper:

make init-test

This will:

• Enable local authentication
• Set a fixed API key for testing: jk_r0_test_key_123456789012345678901234567890
• Create a test admin user (test@example.com / testpassword)

Manual Configuration

# Enable local authentication
jiggablend manager config enable localauth

# Add a user
jiggablend manager config add user <email> <password> --admin

# Generate an API key for runners
jiggablend manager config add apikey <name> --scope manager

# Set OAuth credentials
jiggablend manager config set google-oauth <client-id> <client-secret> --redirect-url <url>
jiggablend manager config set discord-oauth <client-id> <client-secret> --redirect-url <url>

# View current configuration
jiggablend manager config show

# List users and API keys
jiggablend manager config list users
jiggablend manager config list apikeys

Environment Variables

You can also use environment variables with the JIGGABLEND_ prefix:

• JIGGABLEND_PORT - Server port (default: 8080)
• JIGGABLEND_DB - Database path (default: jiggablend.db)
• JIGGABLEND_STORAGE - Storage path (default: ./jiggablend-storage)
• JIGGABLEND_LOG_FILE - Log file path
• JIGGABLEND_LOG_LEVEL - Log level (debug, info, warn, error)
• JIGGABLEND_VERBOSE - Enable verbose logging

Runner

The runner requires an API key to connect to the manager. The runner will auto-detect hostname and IP.

Usage

Building

# Build the unified binary (includes embedded web UI)
make build

# Or build directly
go build -o bin/jiggablend ./cmd/jiggablend

# Build web UI separately
make build-web

Running the Manager

# Using make (includes test setup)
make run-manager

# Or directly
bin/jiggablend manager

# With custom options
bin/jiggablend manager --port 8080 --db jiggablend.db --storage ./jiggablend-storage --log-file manager.log

# Using environment variables
JIGGABLEND_PORT=8080 JIGGABLEND_DB=jiggablend.db bin/jiggablend manager

The manager will start on http://localhost:8080 by default.

Running a Runner

# Using make (uses test API key)
make run-runner

# Or directly (requires API key)
bin/jiggablend runner --api-key <your-api-key>

# With custom options
bin/jiggablend runner --manager http://localhost:8080 --name my-runner --api-key <key> --log-file runner.log

# Using environment variables
JIGGABLEND_MANAGER=http://localhost:8080 JIGGABLEND_API_KEY=<key> bin/jiggablend runner

Render Chunk Size Note

For one heavy production scene/profile, chunked rendering (frames 800-804 in one Blender process) was much slower than one-frame tasks:

• Chunked task (800-804): 27m49s end-to-end (Task assigned -> last Saved)
• Single-frame tasks (800, 801, 802, 803, 804): 15m04s wall clock total

In that test, any chunk size greater than 1 caused a major slowdown after the first frame. Fresh installs should already have it set to 1, but if you see similar performance degradation, try forcing one frame per task (hard reset Blender each frame): jiggablend manager config set frames-per-render-task 1. If 1 is worse on your scene/hardware, benchmark and use a higher chunk size instead.

Running Both (for Testing)

# Run manager and runner in parallel
make run

This will start both the manager and a test runner with a fixed API key.

OAuth Setup

Google OAuth

1. Go to Google Cloud Console
2. Create a new project or select existing
3. Enable Google+ API
4. Create OAuth 2.0 credentials
5. Add authorized redirect URI: http://localhost:8080/api/auth/google/callback
6. Configure using CLI:

   jiggablend manager config set google-oauth <client-id> <client-secret> --redirect-url http://localhost:8080/api/auth/google/callback
   

Discord OAuth

1. Go to Discord Developer Portal
2. Create a new application
3. Go to OAuth2 section
4. Add redirect URI: http://localhost:8080/api/auth/discord/callback
5. Configure using CLI:

   jiggablend manager config set discord-oauth <client-id> <client-secret> --redirect-url http://localhost:8080/api/auth/discord/callback
   

Project Structure

jiggablend/
├── cmd/
│   └── jiggablend/       # Unified CLI application
│       ├── cmd/          # Cobra command definitions
│       └── main.go       # Entry point
├── internal/
│   ├── auth/             # Authentication (OAuth, local, sessions)
│   ├── config/           # Configuration management
│   ├── database/         # SQLite database models and migrations
│   ├── logger/           # Logging utilities
│   ├── manager/          # Manager server logic
│   ├── runner/           # Runner client logic
│   │   ├── api/          # Manager API client
│   │   ├── blender/      # Blender version detection
│   │   ├── encoding/     # Video encoding (H.264, AV1, VP9)
│   │   ├── tasks/        # Task execution (render, encode, process)
│   │   └── workspace/    # Workspace management
│   └── storage/          # File storage operations
├── pkg/
│   ├── executils/        # Execution utilities
│   ├── scripts/          # Python scripts for Blender
│   └── types/             # Shared types and models
├── web/                   # Embedded templates + static assets
│   ├── templates/        # Go HTML templates and partials
│   └── static/           # CSS/JS assets
├── go.mod
└── Makefile

API Endpoints

Authentication

• GET /api/auth/google/login - Initiate Google OAuth
• GET /api/auth/google/callback - Google OAuth callback
• GET /api/auth/discord/login - Initiate Discord OAuth
• GET /api/auth/discord/callback - Discord OAuth callback
• POST /api/auth/login - Local authentication login
• POST /api/auth/register - User registration (if enabled)
• POST /api/auth/logout - Logout
• GET /api/auth/me - Get current user
• POST /api/auth/password/change - Change password

Jobs

• POST /api/jobs - Create a new job
• GET /api/jobs - List user's jobs
• GET /api/jobs/{id} - Get job details
• DELETE /api/jobs/{id} - Cancel a job
• POST /api/jobs/{id}/upload - Upload job file (Blender file)
• GET /api/jobs/{id}/files - List job files
• GET /api/jobs/{id}/files/{fileId}/download - Download job file
• GET /api/jobs/{id}/metadata - Extract metadata from uploaded file
• GET /api/jobs/{id}/outputs - List job output files

Blender

• GET /api/blender/versions - List available Blender versions

Runners (Internal API)

• POST /api/runner/register - Register a runner (uses API key)
• POST /api/runner/heartbeat - Update runner heartbeat
• GET /api/runner/tasks - Get pending tasks for runner
• POST /api/runner/tasks/{id}/complete - Mark task as complete
• GET /api/runner/files/{jobId}/{fileName} - Download file for runner
• POST /api/runner/files/{jobId}/upload - Upload file from runner

Admin (Admin Only)

• GET /api/admin/runners - List all runners
• GET /api/admin/jobs - List all jobs
• GET /api/admin/users - List all users
• GET /api/admin/stats - System statistics

WebSocket

• WS /api/jobs/ws - Optional API channel for advanced clients
• The default web UI uses polling + HTMX for status updates and task views.

Output Formats

The system supports the following output formats. Blender always renders EXR (linear); the chosen format is the deliverable (frames only or frames + video).

Deliverable Formats

• EXR - EXR frame sequence only (no video)
• EXR_264_MP4 - EXR frames + H.264 MP4 (always HDR, HLG)
• EXR_AV1_MP4 - EXR frames + AV1 MP4 (alpha support, always HDR)
• EXR_VP9_WEBM - EXR frames + VP9 WebM (alpha and HDR)

Video encoding (EXR→video) is always HDR (HLG, 10-bit); there is no option to output SDR video. For SDR-only, download the EXR frames and encode locally.

Video encoding features:

• 2-pass encoding for optimal quality
• EXR→video only (no PNG source); always HLG (HDR), 10-bit, full range
• Alpha channel preservation (AV1 and VP9 only)
• Software encoding (libx264, libaom-av1, libvpx-vp9)

Storage Structure

The manager uses a local storage directory (default: ./jiggablend-storage) with the following structure:

jiggablend-storage/
├── blender-versions/     # Bundled Blender versions
│   └── <version>/
├── jobs/                 # Job context files
│   └── <job-id>/
│       └── context.tar
├── outputs/              # Rendered outputs
│   └── <job-id>/
├── temp/                 # Temporary files
└── uploads/              # Uploaded files

Development

Running Tests

make test
# Or directly
go test ./... -timeout 30s

Web UI Development

The web UI is server-rendered from embedded templates and static assets in web/templates and web/static. No Node/Vite build step is required.

License

MIT