184 lines
4.9 KiB
Markdown
184 lines
4.9 KiB
Markdown
# JiggaBlend - Blender Render Farm
|
|
|
|
A distributed Blender render farm system built with Go. The system consists of a manager server that handles job submission, file storage, and runner coordination, and runner clients that execute Blender renders on Linux amd64 systems.
|
|
|
|
## Architecture
|
|
|
|
- **Manager**: Central server with REST API, web UI, DuckDB database, and local file storage
|
|
- **Runner**: Linux amd64 client that connects to manager, receives jobs, executes Blender renders, and reports back
|
|
|
|
## Features
|
|
|
|
- OAuth authentication (Google and Discord)
|
|
- Web-based job submission and monitoring
|
|
- Distributed rendering across multiple runners
|
|
- Real-time job progress tracking
|
|
- File upload/download for Blender files and rendered outputs
|
|
- Runner health monitoring
|
|
|
|
## Prerequisites
|
|
|
|
### Manager
|
|
- Go 1.21 or later
|
|
- DuckDB (via Go driver)
|
|
|
|
### Runner
|
|
- Linux amd64
|
|
- Blender installed and in PATH
|
|
- FFmpeg installed (optional, for video processing)
|
|
|
|
## Installation
|
|
|
|
1. Clone the repository:
|
|
```bash
|
|
git clone <repository-url>
|
|
cd jiggablend
|
|
```
|
|
|
|
2. Install dependencies:
|
|
```bash
|
|
go mod download
|
|
```
|
|
|
|
## Configuration
|
|
|
|
### Manager
|
|
|
|
Set the following environment variables for authentication (optional):
|
|
|
|
```bash
|
|
# OAuth Providers (optional)
|
|
export GOOGLE_CLIENT_ID="your-google-client-id"
|
|
export GOOGLE_CLIENT_SECRET="your-google-client-secret"
|
|
export GOOGLE_REDIRECT_URL="http://localhost:8080/api/auth/google/callback"
|
|
|
|
export DISCORD_CLIENT_ID="your-discord-client-id"
|
|
export DISCORD_CLIENT_SECRET="your-discord-client-secret"
|
|
export DISCORD_REDIRECT_URL="http://localhost:8080/api/auth/discord/callback"
|
|
|
|
# Local Authentication (optional)
|
|
export ENABLE_LOCAL_AUTH="true"
|
|
|
|
# Test User (optional, for testing only)
|
|
# Creates a local user on startup if it doesn't exist
|
|
export LOCAL_TEST_EMAIL="test@example.com"
|
|
export LOCAL_TEST_PASSWORD="testpassword"
|
|
```
|
|
|
|
### Runner
|
|
|
|
No configuration required. Runner will auto-detect hostname and IP.
|
|
|
|
## Usage
|
|
|
|
### Running the Manager
|
|
|
|
```bash
|
|
# Using make
|
|
make run-manager
|
|
|
|
# Or directly
|
|
go run ./cmd/manager
|
|
|
|
# With custom options
|
|
go run ./cmd/manager -port 8080 -db jiggablend.db -storage ./storage
|
|
```
|
|
|
|
The manager will start on `http://localhost:8080` by default.
|
|
|
|
### Running a Runner
|
|
|
|
```bash
|
|
# Using make
|
|
make run-runner
|
|
|
|
# Or directly
|
|
go run ./cmd/runner
|
|
|
|
# With custom options
|
|
go run ./cmd/runner -manager http://localhost:8080 -name my-runner
|
|
```
|
|
|
|
### Building
|
|
|
|
```bash
|
|
# Build manager
|
|
make build-manager
|
|
|
|
# Build runner (Linux amd64)
|
|
make build-runner
|
|
```
|
|
|
|
## OAuth Setup
|
|
|
|
### Google OAuth
|
|
|
|
1. Go to [Google Cloud Console](https://console.cloud.google.com/)
|
|
2. Create a new project or select existing
|
|
3. Enable Google+ API
|
|
4. Create OAuth 2.0 credentials
|
|
5. Add authorized redirect URI: `http://localhost:8080/api/auth/google/callback`
|
|
6. Set environment variables with Client ID and Secret
|
|
|
|
### Discord OAuth
|
|
|
|
1. Go to [Discord Developer Portal](https://discord.com/developers/applications)
|
|
2. Create a new application
|
|
3. Go to OAuth2 section
|
|
4. Add redirect URI: `http://localhost:8080/api/auth/discord/callback`
|
|
5. Set environment variables with Client ID and Secret
|
|
|
|
## Project Structure
|
|
|
|
```
|
|
jiggablend/
|
|
├── cmd/
|
|
│ ├── manager/ # Manager server application
|
|
│ └── runner/ # Runner client application
|
|
├── internal/
|
|
│ ├── api/ # REST API handlers
|
|
│ ├── auth/ # OAuth authentication
|
|
│ ├── database/ # DuckDB database models and migrations
|
|
│ ├── queue/ # Job queue management
|
|
│ ├── storage/ # File storage operations
|
|
│ └── runner/ # Runner management logic
|
|
├── pkg/
|
|
│ └── types/ # Shared types and models
|
|
├── web/ # Static web UI files
|
|
├── go.mod
|
|
└── Makefile
|
|
```
|
|
|
|
## API Endpoints
|
|
|
|
### Authentication
|
|
- `GET /api/auth/google/login` - Initiate Google OAuth
|
|
- `GET /api/auth/google/callback` - Google OAuth callback
|
|
- `GET /api/auth/discord/login` - Initiate Discord OAuth
|
|
- `GET /api/auth/discord/callback` - Discord OAuth callback
|
|
- `POST /api/auth/logout` - Logout
|
|
- `GET /api/auth/me` - Get current user
|
|
|
|
### Jobs
|
|
- `POST /api/jobs` - Create a new job
|
|
- `GET /api/jobs` - List user's jobs
|
|
- `GET /api/jobs/{id}` - Get job details
|
|
- `DELETE /api/jobs/{id}` - Cancel a job
|
|
- `POST /api/jobs/{id}/upload` - Upload job file
|
|
- `GET /api/jobs/{id}/files` - List job files
|
|
- `GET /api/jobs/{id}/files/{fileId}/download` - Download job file
|
|
|
|
### Runners
|
|
- `GET /api/admin/runners` - List all runners (admin only)
|
|
- `POST /api/runner/register` - Register a runner (uses registration token)
|
|
- `POST /api/runner/heartbeat` - Update runner heartbeat (runner authenticated)
|
|
- `GET /api/runner/tasks` - Get pending tasks for runner
|
|
- `POST /api/runner/tasks/{id}/complete` - Mark task as complete
|
|
- `GET /api/runner/files/{jobId}/{fileName}` - Download file for runner
|
|
- `POST /api/runner/files/{jobId}/upload` - Upload file from runner
|
|
|
|
## License
|
|
|
|
MIT
|
|
|