Re-verify job-complete race under system-Blender lock #16

Closed
opened 2026-10-07 02:28:48 +00:00 by eva · 2 comments
Owner

Problem / why

Earlier job-complete / upload race concerns under a system-Blender lock need a fresh pass. Either the race is dead (close with proof), still live (small fix issue), or unclear (park). Do not expand scope.

Scope

  • Reproduce complete + upload while a system-Blender lock is held.
  • If dead: close this issue with reproduction proof that it no longer races.
  • If live: file a small, focused fix issue (or convert this one) — no drive-by refactors.
  • If you cannot get a clean read: park with notes.

Acceptance criteria

  • Attempted reproduce of complete+upload under system-Blender lock
  • Dead → close with proof attached/commented
  • Live → small fix issue filed (or this issue narrowed to the fix) with clear repro
  • Otherwise park with what you tried and what blocked a call

Out of scope

  • Blender install / packaging rewrites
  • Unrelated job pipeline cleanup
  • Broad concurrency redesign
  • Repo: s1d3sw1ped/jiggablend

Do not merge yourself — if a fix lands, open a PR and wait for review.

## Problem / why Earlier job-complete / upload race concerns under a system-Blender lock need a fresh pass. Either the race is dead (close with proof), still live (small fix issue), or unclear (park). Do not expand scope. ## Scope - Reproduce complete + upload while a system-Blender lock is held. - If dead: close this issue with reproduction proof that it no longer races. - If live: file a small, focused fix issue (or convert this one) — no drive-by refactors. - If you cannot get a clean read: park with notes. ## Acceptance criteria - [ ] Attempted reproduce of complete+upload under system-Blender lock - [ ] Dead → close with proof attached/commented - [ ] Live → small fix issue filed (or this issue narrowed to the fix) with clear repro - [ ] Otherwise park with what you tried and what blocked a call ## Out of scope - Blender install / packaging rewrites - Unrelated job pipeline cleanup - Broad concurrency redesign ## Links - Repo: `s1d3sw1ped/jiggablend` Do not merge yourself — if a fix lands, open a PR and wait for review.

Parked — no clean live/dead call on jumpbox

Attempted re-verify of complete + upload under lock_system_blender on branch moss/job-complete-race-16 @ 8d10cde.

What I tried

  • Confirmed branch clean at develop tip 8d10cde (includes #14 system-Blender lock).
  • Static read of the lock + complete/upload paths:
    • Manager stamps LockSystemBlender via applySystemBlenderLock on outbound runner metadata.
    • Upload metadata extraction (runBlenderMetadataExtraction) uses ChooseBlenderBinary(lockSystem, "blender") — under lock this is always PATH blender.
    • Job create merge ignores request blender_version when locked and still requires a version from upload analysis.
    • Task complete → updateJobStatusFromTasks still serialized by the per-job mutex from #4.
  • Ran focused unit tests (no Blender needed): LockSystem / ChooseBlender / MergeBlend in internal/config, internal/manager, internal/runner/blender, internal/runner/tasks — all green.
  • Searched jumpbox for a system Blender binary (PATH, /usr, /opt, moss home) — none.

What blocked a call

E2E reproduce of complete + upload while the lock is held needs a host where PATH blender exists and the admin lock can be flipped on. Without that binary, manager-side upload metadata extraction and runner lock path cannot be exercised together, so I cannot honestly mark the race dead or live.

What a real repro needs

  1. Host with system Blender on PATH (and preferably a runner using that same PATH blender).
  2. Manager with lock_system_blender=true.
  3. Concurrent: runner finishing a job (task_complete + output upload) while a user upload session runs metadata extraction (both forced onto system Blender under lock).
  4. Watch for status/metadata races, failed extracts, or jobs created with wrong/missing blender_version under lock.

Leaving this open as parked until that host is available. No fix PR — no drive-by changes on moss/job-complete-race-16.

## Parked — no clean live/dead call on jumpbox Attempted re-verify of complete + upload under `lock_system_blender` on branch `moss/job-complete-race-16` @ `8d10cde`. ### What I tried - Confirmed branch clean at develop tip `8d10cde` (includes #14 system-Blender lock). - Static read of the lock + complete/upload paths: - Manager stamps `LockSystemBlender` via `applySystemBlenderLock` on outbound runner metadata. - Upload metadata extraction (`runBlenderMetadataExtraction`) uses `ChooseBlenderBinary(lockSystem, "blender")` — under lock this is always PATH `blender`. - Job create merge ignores request `blender_version` when locked and still requires a version from upload analysis. - Task complete → `updateJobStatusFromTasks` still serialized by the per-job mutex from #4. - Ran focused unit tests (no Blender needed): `LockSystem` / `ChooseBlender` / `MergeBlend` in `internal/config`, `internal/manager`, `internal/runner/blender`, `internal/runner/tasks` — all green. - Searched jumpbox for a system Blender binary (`PATH`, `/usr`, `/opt`, moss home) — **none**. ### What blocked a call E2E reproduce of **complete + upload while the lock is held** needs a host where PATH `blender` exists and the admin lock can be flipped on. Without that binary, manager-side upload metadata extraction and runner lock path cannot be exercised together, so I cannot honestly mark the race dead or live. ### What a real repro needs 1. Host with system Blender on PATH (and preferably a runner using that same PATH blender). 2. Manager with `lock_system_blender=true`. 3. Concurrent: runner finishing a job (task_complete + output upload) while a user upload session runs metadata extraction (both forced onto system Blender under lock). 4. Watch for status/metadata races, failed extracts, or jobs created with wrong/missing blender_version under lock. Leaving this open as **parked** until that host is available. No fix PR — no drive-by changes on `moss/job-complete-race-16`.
Author
Owner

Closing this as a product call from Mira. We still don't have a host with system Blender to re-check the race on, and it hasn't come back since the fix. Reopen if the job-complete race shows up again.

Closing this as a product call from Mira. We still don't have a host with system Blender to re-check the race on, and it hasn't come back since the fix. Reopen if the job-complete race shows up again.
eva closed this issue 2026-10-07 20:31:57 +00:00
Sign in to join this conversation.
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
s1d3sw1ped/jiggablend#16
No description provided.