Troubleshooting

This page lists the worker's failures by the code they report, and the problems that report no code at all because the worker simply stops offering some kind of work. A failed step shows its code in Pidgen, and the same line is in the worker's log (see Logs and diagnostics).

Installing, starting, stopping and updating the worker, and checking whether it is connected, belong to the Pidgen CLI: see Install a worker and Workers.

Start with pidgen-worker doctor

Whatever the symptom, run this first, as the user the worker runs as:

pidgen-worker doctor

It names every missing command, invalid file and logged-out subscription it finds, with what to do about each. Logs and diagnostics explains its checks.

Some work is never picked up

The worker checks the machine before it offers work to Pidgen, and it does not offer work it cannot do. That keeps a step from failing half-way through, but it also means the problem is silent: nothing fails, the work just waits. These are the usual causes.

A kind of worker never starts

The workspace worker needs git. Without it, the worker starts no workspace worker, and no workspace is prepared or delivered on this machine; prompt and script steps are not affected. pidgen-worker doctor reports it under lane:workspace. Install Git 2.25 or newer and the workspace worker starts within a minute, with no restart.

A coding agent's steps never run here

A coding agent runs on this worker only when all of these hold:

• Pidgen allows it for your Account;
• its command is installed and on the worker's PATH (doctor reports it under harness:<harness>);
• it is logged in: an agent whose login is missing is not offered until you log it in, and doctor reports each subscription under subscription:<harness>/<label>;
• node.yaml gives it a max above 0 (harnesses.<harness>.max).

Fix whichever does not hold. The worker notices an installed command or a new login within a minute, and a change to node.yaml as described in Configure the worker. See Coding agents and subscriptions.

Steps wait while a prompt worker sits idle

The coding agent is at its session cap: it already runs as many sessions as its max allows, and idle prompt workers stop offering it until one of its sessions ends. If the machine and your provider's limits allow more, raise the cap:

pidgen-worker capacity set claude --max 3

Two copies of the worker's processes are running

The manager starts every worker process itself. A prompt, script or workspace worker started any other way, for example by a second service or a copy started by hand, cannot receive work, and it runs beside the manager's own. List what is running:

ps -ef | grep '[p]idgen-worker'

You should see one manager and one line for each worker it started. If there is anything else, stop the worker completely with the Pidgen CLI, check that the command above prints nothing, remove whatever started the extra process, and start the worker again. Do not just restart the manager while the extra processes are still running.

A workspace could not be prepared (WORKSPACE_PROVISION_FAILED)

Preparing a workspace means cloning or fetching the repository and checking out the ticket's branch. When that fails, the failure carries the cause and the remedy (cause_code, finding_code and remediation). When a git command failed, it also says which one:

Field	What it says
git_operation	The git operation that failed, such as fetch or worktree add.
git_exit_status	Its exit status.
git_reason	What the worker read from git's output (below).
git_diagnostic	A short excerpt of git's own message. It never contains the command line, the environment or any credential.

git_reason	What to do
storage_exhausted	The disk is full. Free space on the disk that holds the worker's home, then retry.
permission_denied	The worker cannot write in its home. Make the user the worker runs as the owner of the home and everything in it.
lock_collision	Another git process held a lock on the repository. Retry; if it happens again, check that nothing else is running git in the worker's repos/ or workspaces/.
authentication_failed	The forge refused the credential. See Repository access.
remote_unreachable	The worker could not reach the forge. Check the network, the proxy and the remote URL.
object_read_failed	Git could not read part of the clone. Check the disk, then run git fsck in the clone under repos/.
checkout_filter_failed	A checkout filter the repository uses, such as Git LFS, failed or is not installed. Install it for the user the worker runs as.
unclassified	Anything else: read git_diagnostic.

A new workspace is made in two steps: first its branch, then its checkout. When the checkout fails, the worker removes the branch it just made, but only when it can show the branch is unused: still at the commit it was made from, with no checkout using it and nothing at the workspace's path. branch_compensation says what happened:

• outcome removed: the branch is gone, and a retry starts clean once the cause in git_diagnostic is fixed.
• outcome absent: there was no branch to remove.
• outcome preserved: the worker found something it would not decide about, named by reason (ref_advanced, registered_worktree, path_residue, deletion_failed or inspection_failed). The next attempt is refused with existing_workspace_branch until you deal with the branch.

The worker never deletes files to do this, never force-deletes a branch, and never touches a branch it did not make. A branch left by a worker that was killed part-way is never removed for you. Before you delete such a branch yourself, check in the clone under repos/ that it points at the ticket's base commit, has no commits of its own, has no copy on the remote, is used by no checkout (git worktree list), and that nothing is left at the workspace's path.

A workspace has leftover changes (WORKSPACE_RESIDUE_BLOCKED)

Before each turn of a ticket, the worker reports the workspace's git status to Pidgen. Pidgen cannot look at your machine, so this report is all it knows about the workspace. If the report shows changes that no turn accounts for, Pidgen refuses the turn and the step fails without retrying, because looking again would find the same files.

The failure lists the paths (status_paths), relative to the workspace's checkout, workspaces/<workspace>/code under the worker's home. Decide what they are. If you want to keep them, move them out of the workspace; otherwise delete them, or restore tracked files with git restore. When git status in the checkout is clean, retry the ticket in Pidgen. The retry prepares the workspace again, and a clean report lifts the block.

A delivery conflict repair is blocked (DELIVERY_REPAIR_BLOCKED)

When a ticket's branch no longer merges cleanly into its base, Pidgen asks a coding agent to repair it. The worker does the git part itself: it saves any uncommitted work in a checkpoint commit, starts the merge, lets the agent resolve the conflicting files and run the checks, and then commits exactly the tree the agent checked. The agent never gets to change branches or the repository's own files.

DELIVERY_REPAIR_BLOCKED means the worker could not safely do its part. More attempts would only repeat it, so the step fails without retrying. The failure says whether it happened while preparing or finishing the merge, and what to fix:

• The worker could not use git or its repair record, for example a full disk or files in the workspace it cannot write. Fix the access, then retry the ticket in Pidgen.
• The branch or the files changed under it since the repair was prepared. Find out what changed them, put the branch back where the repair left it, then retry.
• The workspace already had staged changes, or a merge in progress that the worker did not start. The worker will not guess whose they are. Look at them in the workspace's checkout: commit or unstage the staged changes, and finish or abort that merge yourself. Then retry.

A retry continues from the saved checkpoint, or finishes a merge already resolved, and it refuses files or branches that changed since. Do not:

• delete pidgen-delivery-repair.json, the worker's record of the repair, kept in the workspace's git directory;
• delete or reset any of the workspace's files;
• abort or reset a merge the worker started;
• give the coding agent access to the whole repository to get around it.

Conflict markers left in files are not this failure: the agent is asked to correct those files, a bounded number of times, as part of the ordinary repair.

The repository needs a local credential (LOCAL_GIT_CREDENTIAL_REQUIRED)

The project's repository must be reached with a credential on this machine, and neither the worker's git_credentials table nor the machine's own git credentials could reach it. The failure names the remote and which credential git ran with, never the token. Add a credential as Repository access describes, or give the machine access to the forge; the next attempt succeeds.

A run is stranded on a worker that is gone

Once a run's workspace is on one worker, the rest of the run happens on that worker, because its checkout and branch are there. When that worker disappears, for example because the machine was switched off, removed or replaced, the run does not move to another worker by itself. It waits, and Pidgen shows it waiting for that worker (locality_unavailable). This is deliberate: it is not a stuck queue, and other workers being idle does not change it.

• If the machine is coming back, start the worker again. The run continues in the same workspace, and nothing else is needed.
• If it is not, the run has to be relocated: moved, on purpose, to another worker, which prepares a new workspace from the remote repository. Anything that was only in the old workspace and never pushed is lost. A person in your Account with the right to do so has to ask for it; Pidgen never relocates a run on its own. It refuses while the old worker is still healthy, while a step of the run is still running, and for a run that is not waiting for its worker.

Give each machine its own worker. Two machines set up under one name count as one worker, and a run pinned to that name can then neither move nor be recovered cleanly.

Events pile up in the spool

Files accumulate in spool/ under the worker's home, the log repeats could not ship a batch; the spool keeps it for the next pass, and steps stay running instead of finishing. The worker is running, but it cannot send its events to Pidgen, and a step does not finish while its events are missing.

This is almost always the connection: a proxy or firewall between the worker and Pidgen that cuts long-lived connections when they are quiet. Fix that first, allowing idle connections of at least five minutes. Then restart the worker to send the backlog at once; otherwise it sends it on its own once the connection holds. Do not delete anything in spool/ or staging/.

The worker stopped and does not reconnect

The log says worker node credential revoked by the authority: the worker was removed from your Account, or its credential was withdrawn. No retry can fix that, so the worker stops for good. Set it up again with the Pidgen CLI if you want this machine back.