Durable background agents for pull requests

Interactive coding agents run in a loop with a human in the seat. When a tool call hangs, an endpoint times out, or a prompt drifts off-target, the human hits interrupt, restates the instruction, or fixes the line by hand.

Background agents get no such intervention.

When you wire an autonomous model into a GitHub webhook or CI trigger, it runs in an unattended distributed environment. Webhooks redeliver on network blips. Upstream LLM providers throw rate limits (HTTP 429) mid-step. Push events arrive concurrently while an earlier commit is still being inspected.

If the agent is designed as a naive linear script, it fails in predictable, expensive ways: duplicate review spam on pull requests, state loss on retries, and shallow hallucinations when inspecting large diffs without caller context.

Making an asynchronous review agent production-ready requires treating it as a durable distributed state machine rather than a one-shot prompt.

01of 03The failure modes of stateless runners

Most agent prototypes start as an action script triggered on pull_request.opened. In a test repository with small diffs and one developer, it looks functional. In a busy engineering pipeline, three mechanical failures emerge immediately.

1. The redelivery comment storm

GitHub's webhook delivery guarantees are at-least-once, not exactly-once. If your webhook receiver takes longer than 10 seconds to acknowledge receipt, GitHub retries the payload.

A stateless agent processing a 40-second evaluation will receive the duplicate webhook while the first is still executing. Without explicit idempotency keys tied to (repository_id, pull_request_number, head_sha), the system spawns a second run and posts two conflicting reviews on the developer's pull request.

The same problem happens when a developer pushes a fast follow-up commit while the agent is halfway through reviewing the previous SHA. The agent must cancel the stale run or idempotently update a single designated review comment rather than appending noise.

2. Cascading rate limits on long tool chains

Evaluating a complex pull request is rarely a single model call. A proper architectural audit requires multiple dependent steps:

1. Ingesting the patch and changed file manifests.
2. Querying repository architectural decision records (docs/adr/).
3. Running static search tools (ast-grep or ripgrep) to find callers of modified public signatures.
4. Synthesizing a structured critique against repository policies.
5. Publishing the formatted review via the GitHub API.

If step 4 hits an LLM token quota or network timeout, a naive runner restarts the entire job from step 1. It re-fetches the diff, re-runs the caller search, and consumes redundant tokens — often worsening the rate limit that caused the initial failure.

3. Shallow reviews from isolated diffs

A diff shows what changed, but never what depends on the change.

When a pull request alters an internal database contract or an API response signature, the diff itself may look clean. If the agent only inspects the lines enclosed in the patch, it cannot tell whether a downstream caller three directories away is now passing undefined arguments.

Prompting the model to "be careful about callers" does nothing if the agent has no tool access to verify those callers against the live repository tree.

02of 03Designing around durable execution

To solve these failure modes, the agent's control loop must be decoupled into isolated, deterministic steps orchestrated by a durable execution engine (such as Inngest or a persistent workflow queue).

export const reviewPullRequestWorkflow = inngest.createFunction(
  {
    id: "github-pr-architectural-sentinel",
    concurrency: {
      limit: 5,
      key: "event.data.repository.full_name",
    },
    retries: 3,
  },
  { event: "github/pr.received" },
  async ({ event, step }) => {
    const { owner, repo, pullNumber, commitSha } = event.data;

    // Step 1: Idempotently fetch PR diff and changed files
    const prContext = await step.run("fetch-pr-context", async () => {
      return await fetchPrContext(owner, repo, pullNumber, commitSha);
    });

    // Step 2: Extract relevant repository architectural contracts
    const adrContracts = await step.run("fetch-architectural-contracts", async () => {
      return await findRelevantADRs(owner, repo, prContext.changedFiles);
    });

    // Step 3: Run tool-assisted caller graph inspection
    const callerImpact = await step.run("inspect-caller-impact", async () => {
      return await searchAffectedCallers(owner, repo, prContext.diff);
    });

    // Step 4: Synthesize structured architectural critique
    const reviewResult = await step.run("synthesize-architectural-review", async () => {
      return await generateStructuredReview({
        prContext,
        adrContracts,
        callerImpact,
      });
    });

    // Step 5: Idempotently publish or update the GitHub PR review
    return await step.run("publish-github-review", async () => {
      return await postOrUpdateReview({
        owner,
        repo,
        pullNumber,
        commitSha,
        review: reviewResult,
      });
    });
  }
);

Checkpointing intermediate outputs

Each step.run boundary is a durable checkpoint. When inspect-caller-impact completes, its output is serialized and stored. If synthesize-architectural-review subsequently fails on a temporary 503 from the AI provider, the orchestrator sleeps with exponential backoff and retries only Step 4.

The diff is not re-fetched, the repository tree is not re-indexed, and the GitHub API quota remains untouched.

Bounding the review to physical contracts

An automated review comment that asserts "this might violate architectural conventions" without evidence is friction, not help.

The synthesizer step enforces a strict Zod schema. Every reported violation must link directly to a recognized repository contract (like an ADR file) and specify the affected call site discovered during Step 3:

export const RuleViolationSchema = z.object({
  ruleId: z.string().describe("e.g. ADR-0003 or SCHEMA-01"),
  title: z.string(),
  file: z.string(),
  lineRange: z.string().optional(),
  explanation: z.string().describe("Concrete mechanical failure, not opinion"),
  suggestedRemediation: z.string().describe("Direct code replacement or refactor step"),
  severity: z.enum(["blocker", "warning", "note"]),
});

If a rule violation cannot cite a specific file and documented rule, the schema rejects the output and prevents the comment from being published.

03of 03What survives the transition to autonomy

Background agents are not chatbots with the chat UI stripped away. They are asynchronous worker processes that happen to use language models for semantic evaluation.

When we treat them with the same operational rigor as payment webhooks or database migrations — enforcing strict idempotency, durable step boundaries, and deterministic tool grounding — they stop being unpredictable novelties and become reliable parts of the delivery pipeline.

---

The reference implementation with runnable Inngest workflows, Zod review schemas, and an offline simulation harness is open-source at jarnielcataluna/sentinel-agent-poc.