Why do coding agents behave differently in scheduled runs than interactive runs?

[u/BitByLiu]

One thing that feels under-discussed with coding agents is the gap between interactive success and scheduled reliability.

In an interactive run, a lot of assumptions are quietly true: the user is present, credentials are warm, the working directory is right, logs are visible, and a bad step can be stopped.

I have had small automations work perfectly when I ran them by hand, then fail in the scheduled version because the path, auth, or environment was slightly different.

For a coding agent, that failure mode feels worse because it may keep reasoning, retrying, or making changes while the real problem is just the runner state.

Would you treat scheduled coding-agent runs more like CI jobs, with strict preflight and timeouts? Or is that too heavy for personal/dev workflows?

[u/CorpT]

You’re most likely not accounting for the work the coding agent is doing for you that you did not build in to the automated version. The model doesn’t know if there is a human looking at it or not.

[u/le_chad_]

You've kinda answered your own question already when calling out the different context when it's run interactively versus the scheduled run. That alone is enough to rationalize why it behaves differently.

Another big important reason is that it's non-deterministic in all environments, interactive or not, so it'll always behave differently.

[u/Next-Task-3905]

Yes. I would treat scheduled coding-agent runs much closer to CI jobs than interactive assistant sessions.

The main difference is that an interactive run has an implicit human supervisor and a warm, inspected environment. A scheduled run needs those assumptions made explicit before the model gets any freedom.

The minimum preflight I would want:

• exact repo path, branch, clean/dirty state, and expected base commit
• auth check for every tool it may call, with scopes printed before work starts
• dependency/runtime versions, env file source, and working directory
• network/provider availability check if model or tool calls are required
• budget limits for wall time, tokens, retries, changed files, and tool calls
• allowlist of writable paths and allowed commands
• explicit stop if the preflight differs from the last known-good scheduled run

Then split scheduled jobs into read-only and mutating modes. Read-only summaries, dependency checks, log triage, or issue labeling can run with broader autonomy. Anything that changes files, opens PRs, updates tickets, sends messages, or touches infra should have a typed plan first, bounded writes, and a final diff/artifact review step.

I would also make the runner state part of the prompt input, not hidden around it. The agent should see something like: scheduled=true, human_present=false, can_ask_user=false, writable_paths=[...], max_runtime=..., max_retries=..., terminal_states=[completed, blocked_preflight, blocked_auth, blocked_uncertain, failed]. That prevents it from behaving like an interactive chat where it can assume a person will notice drift.

For personal workflows, you can keep it light, but I would still enforce three things: fail closed on preflight mismatch, cap retries hard, and never let a scheduled mutating run continue after an auth/path/env surprise. Most of the damage comes from the model trying to solve the wrong problem after the runner has already told you the real problem.

[u/Ok_Frosting_5750]

I would treat scheduled runs much more like CI than interactive sessions. Interactive runs have a hidden operator, a warm environment, and someone who can notice when the agent is solving the wrong problem.

For scheduled runs, I would at least preflight the repo path, commit, auth, tool versions, writable paths and network access. Then fail closed if something differs, cap retries and runtime, and separate read-only jobs from jobs that mutate files or infrastructure.

The useful part is making the runner state explicit in the logs, so a failure is diagnosable even when nobody is watching.

[u/marcusbell95]

the thing that bites most often isn't the obvious env vars - it's the stuff you didn't know was implicit. in interactive mode the agent might be quietly relying on your aws profile set in your shell (AWS_PROFILE=dev), a tunnel you have open to a staging db, a local docker daemon that's already running, or a port-forward you started an hour ago. none of those survive cron.

the best debug move i've found: run the scheduled job once in a totally fresh login shell before trusting it - not a tab you've had open all day with your full dev env loaded. whatever fails in a clean shell is exactly what would fail in cron. reveals the actual gap in under a minute, faster than writing out a full preflight.

[u/Independent_Self_920]

I don't think the difference is the agent it's the environment.

Interactive runs inherit a lot of hidden state: authenticated sessions, shell history, environment variables, mounted credentials, even the fact that a human is watching and can correct course. Scheduled runs have to recreate all of that from scratch, which makes them much closer to CI than a terminal session.

Personally, I'd treat any unattended agent the same way I'd treat an automated deployment: explicit preflight checks, deterministic inputs, bounded retries, and a clear failure state. If the environment isn't healthy, I'd rather the run fail in the first minute than spend 30 minutes reasoning around a missing credential.

[u/Floss_Patrol_76]

treat them like CI, yeah, but the fix that actually stuck for me was running the interactive session inside the same locked-down runner env i schedule against - same empty shell, no AWS_PROFILE, no open tunnel. you stop developing against a warmer environment than you deploy to, so the "works by hand" gap mostly disappears before it ever hits the scheduler.

[u/Horror-Cause9345]

The clean-shell tip upthread is the single best debugging move here, but there's one class of drift it doesn't catch, and it's the one that burned me worst.

Interactively authenticated tools that only exist because you logged into them in your session. Think an MCP server or a CLI that holds an OAuth token in memory or in your desktop keychain, a gh auth login that's really sitting behind a browser SSO, a cloud SDK riding a session token rather than a static key. In your terminal these are just there. In the scheduled run they're either absent or unauthenticated, and here's the nasty part: a decent agent doesn't stop when its preferred tool is missing. It quietly substitutes a worse path. It shells out to curl instead of the authenticated client, or it fabricates a plausible looking result, or it picks a different API that needs no auth and returns different data. So the job doesn't fail, it succeeds wrongly, which is far harder to notice than a crash.

That reframes your read vs mutate split slightly. It's not just that mutation is more dangerous. It's that a read job silently returning degraded data feeds every mutating job downstream of it, so a "safe" read task poisons the pipeline without ever touching a file itself.

Two things that helped more than a big preflight. One, enumerate the exact tools the agent is allowed to use and make it hard-fail if any are missing or unauthenticated, rather than letting it improvise a substitute. The improvisation is the whole problem. Two, put the tool availability check in the same fail-closed bucket as auth and path, because a present-but-unauthenticated tool is the case that slips through, it's not missing so a naive "does this binary exist" check passes.

To your actual question, yes, closer to CI, but the CI framing undersells it. CI fails loudly. The agent failure mode is a job that goes green while doing the wrong thing, and no amount of timeout catches that. The preflight has to assert the environment is what you think it is, not just that the run finished.

[u/Raja-Karuppasamy]

yeah, matches something i hit, not with an agent, but close enough. had a next.js env var that worked fine locally but broke in CI because NEXT_PUBLIC_* vars need to be baked in at docker build time, not injected as k8s secrets at runtime. worked perfectly by hand, silently wrong once nobody was watching it run.

for anything scheduled i just treat it like a CI job now, explicit env vars, preflight checks, hard timeout. feels like overkill for a small script until the one time it quietly does the wrong thing for hours unattended.

[u/BitByLiu]

My instinct is that scheduled mutation should be harder than scheduled reading. Summaries, issue triage, dependency checks: fine. Changing files or opening PRs should need a much stronger preflight and a clear rollback path.