Skip to content

How your workflow code executes

Your workflow is plain TypeScript, but different parts of it run at three distinct moments, on three different machines. Knowing which part runs where is the difference between a workflow that behaves and one that surprises you. This page is the map.

PhaseWhere it runsWhat runsWhen
CompileYour dev machine or CI (kici compile)Load your workflow modules, validate the DAG, assign step IDs, emit kici.lock.jsonBefore anything is pushed
OrchestratorYour orchestrator (no repo clone)Match triggers against the lock and dispatch jobs; it never evaluates workflow codeOn each incoming event
AgentAn ephemeral agent (fresh clone per job)Load the workflow module, evaluate job and step rules, run step bodies and hooks, run dynamic-value init steps and dynamicJob generators (both forms)After dispatch

The lock file is the seam. Everything left of it is decided once at compile time and frozen into JSON; everything right of it reads that JSON. See the lock file and workflow drift and the three-tier architecture for the wider picture.

kici compile loads your .kici/workflows/*.ts, validates dependencies (no cycles, no missing needs), assigns compile-time step IDs (unnamed steps become step-1, step-2, …), and writes kici.lock.json.

The compiler runs your module’s top-level code to build the workflow object — but that execution’s side effects and in-memory state do not travel. Only the resulting workflow structure lands in the lock. Anything your top-level code computes that isn’t part of the returned workflow object doesn’t exist past this point.

See compile the workflow for the command in context.

The lock is portable JSON. It carries:

  • Workflow and trigger metadata.
  • The job and step DAG, with compile-time step IDs.
  • Static values, verbatim.
  • Markers noting which fields are dynamic, so the orchestrator knows to resolve them on the agent’s init step.

It does not carry:

  • Your module’s runtime state or module-level variables.
  • Closures over those variables.
  • Live instances of modules you imported.
  • Anything computed at top level that isn’t part of the returned workflow object.

The consequence is blunt: if a value isn’t in the lock, the orchestrator can’t see it — it has no copy of your repository.

On each event the orchestrator matches triggers using only the lock — it never clones your repository and never evaluates workflow code. Dynamic context, env, and concurrencyGroup functions are not run here: the orchestrator dispatches a short init step to an agent to resolve them (see below).

Trigger matching can query the contents of individual source files, not just their paths: a pr(), push(), or tag() trigger with a requires filter is matched by reading the named files at the event’s commit and evaluating the filter as declarative data — still with no repository clone and no workflow code executed. A requires regex is checked for catastrophic (ReDoS) shapes at kici compile time and rejected there, so only safe patterns reach the orchestrator.

The orchestrator also does not run dynamicJob generator bodies itself: for the event-only (function) form it dispatches a dedicated dynamic-evaluation job to an agent at event time; the generator function then runs agent-side (see below).

A workflow-level filter predicate is the same shape: the orchestrator sees only the lock’s hasFilter flag, never the predicate, so it dispatches an evaluation job and lets an agent decide whether the workflow applies. Prefer the declarative filters where they answer the question — commitMessage on the trigger and requires over source files cost no evaluation job at all.

See dynamic values for how dynamic context, env, and concurrencyGroup functions resolve.

After dispatch, each job runs in its own ephemeral agent sandbox: a shallow clone at the dispatch ref (or a source-tarball extract for non-build jobs), then the workflow module is loaded fresh — TypeScript is transformed on import. On the agent, in order:

  1. Job-level rules are evaluated. By this point the agent has already spawned and the source has already been restored, so a job that its rules skip has still paid for that spawn and clone; only its steps are avoided.
  2. Step-level rules, then each step’s run() body and its hooks.
  3. Dynamic values (context, env, concurrencyGroup functions) are resolved here, via a short __init__ job that runs the function before the real job runs; this shows in the run timeline as an Init: entry.
  4. dynamicJob generators run here — both forms. The event-only (function) form runs in a dedicated evaluation job dispatched at event time; the result-aware (options) form is deferred until its declared needs complete, then run with the upstream outputs frozen as ctx.needs.
  5. A workflow-level filter predicate runs here too, before the jobs it gates. A global workflow evaluates it once per (event × workflow repo), before any run row exists, so a false verdict leaves no run at all. A same-repo workflow evaluates it once per job that reaches dispatch and once per job generator, after the run row exists, so a false verdict leaves a run whose only entries are the evaluation jobs. Keep the predicate cheap, pure, and side-effect free — a ten-job workflow calls it ten times for one event. See narrowing with a filter.

See job execution and hooks and rules for the details.

How a reused agent stays clean between jobs

Section titled “How a reused agent stays clean between jobs”

An ephemeral agent is discarded after one job, so residue never matters. A reused agent — a long-lived process on a shared host (the bare-metal profile) — serves many jobs in turn. Between them, the agent runs a supervisor-owned cleanup phase so one job’s leftovers never reach the next:

  1. It reaps the finished job’s process tree. A step may background a daemon that outlives the job. The agent runs each job’s process in its own process group and signals the whole group when the job ends, so a stray daemon does not survive into the next job. Set KICI_AGENT_ORPHAN_CLEANUP=false to keep only the runner and leave a backgrounded process alive on purpose.
  2. It re-runs declared cleanup after a hard kill. A job’s cleanup / onFailure hooks normally run in the job process. If that process is killed hard (out of memory, forced stop), the agent re-runs the declared cleanup against the preserved work directory.
  3. It deletes the work directory.
  4. It runs an optional operator reset command. Set KICI_AGENT_BETWEEN_JOBS_RESET_COMMAND to a host-reset command (for example, pruning a container cache). It runs after the reap and work-directory deletion. A failure never fails the finished job.

This phase is the primary cross-job cleanup. The agent’s startup temp-directory sweep stays as a backstop for anything a between-jobs phase missed. See agent configuration for the full env-var reference.

ConstructRuns onWhen
Static valueCompile → lockNever re-evaluated
Dynamic valueAgent init stepPer event
Job-level rulesAgentAfter clone
Step-level rulesAgentPer step
dynamicJob (function form)Agent (eval job)Dispatched at event time
dynamicJob (options form)AgentDeferred until needs complete
Workflow filter predicateAgent (eval job)Per event (global) / per job + generator (same-repo)
Step / job body + hooksAgentPer job

Determinism note. ctx.event and ctx.needs are frozen snapshots — captured once and replayed unchanged on any re-evaluation. A generator that derives its output from them is stable across re-evaluations; one that reads the wall clock (Date.now()) or a random source (Math.random()) is not.

step(...).result and job(...).result return an OutputProxy — a lazy proxy that, at the type level, mirrors the shape of the step or job’s declared outputs so that reading result.foo is type-checked, and at runtime defers each property read to a shared outputs map populated as the run progresses.

import { workflow, job, step, z } from '@kici-dev/sdk';
const build = job('build', {
runsOn: 'default',
steps: [
step('compile', {
outputs: { artifact: z.string() },
run: async () => ({ artifact: 'app.tar.gz' }),
}),
step('publish', {
// `compile.result.artifact` is typed from the `outputs` schema above.
run: async ({ steps }) => {
await Promise.resolve(steps.compile.result.artifact);
},
}),
],
});
export default workflow('build-and-publish', { jobs: [build] });

Outputs are typed across the job boundary too: reading jobRef.result.… or ctx.jobOutputs(jobRef) on a job reference threads the upstream job’s inferred output shape through — from any job, in a step body or a run: shorthand — so a typo on an output field or a renamed step is a compile error. Typed ctx.needs.jobRef.result.… additionally works in a run: shorthand job (where the run function’s ctx derives from the enclosing job’s needs tuple). Name your steps and use the options form (step('name', { run })) to give a job a typed output shape, and pass references rather than string names — string-form needs stay loosely typed. See output chaining for the authoring rules.

SymptomWhyFix
A top-level let seen = 0 (or a cache filled in job A) is empty in job BEach job loads the workflow module fresh in its own agent process after its own clone — there is no shared memory between jobsPass data through step/job outputs (OutputProxy / needs), not module variables
Fan-out job identities shift between re-evaluationsctx.event / ctx.needs are frozen and replayed, but Date.now() / Math.random() are notDerive job identity only from the frozen event/needs snapshot
A rule-skipped job still spawned an agent and clonedJob-level rules evaluate agent-side, after dispatch and clone — not on the orchestratorThis is by design: rules can read true runtime context ($, changedFiles, env). See step-level rules