Testing guide

Test your workflows remotely against the full CI pipeline from your local machine. kici run remote uploads your current repo state (including uncommitted changes), triggers the pipeline, and streams execution logs back in real time.

Overview

KiCI gives you four ways to exercise a workflow before it runs in real CI, ordered from fastest / least faithful to slowest / most faithful:

• kici preview <event> — a pure dry-run that shows which workflows and jobs a trigger event would match. No steps execute. Use it to check trigger and rule logic. See kici preview.
• Unit-test a step function — call your step’s function directly in vitest with a fabricated context from @kici-dev/sdk/testing. No orchestrator, no agent — just your step logic. See Unit-testing step functions below.
• kici run <event> --local — executes the whole workflow on your own machine, which becomes an ephemeral agent. A faithful end-to-end run without a remote round-trip. See Local execution as an alternative.
• kici run remote — uploads your working tree and runs the full pipeline (orchestrator + agent) remotely, streaming logs back. The most faithful check.

The rest of this guide covers kici run remote.

kici run remote connects your local development environment to the remote orchestrator/agent pipeline. Instead of pushing a commit and waiting for CI, you can:

• Run any workflow against your current working tree (including unstaged changes)
• Get real-time log output streamed back to your terminal
• Give test runs test-scoped secrets — your local secret files and --env values (uploaded encrypted) plus any context flagged allowLocalExecution: true — while production contexts stay unreachable
• Detect test mode in workflow code via ctx.isTestRun

The command is remote-only — all execution happens on the orchestrator and agent. For local-only trigger matching previews, use kici preview <event>.

Orchestrator prerequisite: cache storage

kici run remote uploads your working-tree overlay to the orchestrator’s cache storage via a pre-signed URL, and the agent fetches it from there (see Repo state transfer). The target orchestrator must therefore have cache storage enabled (KICI_STORAGE_TYPE = s3 or filesystem).

• Both quickstarts wire this up for you — the Docker / Podman quickstart and the bare-metal quickstart each ship a SeaweedFS object store and pre-fill the orchestrator’s KICI_STORAGE_* block, so kici run remote works out of the box (see each guide’s “run a workflow without pushing” step).
• A hand-rolled orchestrator deploy does not configure storage by default — enable a backend before using kici run remote:
  • filesystem — simplest for a single-host orchestrator: set KICI_STORAGE_TYPE=filesystem and KICI_STORAGE_FS_PATH=/var/lib/kici/cache. No external service needed; blobs are served through the orchestrator’s own HMAC-signed HTTP route.
  • s3 — any S3-compatible bucket. A non-public / self-hosted endpoint works: set KICI_STORAGE_TYPE=s3, KICI_STORAGE_BUCKET, KICI_STORAGE_ENDPOINT=https://your-endpoint and (for most self-hosted services) KICI_STORAGE_FORCE_PATH_STYLE=true. If the developer machine running kici run remote reaches the bucket at a different address than the orchestrator, set KICI_STORAGE_UPLOAD_ENDPOINT to the developer-reachable address; if agents reach it at yet another address (e.g. agents in containers), set KICI_STORAGE_EXTERNAL_ENDPOINT to the agent-routable URL.

See Storage layout for the full env-var reference.

Unit-testing step functions

A KiCI step is just a TypeScript function that takes a context and does work — so you can test it in isolation with vitest, no orchestrator or agent required. @kici-dev/sdk/testing builds a real step context for you: ctx.$ runs real shell commands, ctx.secrets resolves values you seed, and ctx.emit(...) is recorded so you can assert on it.

Extract your step body into a named function so both your workflow and your test can call it:

steps/deploy.ts

import type { StepContext } from '@kici-dev/sdk';

export async function deployStep(ctx: StepContext): Promise<void> {
  const branch = (await ctx.$`git rev-parse --abbrev-ref HEAD`).stdout.trim();
  const token = await ctx.secrets.get('DEPLOY_TOKEN');
  ctx.setEnv('DEPLOYED_BRANCH', branch);
  await ctx.emit('deploy-complete', { branch });
  ctx.log.info(`deployed ${branch} with a ${token.length}-char token`);
}

steps/deploy.test.ts

import { describe, it, expect, afterEach } from 'vitest';
import { createTestStepContext, type TestStepContext } from '@kici-dev/sdk/testing';
import { deployStep } from './deploy.js';

describe('deployStep', () => {
  let harness: TestStepContext | undefined;
  afterEach(async () => {
    await harness?.dispose();
    harness = undefined;
  });

  it('records the deploy event and sets the output env var', async () => {
    harness = createTestStepContext({
      repoRoot: process.cwd(),
      secrets: { flat: { DEPLOY_TOKEN: 'test-token' } },
    });

    await deployStep(harness.ctx);

    expect(harness.ctx.env.DEPLOYED_BRANCH).toBeDefined();
    expect(harness.emitCalls).toHaveLength(1);
    expect(harness.emitCalls[0].eventName).toBe('deploy-complete');
  });
});

createTestStepContext(options?) takes zero required arguments. Common options:

Option	Purpose
repoRoot	Directory ctx.$ runs in. Defaults to the current working directory.
secrets	Seed ctx.secrets: { flat: { KEY: 'value' } }, or { contexts: { prod: { … } } }.
inputs	Typed ctx.inputs from upstream needs.
matrix	ctx.matrix values for a matrix job instance.
$	Inject a fake shell for pure-logic tests that must not actually run commands.
log	Replace the default console logger (e.g. a spy).

Every other StepContext member (cache, kici, artifacts, attestProvenance, …) has a safe default and can be overridden the same way. The returned handle exposes:

• ctx — pass it to your step.
• emitCalls — assert the emitted events.
• tempScope — the allocator behind ctx.mktemp / ctx.mktempFile. Call tempScope.disposeAll() to remove every live temp path early.
• dispose() — call it in afterEach. It cleans up seeded secret state and restores any process.env variables the step set via setEnv / addPath.

Orchestrator-backed APIs — ctx.kici.*, ctx.artifacts.*, ctx.attestProvenance(...) — reject by default (there is no orchestrator in a unit test); override them with a stub if your step calls them.

Getting started

1. Authenticate

kici login

This opens your browser for OAuth authentication and stores a personal access token in ~/.kici/config. For headless environments, use kici login --device instead. For CI/CD pipelines, see non-interactive environments. See CLI authentication for details.

No Platform account? Run against your own orchestrator directly. Ask its operator for a personal orchestrator token, then save the target:

kici connect https://ci.example.com

See Direct mode for what changes when a run goes straight to an orchestrator.

2. Write a test fixture

Fixtures define the events you want to simulate. They live in .kici/tests/*.ts and use the same SDK trigger functions as workflows.

.kici/tests/push-tests.ts

import { fixture, push } from '@kici-dev/sdk';

export const pushMain = fixture('push-main', {
  event: push({ branches: ['main'] }),
});

export const pushDevelop = fixture('push-develop', {
  event: push({ branches: ['develop'] }),
});

Each file can export multiple fixtures. The fixture() factory takes an ID (used on the command line) and options including the event to simulate.

3. Run a fixture

# List available fixtures
kici run remote

# Run a specific fixture
kici run remote push-main

# Run all fixtures matching a glob
kici run remote 'push-*'

# Run everything
kici run remote --all

The single quotes keep your shell from expanding push-* against local files, so the pattern reaches KiCI intact for its own fixture-glob matching.

Fixture reference

Event types

Fixtures accept any SDK trigger function as their event:

import { fixture, push, pr, comment, tag, release } from '@kici-dev/sdk';

// Push event
export const pushMain = fixture('push-main', {
  event: push({ branches: ['main'] }),
});

// PR event
export const prOpen = fixture('pr-open', {
  event: pr({ target: 'main', events: ['opened'] }),
});

// Comment event
export const prComment = fixture('pr-comment', {
  event: comment({ actions: ['created'] }),
});

// Tag event
export const tagRelease = fixture('tag-release', {
  event: tag({ patterns: ['v*'] }),
});

// Release event
export const published = fixture('release-published', {
  event: release({ actions: ['published'] }),
});

Overrides

Override default payload values per fixture:

export const pushFeature = fixture('push-feature', {
  event: push({ branches: ['feature/*'] }),
  branch: 'feature/auth', // Override branch name
  sha: 'abc123def456', // Override commit SHA
  repo: 'myorg/myrepo', // Override repository
  pr: 42, // Override PR number (for PR events)
});

When not specified, these default to values detected from your local git repo (current branch, HEAD SHA, remote URL).

Branch and commit

The trigger you pass as event only selects the event type. Its filters are the trigger-side shape, so they do not set the event’s branch. push({ branches: ['main'] }) in a fixture does not make the event a push to main.

The event gets its branch and commit as follows:

• Branch: the fixture’s branch when you set it, else the branch you have checked out. The step sees it as refs/heads/<branch> in the payload ref and as the branch name in ctx.repo.ref.
• Commit: the fixture’s sha when you set it, else your HEAD commit. The run records this commit as its SHA.
• PR events: branch is the base branch (default main), and the branch you have checked out is the head branch.
• Tag events: branch is the tag name (default v0.0.0).
• Detached HEAD: the event uses main (the head branch of a PR event uses feature/test), and kici run remote prints a warning. Set branch to choose one. (kici run <event> --local uses kici-local on a detached HEAD.)

So a fixture without branch follows your checkout: run it from feature/auth and the event is a push to feature/auth. A workflow that triggers on push({ branches: ['main'] }) then does not match. Set branch: 'main' on the fixture to test that workflow from any branch.

Secret context mappings

Map secret contexts to your fixture:

export const pushWithSecrets = fixture('push-with-secrets', {
  event: push({ branches: ['main'] }),
  secrets: {
    db: 'test-database',
    api: 'test-api-keys',
  },
});

This maps the db secret context to the test-database context, and api to test-api-keys.

This mapping is honored by both kici run <event> --local and kici run remote:

• For a local kici run <event> --local (see kici run <event> --local), each named context is resolved from your local secret files (.kici/.secrets, .env.local, secrets.yaml, and --env flags).
• For kici run remote, each named context maps to an orchestrator context, and the orchestrator resolves that context’s secrets for the run. The target context must be flagged allowLocalExecution: true — mapping a context to a missing or non-test context rejects the run (see Secret contexts for testing below).

A fixture secrets: mapping is fail-closed; a job’s bound context: is not. The reject above applies only to the fixture secrets: mapping — an explicit request for that context’s secrets. A job’s own bound context: (job('deploy', { context: 'production', … })) is treated differently on a test run: if it resolves to a non-test or unconfigured context it is skipped with a warning, not rejected. So a job that deploys to production in real runs stays locally testable for its non-secret logic. kici run remote prints a warning naming the skipped context(s), and the dashboard run view shows the same notice. See Skip-on-test in the contexts guide.

Async fixtures

For dynamic fixture configuration, export an async function:

export const dynamicFixture = fixture('dynamic', async () => ({
  event: push({ branches: ['main'] }),
  sha: await getCurrentSha(),
}));

Running tests

Basic commands

# List all available fixtures (discovers .kici/tests/*.ts)
kici run remote

# Run a single fixture by ID
kici run remote push-main

# Glob matching -- run all push-related fixtures
kici run remote 'push-*'

# Run all fixtures sequentially
kici run remote --all

# Run all fixtures in parallel
kici run remote --all --parallel

Direct workflow run

Bypass trigger matching and run a specific workflow directly:

kici run remote --workflow ci

This skips the trigger evaluation step and runs all jobs in the named workflow. The run carries the branch you have checked out and your HEAD commit.

Add a fixture to send that fixture’s event to the named workflow only:

kici run remote push-main --workflow ci

The workflow gets the fixture’s branch, SHA and payload, and no other workflow runs. A glob or --all runs each selected fixture against that one workflow. A workflow name that is not in the compiled lock file stops the run before upload, and the error lists the workflows that exist.

Output modes

# Default: full log streaming with colored job prefixes
kici run remote push-main

# Quiet: minimal output (just pass/fail result)
kici run remote push-main --quiet

# JSON: machine-readable structured output
kici run remote push-main --json

# JUnit XML: for CI integration
kici run remote push-main --junit results.xml

Non-blocking execution

# Fire and forget -- returns immediately with run ID
kici run remote push-main --no-wait

# Check status later
kici runs show <run-id>

Cancellation

Press Ctrl+C during a running test to send a cancel request to the orchestrator. The agent stops the job gracefully. The CLI waits up to 15 seconds for the orchestrator to answer, then prints Cancelled run <run-id>. or the reason the cancel failed. Press Ctrl+C a second time to exit without waiting.

Repo state transfer

When you run kici run remote, the CLI:

1. Collects your working tree: every tracked file, every untracked file that .gitignore does not exclude, and the .git directory
2. Creates a compressed tarball of those files
3. Encrypts the tarball using X25519 ECDH key exchange
4. Uploads the encrypted tarball to storage via a signed URL
5. Triggers the pipeline with a reference to the upload

The agent does not clone your repository. It extracts the tarball into an empty workspace, so the job runs on the files of your local working tree, uncommitted changes included. The .git directory ships too, so steps that run git commands see your local history and branch.

kici run <event> --local builds its isolated checkout differently: it clones your repository at HEAD and copies only your changed files on top.

What gets included

• Tracked files, with your staged and unstaged changes
• New untracked files (not in .gitignore)
• The .git directory
• Symbolic links. A link to a file ships with the file it points at, and the agent writes that file’s content in place of the link. A link to a directory ships as a link, and the agent recreates it. A link that points outside the repository, or at a file that .gitignore or .kiciignore excludes, fails the job with an error that names the link.
• Hard links. Each path ships as a separate file with its own content, also when it shares its data with other files. A node_modules directory that pnpm installs contains many hard links.

What gets excluded

• Files matching .gitignore patterns
• Files matching .kiciignore patterns (additional exclusions, never applied to the .git directory)
• Tracked files you deleted locally
• The files of submodules and of other nested git repositories. The CLI prints a warning that names each one.
• Symbolic links whose target does not exist. The CLI prints a warning that names each one.

.kiciignore

Create a .kiciignore file in your repo root to exclude additional files from the upload:

# Large binaries
*.bin
*.iso
data/fixtures/large-dataset.csv

# Local-only configs
.env.local
docker-compose.override.yml

The format is the same as .gitignore — one glob pattern per line, # for comments.

Size limits

Threshold	Behavior
< 50 MB	Normal upload
50-500 MB	Warning displayed, upload proceeds
> 500 MB	Error — reduce bundle size via .kiciignore or check for unintended large files

The CLI always shows a pre-upload summary before transferring:

12 files changed, 3 new, 1 deleted (2.3 MB compressed)

Secret contexts for testing

The goal of the test-secret model is to let test runs reach test-only credentials while keeping production credentials out of reach. kici run remote combines two sources of secrets for a test run, then merges them with a clear precedence and a fail-closed gate.

CLI-uploaded local secrets

kici run remote collects the same local secret values that kici run <event> --local reads — .kici/.secrets, .kici/.env.local, .kici/secrets.yaml, and any --env KEY=VALUE flags — and uploads them encrypted to the orchestrator alongside the run. The orchestrator decrypts them only to inject them into the agent for that run; the control plane never sees the values.

# Provide an ad-hoc test value for a single remote run
kici run remote push-main --env KICI_DATABASE_URL=postgresql://localhost/test

--env provides a flat per-run override; --context <ctx>.<KEY>=<value> is its sibling for a namespaced per-run override, placing the value under the named context ctx. Both are uploaded encrypted and follow the same precedence rule below — a CLI-supplied value wins over the orchestrator test-context secret on a key collision.

# Provide a namespaced per-run value under the 'db' context
kici run remote push-db --context db.KICI_DATABASE_URL=postgresql://localhost/test

Because these values originate on your machine, they are the natural place to put throwaway test credentials without touching any orchestrator-stored secret.

Orchestrator test-context secrets

In addition to your uploaded values, the orchestrator resolves test-scoped secrets from its own store for a remote test run:

• The job’s own declared context contributes its resolved secrets (flat). Static strings and pure dynamic functions both participate: a pure context: function (see Dynamic values) is evaluated against the fixture’s simulated event, and the resolved name is gated and resolved like a static one. Impure dynamic functions (those requiring an init job) are not evaluated for test runs — use a fixture secrets: mapping (or --context) to supply such a job’s secrets.
• Each fixture secrets: { ctx: envName } mapping resolves the named context’s secrets under the namespaced context ctx. envName is matched like a job’s declared context: a context with exactly that name, or else a glob context whose pattern matches it.

Both paths are restricted to contexts flagged allowLocalExecution: true. A production context left at the default false is never resolvable for a test run.

export const pushWithDb = fixture('push-db', {
  event: push({ branches: ['main'] }),
  secrets: { db: 'test-database' }, // 'test-database' must be allowLocalExecution: true
});

step('migrate', async (ctx) => {
  const dbUrl = await ctx.secrets.get('KICI_DATABASE_URL');
  await ctx.$`npx prisma migrate deploy`;
});

Precedence: CLI values win

When a key exists in both sources, the CLI-uploaded local value wins over the orchestrator test-context value. This makes a local override a per-run knob: set --env KICI_DATABASE_URL=... (or put it in .kici/.secrets) to shadow the test context’s value for just that run, without changing anything on the orchestrator.

The same overlay reaches a container job’s private-registry reference. The orchestrator reads tokenSecret: 'prod:REGISTRY_TOKEN' through the prod context’s secrets for the run, with your uploaded values on top. So --env REGISTRY_TOKEN=... or --context prod.REGISTRY_TOKEN=... replaces that registry token for the run. A flat value replaces the key in every context, so it shadows a reference to any context that names the same key. The named context must still exist, and its protection rules still run first. A gitCredentials reference is not affected: the agent requests it while the job runs, and the orchestrator reads it from its own secret store.

Fail-closed on non-test contexts

Test-run secret resolution is fail-closed:

• If a fixture maps to a name that matches no context, the run is rejected.
• If a fixture maps to a context whose allowLocalExecution is false, the run is rejected.
• The allowLocalExecution gate applies to all remote test runs. A context a job binds with the flag off is skipped for the run: its variables, secrets and protection rules do not apply, and the run shows a warning that names it. The run is not rejected, but it can never resolve that context’s secrets, so a test run never reaches production secrets.

The allowLocalExecution context flag

Each context carries an allowLocalExecution flag (default false) that controls test-run access to that context and to its secrets. Production contexts should leave it at false; create a dedicated test context with allowLocalExecution: true that binds only test-only secret scopes for the jobs you want test runs to use.

The flag is set by the orchestrator operator, either via the CLI:

kici-admin context set-policy --org <orgId> --env test-database --allow-local-execution true

or via the dashboard’s “Test runs” toggle on the context detail page. kici secrets list only surfaces contexts whose allowLocalExecution is true, so production contexts are never advertised as test-accessible.

Local execution as an alternative

kici run <event> --local resolves the same local secret files entirely on your machine and honors the fixture secrets: { ... } mapping to pick which local context backs each name (see kici run <event> --local). Because the values never leave your machine, it’s a good fit when you want to exercise secret-dependent steps without involving the orchestrator at all.

Discovering available contexts

# List test-accessible secret contexts and their key names (not values)
kici secrets list

Detecting test mode in workflows

Use ctx.isTestRun to conditionally skip destructive operations:

step('deploy', async (ctx) => {
  if (ctx.isTestRun) {
    ctx.log.info('Skipping deployment in test mode');
    return;
  }
  await ctx.$`kubectl apply -f k8s/`;
});

Run history

Viewing history

# Show recent test runs (from local history)
kici run remote --history

Run details

# Show run summary (reads the Platform, falls back to local history)
kici runs show <run-id>

# Show full logs
kici runs logs <run-id>

# Show logs for a specific job
kici runs logs <run-id> --job build

# Machine-readable output
kici runs show <run-id> --json

Scaffolding with kici init

Running kici init in a new project scaffolds a sample test fixture alongside the workflow templates:

.kici/
  workflows/
    hello-world.ts     # Sample workflow
    pr-checks.ts       # Sample PR workflow
  tests/
    push-test.ts       # Sample push fixture
  package.json
  tsconfig.json
.kiciignore            # Default exclusion patterns

The generated fixture uses the branch you have checked out. On a detached HEAD, it uses the default branch. On a main branch, the file contains this fixture:

.kici/tests/push-test.ts

import { fixture, push } from '@kici-dev/sdk';

export const pushMain = fixture("push-main", {
  event: push({ branches: ["main"] }),
  // The event's branch. Without it, the run uses the branch you have checked out.
  branch: "main",
});

The fixture sets branch, so kici run remote push-main simulates a push to main from any branch you have checked out.

The fixture ID is push- followed by the branch name. The export name is push followed by each word of the branch name, with a capital first letter. Characters other than ASCII letters and digits separate the words. On a ci/kici branch, the export is pushCiKici and the fixture ID is push-ci/kici.

See also

• CLI reference — complete command reference for all kici commands
• SDK reference — trigger functions, step context, and workflow API
• Workflow patterns — example workflows to test against