Testing guide

Test your workflows remotely against the full CI pipeline from your local machine. kici run remote uploads your current repo state (including uncommitted changes), triggers the pipeline, and streams execution logs back in real time.

Overview

KiCI gives you four ways to exercise a workflow before it runs in real CI, ordered from fastest / least faithful to slowest / most faithful:

• kici preview <event> — a pure dry-run that shows which workflows and jobs a trigger event would match. No steps execute. Use it to check trigger and rule logic. See kici preview.
• Unit-test a step function — call your step’s function directly in vitest with a fabricated context from @kici-dev/sdk/testing. No orchestrator, no agent — just your step logic. See Unit-testing step functions below.
• kici run <event> --local — executes the whole workflow on your own machine, which becomes an ephemeral agent. A faithful end-to-end run without a remote round-trip. See Local execution as an alternative.
• kici run remote — uploads your working tree and runs the full pipeline (orchestrator + agent) remotely, streaming logs back. The most faithful check.

The rest of this guide covers kici run remote.

kici run remote connects your local development environment to the remote orchestrator/agent pipeline. Instead of pushing a commit and waiting for CI, you can:

• Run any workflow against your current working tree (including unstaged changes)
• Get real-time log output streamed back to your terminal
• Give test runs test-scoped secrets — your local secret files and --env values (uploaded encrypted) plus any environment flagged allowLocalExecution: true — while production environments stay unreachable
• Detect test mode in workflow code via ctx.isTestRun

The command is remote-only — all execution happens on the orchestrator and agent. For local-only trigger matching previews, use kici preview <event>.

Orchestrator prerequisite: cache storage

kici run remote uploads your working-tree overlay to the orchestrator’s cache storage via a pre-signed URL, and the agent fetches it from there (see Repo state transfer). The target orchestrator must therefore have cache storage enabled (KICI_STORAGE_TYPE = s3 or filesystem).

• Both quickstarts wire this up for you — the Docker / Podman quickstart and the bare-metal quickstart each ship a SeaweedFS object store and pre-fill the orchestrator’s KICI_STORAGE_* block, so kici run remote works out of the box (see each guide’s “run a workflow without pushing” step).
• A hand-rolled orchestrator deploy does not configure storage by default — enable a backend before using kici run remote:
  • filesystem — simplest for a single-host orchestrator: set KICI_STORAGE_TYPE=filesystem and KICI_STORAGE_FS_PATH=/var/lib/kici/cache. No external service needed; blobs are served through the orchestrator’s own HMAC-signed HTTP route.
  • s3 — any S3-compatible bucket. A non-public / self-hosted endpoint works: set KICI_STORAGE_TYPE=s3, KICI_STORAGE_BUCKET, KICI_STORAGE_ENDPOINT=https://your-endpoint and (for most self-hosted services) KICI_STORAGE_FORCE_PATH_STYLE=true. If the developer machine running kici run remote reaches the bucket at a different address than the orchestrator, set KICI_STORAGE_UPLOAD_ENDPOINT to the developer-reachable address; if agents reach it at yet another address (e.g. agents in containers), set KICI_STORAGE_EXTERNAL_ENDPOINT to the agent-routable URL.

See Storage layout for the full env-var reference.

Unit-testing step functions

A KiCI step is just a TypeScript function that takes a context and does work — so you can test it in isolation with vitest, no orchestrator or agent required. @kici-dev/sdk/testing builds a real step context for you: ctx.$ runs real shell commands, ctx.secrets resolves values you seed, and ctx.emit(...) is recorded so you can assert on it.

Extract your step body into a named function so both your workflow and your test can call it:

steps/deploy.ts

import type { StepContext } from '@kici-dev/sdk';

export async function deployStep(ctx: StepContext): Promise<void> {
  const branch = (await ctx.$`git rev-parse --abbrev-ref HEAD`).stdout.trim();
  const token = await ctx.secrets.get('DEPLOY_TOKEN');
  ctx.setEnv('DEPLOYED_BRANCH', branch);
  await ctx.emit('deploy-complete', { branch });
  ctx.log.info(`deployed ${branch} with a ${token.length}-char token`);
}

steps/deploy.test.ts

import { describe, it, expect, afterEach } from 'vitest';
import { createTestStepContext, type TestStepContext } from '@kici-dev/sdk/testing';
import { deployStep } from './deploy.js';

describe('deployStep', () => {
  let harness: TestStepContext | undefined;
  afterEach(async () => {
    await harness?.dispose();
    harness = undefined;
  });

  it('records the deploy event and sets the output env var', async () => {
    harness = createTestStepContext({
      repoRoot: process.cwd(),
      secrets: { flat: { DEPLOY_TOKEN: 'test-token' } },
    });

    await deployStep(harness.ctx);

    expect(harness.ctx.env.DEPLOYED_BRANCH).toBeDefined();
    expect(harness.emitCalls).toHaveLength(1);
    expect(harness.emitCalls[0].eventName).toBe('deploy-complete');
  });
});

createTestStepContext(options?) takes zero required arguments. Common options:

Option	Purpose
repoRoot	Directory ctx.$ runs in. Defaults to the current working directory.
secrets	Seed ctx.secrets: { flat: { KEY: 'value' } }, or { contexts: { prod: { … } } }.
inputs	Typed ctx.inputs from upstream needs.
matrix	ctx.matrix values for a matrix job instance.
$	Inject a fake shell for pure-logic tests that must not actually run commands.
log	Replace the default console logger (e.g. a spy).

Every other StepContext member (cache, kici, artifacts, attestProvenance, …) has a safe default and can be overridden the same way. The returned handle exposes ctx (pass to your step), emitCalls (assert emitted events), and dispose() (call in afterEach to clean up seeded secret state and restore any process.env variables the step set via setEnv / addPath).

Orchestrator-backed APIs — ctx.kici.*, ctx.artifacts.*, ctx.attestProvenance(...) — reject by default (there is no orchestrator in a unit test); override them with a stub if your step calls them.

Getting started

1. Authenticate

kici login

This opens your browser for OAuth authentication and stores a personal access token in ~/.kici/config. For CI/CD pipelines or headless environments, use kici login --token <your-api-key> or kici login --device instead. See CLI authentication for details.

2. Write a test fixture

Fixtures define the events you want to simulate. They live in .kici/tests/*.ts and use the same SDK trigger functions as workflows.

.kici/tests/push-tests.ts

import { fixture, push } from '@kici-dev/sdk';

export const pushMain = fixture('push-main', {
  event: push({ branches: ['main'] }),
});

export const pushDevelop = fixture('push-develop', {
  event: push({ branches: ['develop'] }),
});

Each file can export multiple fixtures. The fixture() factory takes an ID (used on the command line) and options including the event to simulate.

3. Run a fixture

# List available fixtures
kici run remote

# Run a specific fixture
kici run remote push-main

# Run all fixtures matching a glob
kici run remote 'push-*'

# Run everything
kici run remote --all

The single quotes keep your shell from expanding push-* against local files, so the pattern reaches KiCI intact for its own fixture-glob matching.

Fixture reference

Event types

Fixtures accept any SDK trigger function as their event:

import { fixture, push, pr, comment, tag, release } from '@kici-dev/sdk';

// Push event
export const pushMain = fixture('push-main', {
  event: push({ branches: ['main'] }),
});

// PR event
export const prOpen = fixture('pr-open', {
  event: pr({ branches: ['main'], actions: ['opened'] }),
});

// Comment event
export const prComment = fixture('pr-comment', {
  event: comment({ actions: ['created'] }),
});

// Tag event
export const tagRelease = fixture('tag-release', {
  event: tag({ tags: ['v*'] }),
});

// Release event
export const published = fixture('release-published', {
  event: release({ actions: ['published'] }),
});

Overrides

Override default payload values per fixture:

export const pushFeature = fixture('push-feature', {
  event: push({ branches: ['feature/*'] }),
  branch: 'feature/auth', // Override branch name
  sha: 'abc123def456', // Override commit SHA
  repo: 'myorg/myrepo', // Override repository
  pr: 42, // Override PR number (for PR events)
});

When not specified, these default to values detected from your local git repo (current branch, HEAD SHA, remote URL).

Secret context mappings

Map secret contexts to your fixture:

export const pushWithSecrets = fixture('push-with-secrets', {
  event: push({ branches: ['main'] }),
  secrets: {
    db: 'test-database',
    api: 'test-api-keys',
  },
});

This maps the db secret context to the test-database context, and api to test-api-keys.

This mapping is honored by both kici run <event> --local and kici run remote:

• For a local kici run <event> --local (see kici run <event> --local), each named context is resolved from your local secret files (.kici/.secrets, .env.local, secrets.yaml, and --env flags).
• For kici run remote, each named context maps to an orchestrator context, and the orchestrator resolves that context’s secrets for the run. The target context must be flagged allowLocalExecution: true — mapping a context to a missing or non-test context rejects the run (see Secret contexts for testing below).

A fixture secrets: mapping is fail-closed; a job’s bound context: is not. The reject above applies only to the fixture secrets: mapping — an explicit request for that context’s secrets. A job’s own bound context: (job('deploy', { context: 'production', … })) is treated differently on a test run: if it resolves to a non-test or unconfigured context it is skipped with a warning, not rejected, so a job that deploys to production in real runs stays locally testable for its non-secret logic. kici run remote prints a warning naming the skipped context(s), and the dashboard run view shows the same notice. See Skip-on-test in the contexts guide.

Async fixtures

For dynamic fixture configuration, export an async function:

export const dynamicFixture = fixture('dynamic', async () => ({
  event: push({ branches: ['main'] }),
  sha: await getCurrentSha(),
}));

Running tests

Basic commands

# List all available fixtures (discovers .kici/tests/*.ts)
kici run remote

# Run a single fixture by ID
kici run remote push-main

# Glob matching -- run all push-related fixtures
kici run remote 'push-*'

# Run all fixtures sequentially
kici run remote --all

# Run all fixtures in parallel
kici run remote --all --parallel

Direct workflow run

Bypass trigger matching and run a specific workflow directly:

kici run remote --workflow ci

This skips the trigger evaluation step and runs all jobs in the named workflow.

Output modes

# Default: full log streaming with colored job prefixes
kici run remote push-main

# Quiet: minimal output (just pass/fail result)
kici run remote push-main --quiet

# JSON: machine-readable structured output
kici run remote push-main --json

# JUnit XML: for CI integration
kici run remote push-main --junit results.xml

Non-blocking execution

# Fire and forget -- returns immediately with run ID
kici run remote push-main --no-wait

# Check status later
kici runs show <run-id>

Cancellation

Press Ctrl+C during a running test to send a cancel signal to the orchestrator. The agent job will be terminated gracefully.

Repo state transfer

When you run kici run remote, the CLI:

1. Detects all files differing from HEAD (staged, unstaged, and untracked)
2. Creates a compressed tarball of changed files
3. Encrypts the tarball using X25519 ECDH key exchange
4. Uploads the encrypted tarball to storage via a signed URL
5. Triggers the pipeline with a reference to the upload

The agent clones your repo at HEAD, then applies the overlay tarball on top — giving you the exact same file state as your local working tree.

What gets included

• Modified tracked files (staged and unstaged)
• New untracked files (not in .gitignore)
• File deletions (tracked files you deleted locally)

What gets excluded

• Files matching .gitignore patterns
• Files matching .kiciignore patterns (additional exclusions)
• The .git directory itself

.kiciignore

Create a .kiciignore file in your repo root to exclude additional files from the upload:

# Large binaries
*.bin
*.iso
data/fixtures/large-dataset.csv

# Local-only configs
.env.local
docker-compose.override.yml

The format is the same as .gitignore — one glob pattern per line, # for comments.

Size limits

Threshold	Behavior
< 50 MB	Normal upload
50-500 MB	Warning displayed, upload proceeds
> 500 MB	Error — reduce bundle size via .kiciignore or check for unintended large files

The CLI always shows a pre-upload summary before transferring:

12 files changed, 3 new, 1 deleted (2.3 MB compressed)

Secret contexts for testing

The goal of the test-secret model is to let test runs reach test-only credentials while keeping production credentials out of reach. kici run remote combines two sources of secrets for a test run, then merges them with a clear precedence and a fail-closed gate.

CLI-uploaded local secrets

kici run remote collects the same local secret values that kici run <event> --local reads — .kici/.secrets, .kici/.env.local, .kici/secrets.yaml, and any --env KEY=VALUE flags — and uploads them encrypted to the orchestrator alongside the run. The orchestrator decrypts them only to inject them into the agent for that run; the control plane never sees the values.

# Provide an ad-hoc test value for a single remote run
kici run remote push-main --env KICI_DATABASE_URL=postgresql://localhost/test

--env provides a flat per-run override; --context <ctx>.<KEY>=<value> is its sibling for a namespaced per-run override, placing the value under the named context ctx. Both are uploaded encrypted and follow the same precedence rule below — a CLI-supplied value wins over the orchestrator test-context secret on a key collision.

# Provide a namespaced per-run value under the 'db' context
kici run remote push-db --context db.KICI_DATABASE_URL=postgresql://localhost/test

Because these values originate on your machine, they are the natural place to put throwaway test credentials without touching any orchestrator-stored secret.

Orchestrator test-context secrets

In addition to your uploaded values, the orchestrator resolves test-scoped secrets from its own store for a remote test run:

• The job’s own declared context contributes its resolved secrets (flat). Static strings and pure dynamic functions both participate: a pure context: function (see Dynamic values) is evaluated against the fixture’s simulated event, and the resolved name is gated and resolved like a static one. Impure dynamic functions (those requiring an init job) are not evaluated for test runs — use a fixture secrets: mapping (or --context) to supply such a job’s secrets.
• Each fixture secrets: { ctx: envName } mapping resolves the named context’s secrets under the namespaced context ctx.

Both paths are restricted to contexts flagged allowLocalExecution: true. A production context left at the default false is never resolvable for a test run.

export const pushWithDb = fixture('push-db', {
  event: push({ branches: ['main'] }),
  secrets: { db: 'test-database' }, // 'test-database' must be allowLocalExecution: true
});

step('migrate', async (ctx) => {
  const dbUrl = await ctx.secrets.get('KICI_DATABASE_URL');
  await ctx.$`npx prisma migrate deploy`;
});

Precedence: CLI values win

When a key exists in both sources, the CLI-uploaded local value wins over the orchestrator test-context value. This makes a local override a per-run knob: set --env KICI_DATABASE_URL=... (or put it in .kici/.secrets) to shadow the test context’s value for just that run, without changing anything on the orchestrator.

Fail-closed on non-test contexts

Test-run secret resolution is fail-closed:

• If a fixture maps to a context that does not exist, the run is rejected.
• If a fixture maps to a context whose allowLocalExecution is false, the run is rejected.
• The allowLocalExecution gate applies to all remote test runs: a run whose matched workflow targets a context with the flag off is rejected, so a test run can never resolve production secrets.

The allowLocalExecution context flag

Each context carries an allowLocalExecution flag (default false) that controls test-run access to that context and to its secrets. Production contexts should leave it at false; create a dedicated test context with allowLocalExecution: true that binds only test-only secret scopes for the jobs you want test runs to use.

The flag is set by the orchestrator operator, either via the CLI:

kici-admin context set-policy --env test-database --allow-local-execution true

or via the dashboard’s “Test runs” toggle on the context detail page. kici secrets list only surfaces contexts whose allowLocalExecution is true, so production contexts are never advertised as test-accessible.

Local execution as an alternative

kici run <event> --local resolves the same local secret files entirely on your machine and honors the fixture secrets: { ... } mapping to pick which local context backs each name (see kici run <event> --local). Because the values never leave your machine, it’s a good fit when you want to exercise secret-dependent steps without involving the orchestrator at all.

Discovering available contexts

# List test-accessible secret contexts and their key names (not values)
kici secrets list

Detecting test mode in workflows

Use ctx.isTestRun to conditionally skip destructive operations:

step('deploy', async (ctx) => {
  if (ctx.isTestRun) {
    ctx.log.info('Skipping deployment in test mode');
    return;
  }
  await ctx.$`kubectl apply -f k8s/`;
});

Run history

Viewing history

# Show recent test runs (from local history)
kici run remote --history

Run details

# Show run summary (reads the Platform, falls back to local history)
kici runs show <run-id>

# Show full logs
kici runs logs <run-id>

# Show logs for a specific job
kici runs logs <run-id> --job build

# Machine-readable output
kici runs show <run-id> --json

Scaffolding with kici init

Running kici init in a new project scaffolds a sample test fixture alongside the workflow templates:

.kici/
  workflows/
    hello-world.ts     # Sample workflow
    pr-checks.ts       # Sample PR workflow
  tests/
    push-test.ts       # Sample push fixture
  package.json
  tsconfig.json
.kiciignore            # Default exclusion patterns

The generated fixture uses the detected default branch:

.kici/tests/push-test.ts

import { fixture, push } from '@kici-dev/sdk';

export const pushMain = fixture('push-main', {
  event: push({ branches: ['main'] }),
});

See also

• CLI reference — complete command reference for all kici commands
• SDK reference — trigger functions, step context, and workflow API
• Workflow patterns — example workflows to test against