Reports for humans and CI
Every run writes a Markdown summary, a JSON file for pipelines, and JUnit XML when you ask for it. Exit codes distinguish fail from error.
Reports & output →semtest/Overview
semtest · v0.2.0
Tests you
write in
plain language.
Describe what should be true about your codebase in a Markdown file. semtest hands it to the LLM CLI you already have on your PATH, and reports pass, fail or error for every scenario.
Setup1fileOne semtest.config.ts picks your model. Drop *.spec.md files anywhere and they get discovered.
$brew install westopp/semtest/semtestRequires Node 20+ · One LLM CLI on PATH
Specs, not assertions
*.spec.md · *.test.md · frontmatterA spec is a Markdown file stating what should hold — no test runner API to learn.
One file can carry several scenarios; each gets its own verdict.
Frontmatter overrides the model, timeout and tags per file.
Verify what unit tests cannot
Accessibility, copy quality, translation fidelity, absence of hardcoded secrets — properties that were always policy but never enforceable.
The what and why →Built for pipelines
Bail early, repeat flaky specs, filter by pattern, and skip permission prompts on CI runners that support it.
CI integration →First semtest in 5 minutes
From semtest init to a passing spec and a report you can read.