Promptctl
Web|Live
Promptctl
The definitive CLI for Prompt Engineering as Code - develop, test, and monitor LLM prompts with software engineering rigor.
Overview
Promptctl is a CLI that brings software engineering best practices to LLM development. Define prompts in Markdown, write test cases in JSON, run bulk evaluations, and A/B test prompt versions with statistical confidence intervals. Includes a local React dashboard for visualizing runs, traces, and costs.
Specs
CoreCLI for prompt versioning, testing, and A/B evaluation
PromptsMarkdown files with YAML frontmatter, version controlled
TestingJSON test cases with exact match, regex, semantic similarity, JSON schema
StackTypeScript, Node.js, React, Vite, Express
ProvidersGoogle Gemini (1.5-3.0), OpenAI, Anthropic, local models
Features
- Prompts as Code - Markdown with YAML frontmatter
- Test-Driven Development with JSON test cases
- Bulk evaluation with pass/fail rates and latency metrics
- A/B testing with 95% confidence intervals (Bootstrapping)
- Local React dashboard for history, traces, and cost analysis
- Multi-provider support: Gemini, OpenAI, Anthropic, local models
Tech Stack
TypeScript
Node.js
React
Google GenAI