> ## Documentation Index
> Fetch the complete documentation index at: https://glasskit.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.
# GlassKit Eval
> Turn recorded video into repeatable evaluations for smart-glasses and camera-based AI apps.
GlassKit Eval turns recorded video into repeatable evals for apps that make decisions from images. Label the moments that matter, connect your app through a language-agnostic adapter, and rerun the same checks locally or in CI.
The eval loop is not tied to glasses or a particular model provider. It works for robotics, camera automation, video-analysis pipelines, multimodal model features, and other systems that can evaluate a sampled frame and return a JSON-like observation.
Use GlassKit Eval through the `glasskit eval` command group. This is its user manual; for contributor implementation notes, see [AGENTS.md](https://github.com/RealComputer/GlassKit/blob/main/cli/AGENTS.md).
## Why use this?
Vision-based apps often turn camera input into a structured decision: whether a workflow step is complete, which objects are present, what state a scene is in, or what action should happen next. Recreating those scenes by hand for every prompt, model, or app logic change is slow and makes regressions difficult to reproduce.
With `glasskit eval`, you provide a recording, label expected outputs at selected moments, and replay the same checks whenever the app changes. The adapter boundary lets the eval exercise existing application logic regardless of its implementation language, while quality gates turn the results into a useful local or CI signal.
GlassKit Eval is a good fit when your behavior can be tested from sampled video frames and expressed as JSON-like outputs. It is intentionally frame-oriented; apps that require continuous video, audio, or other sensor streams may need an adapter that reconstructs that context.
## How it works
Every eval command is a view of one pipeline:
```text theme={null}
case file ─▶ sample schedule ─▶ decoded frames ─▶ adapter ─▶ observations
│
exit code ◀─ quality gates ◀─ report ◀─ compare vs expect ◀┘
```
1. A case file names a video and declares samples — single `at` timestamps or `range` blocks expanded every `every_s` seconds. Labeled samples include an expected JSON-like value; drafts omit `expect`, and ignored samples may omit it.
2. `glasskit eval run` checks the eval structure, videos, and sample times, expands the schedule, and decodes the frame nearest each scheduled timestamp.
3. Each frame goes to your adapter, which runs your app's logic and returns a JSON-like observation.
4. The CLI extracts the configured `field` from the observation, if any, and compares the value against the sample's `expect` using the sample's comparison settings, recording a pass or fail.
5. Completed runs are printed as tables and optionally written as a JSON report, and any configured quality gates turn them into the run's exit code — the CI signal.
The labeling commands work on the same pipeline: `seed` sends draft samples through the adapter and writes the results back as proposed `expect` values instead of comparing them, and `review` opens a browser UI for checking and editing labels against the video.