Show HN：AI 編碼助手的統一本地可觀測性

Hacker News·4 個月前

一個名為 AI Observer 的新開源專案，提供了一個自託管、兼容 OpenTelemetry 的後端，用於統一監控本地 AI 編碼助手的可觀測性，追蹤代幣使用量、成本和效能。

Search code, repositories, users, issues, pull requests...

Provide feedback

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly

To see all available qualifiers, see our documentation.

Unified local observability for AI coding assistants

License

Uh oh!

There was an error while loading. Please reload this page.

tobilg/ai-observer

Folders and files

Latest commit

History

Repository files navigation

AI Observer

Unified local observability for AI coding assistants

AI Observer is a self-hosted, single-binary, OpenTelemetry-compatible observability backend designed specifically for monitoring local AI coding tools like Claude Code, Gemini CLI, and OpenAI Codex CLI.

Track token usage, costs, API latency, error rates, and session activity across all your AI coding assistants in one unified dashboard—with real-time updates and zero external dependencies.

Why AI Observer?

AI coding assistants are becoming essential development tools, but understanding their behavior and costs remains a challenge:

Features

Documentation

Screenshots

Dashboard

Metrics View

Logs View

Traces View

Quick Start

Using Docker (Recommended)

Dashboard: http://localhost:8080

Using a local directory for data persistence:

This stores the DuckDB database in your local ./ai-observer-data directory, making it easy to backup or inspect.

Using Homebrew (macOS Apple Silicon)

Using Binary

Download the latest release for your platform from Releases, then:

Building from Source

Configuration

Environment Variables

CORS and WebSocket origins allow AI_OBSERVER_FRONTEND_URL plus http://localhost:5173 and http://localhost:8080; set AI_OBSERVER_FRONTEND_URL when serving a custom UI origin.

CLI Options

Commands:

Global Options:

Examples:

Import Command

Import historical session data from local AI coding tool files into AI Observer.

File locations:

Override with environment variables: AI_OBSERVER_CLAUDE_PATH, AI_OBSERVER_CODEX_PATH, AI_OBSERVER_GEMINI_PATH

Examples:

See docs/import.md for detailed documentation and docs/pricing.md for pricing calculation details.

Export Command

Export telemetry data to portable Parquet files with an optional DuckDB views database.

Output files:

Examples:

See docs/export.md for detailed documentation.

Delete Command

Delete telemetry data from the database by time range.

Examples:

AI Tool Setup

Configure the following environment variables:

Add these to your ~/.bashrc, ~/.zshrc, or shell profile to persist across sessions.

Claude Code will then automatically send metrics and events to AI Observer.

Add to ~/.gemini/settings.json:

Required environment variables (workaround for Gemini CLI timing issues):

Note: Gemini CLI sends OTLP data to POST / instead of standard paths. AI Observer auto-detects the signal type and routes accordingly.

Add to ~/.codex/config.toml:

Note: Codex CLI exports logs and traces (no metrics). The trace_exporter option is undocumented but available—if omitted, traces are sent to the same endpoint as logs.

Architecture

Tech Stack:

API Reference

AI Observer exposes two HTTP servers:

OTLP Ingestion (Port 4318)

Standard OpenTelemetry Protocol endpoints for receiving telemetry data.

Query API (Port 8080)

REST API for querying stored telemetry data. Unless otherwise specified, from/to default to the last 24 hours.

Query parameters for /api/traces:

Query parameters for /api/metrics/series:

Batch series (POST /api/metrics/batch-series) request body:

Query parameters for /api/logs:

Data Collected

AI Observer receives standard OpenTelemetry data:

All data is stored locally in DuckDB. Nothing is sent to external services.

Telemetry Reference

Each AI coding tool exports different telemetry signals. Here's what you can observe:

Metrics

Common attributes: session.id, organization.id, user.account_uuid, terminal.type, model

Derived Metrics

AI Observer computes user-facing metrics that filter out tool-routing API calls (which have no cache tokens). These metrics match the token counts shown by tools like ccusage:

Note: Claude Code makes internal API calls for tool routing that don't involve user interaction. These calls have no cache tokens. The user-facing metrics exclude these calls to provide counts that match what users see in their billing and usage reports.

Events (Logs)

Metrics

Derived Metrics

AI Observer computes delta metrics from cumulative counters to show per-interval changes:

Logs

Codex CLI exports logs and traces directly. AI Observer derives metrics from these log events.

Derived Metrics

AI Observer computes these metrics from Codex CLI log events:

Events (Logs)

Note: codex.sse_event events are filtered out by AI Observer to reduce noise—these are emitted for every SSE streaming chunk from the API.

Traces

Codex CLI uses a single trace per session—all operations within a CLI session share the same trace ID with spans nested hierarchically:

This means long CLI sessions produce traces with thousands of spans spanning hours, rather than many short traces.

AI Observer Trace Handling: To improve usability, AI Observer treats each first-level child span (direct children of the session root) as a separate "virtual trace" in the dashboard. This splits long sessions into manageable units. However, since spans may arrive out of order, you may briefly see intermediate states where a span appears as its own trace before its parent arrives—once the parent span is received, the child automatically merges into the parent's trace on the next query refresh.

Understanding Token Metrics: OTLP vs Local Files

When comparing token usage from AI Observer's OTLP ingestion with tools like ccusage that parse local session files, you may notice significant differences in reported values. This is expected behavior due to different counting semantics.

Example Comparison

Here's a real comparison from a single day of Claude Code usage:

Why This Happens

The discrepancy is most pronounced for input and output tokens:

Claude Code OTLP metrics appear to report tokens differently than the API response's usage object that gets written to JSONL files.

Local JSONL files store the exact usage.input_tokens and usage.output_tokens values from Claude's API response, which ccusage reads directly.

Cache tokens (creation and read) are much closer between the two sources, suggesting these are counted consistently.

Token Type Comparison

Which Data Source Should I Use?

Reconciling the Data

If you need ccusage-compatible metrics in AI Observer:

Imported data uses the same token counting as ccusage and will show matching values.

Technical Details

Development

Developer quickstart

Prerequisites

Commands

Project Structure

CI/CD

GitHub Actions automatically:

Creating a Release

Troubleshooting

Change the ports using environment variables:

Set the AI_OBSERVER_FRONTEND_URL environment variable to match your frontend origin: