Status: [LIVE] Endpoints found: https://sonarcloud.io/api/project_badges/measure?project=juusolatva_metrics_proxy&metric=alert_status)](https://sonarcloud.io/summary/new_code?id=juusolatva_metrics_proxy https://sonarcloud.io/api/project_badges/measure?project=juusolatva_metrics_proxy&metric=coverage)](https://sonarcloud.io/summary/new_code?id=juusolatva_metrics_proxy Source: https://github.com/juusolatva/metrics_proxy # metrics proxy [](https://github.com/juusolatva/metrics_proxy/actions/workflows/ci.yml) [](https://sonarcloud.io/summary/new_code?id=juusolatva_metrics_proxy) [](https://sonarcloud.io/summary/new_code?id=juusolatva_metrics_proxy) [](go.mod) [](LICENSE) A lightweight HTTP reverse proxy that sits between an LLM client and a local LLM server and prints inference metrics for every query: model, token counts, tokens per second, time to first token and duration, plus running aggregates. Supported upstreams: - **LM Studio**: native REST API (`/api/v1/...`) and OpenAI-compatible API (`/v1/...`) - **Ollama**: native API (`/api/chat`, `/api/generate`) and OpenAI-compatible API (`/v1/...`) - Other OpenAI-compatible HTTP servers Both regular JSON responses and streamed responses (SSE and NDJSON) are supported. Streams are passed through to the client as they arrive and are never buffered. ## Requirements - Go 1.21 or newer ## Usage ```bash go build -o metrics_proxy . ./metrics_proxy ``` By default the proxy reads `config.yaml` from the current working directory. Use `--config` to load another file, e.g. one per upstream server: ```bash ./metrics_proxy --config ~/metrics_proxy/ollama.yaml ``` Run `./metrics_proxy --help` to list all options. The default port was 8080 before; it moved to 11436 to stay clear of the many programs using 8080 (llama.cpp's `llama-server` among them) and of Ollama's 11434 and a second Ollama instance's 11435. Set `proxy_port: 8080` to keep the old po