[Schneewolf-Labs/Merlina] Merlina API Documentation
Universal
API Leak/Open
35,731 characters
# Merlina API Documentation
**Version:** 1.1.0
**Base URL:** `http://localhost:8000` (configurable via `.env`)
## Table of Contents
1. [Overview](#overview)
2. [Authentication](#authentication)
3. [Base URL & Configuration](#base-url--configuration)
4. [Common Concepts](#common-concepts)
5. [API Endpoints](#api-endpoints)
- [General](#general)
- [Training Job Management](#training-job-management)
- [Queue Management](#queue-management)
- [GPU Management](#gpu-management)
- [Dataset Management](#dataset-management)
- [Configuration Management](#configuration-management)
- [Model Management](#model-management)
6. [WebSocket API](#websocket-api)
7. [Request/Response Schemas](#requestresponse-schemas)
8. [Error Handling](#error-handling)
9. [Examples](#examples)
---
## Overview
The Merlina API provides a comprehensive interface for training language models using ORPO (Odds Ratio Preference Optimization) with LoRA adapters. The API supports:
- **Training Job Management:** Create, monitor, and control training jobs
- **Job Queue:** Priority-based job queue with configurable concurrency
- **Real-time Updates:** WebSocket connections for live training progress
- **Dataset Management:** Load from HuggingFace, local files, or upload custom datasets
- **GPU Management:** Monitor and select GPUs for training
- **Configuration Management:** Save and reuse training configurations
- **Persistent Storage:** All jobs persisted in SQLite database
### Interactive API Documentation
FastAPI provides interactive API documentation:
- **Swagger UI:** `/docs` - Interactive API explorer
- **ReDoc:** `/redoc` - Alternative documentation view
- **OpenAPI spec:** `/openapi.json` - Machine-readable OpenAPI 3.x specification
The legacy URLs `/api/docs` and `/api/redoc` redirect to `/docs` and `/redoc`.
---
## Authentication
**Current Version:** No authentication required (v1.1.0)
For production deployments, it is recommended to:
- Use reverse proxy with authentication (nginx, Caddy, etc.)
- Restrict `CORS_ORIGINS` in `.env` file
- Implement API key authentication if needed
---
## Base URL & Configuration
The API base URL is configurable via environment variables in `.env`:
```bash
HOST=0.0.0.0 # Server bind address
PORT=8000 # Server port
DOMAIN= # Public domain (e.g., https://merlina.example.com)
```
**Default Base URL:** `http://localhost:8000`
---
## Common Concepts
### Job States
Jobs progress through the following states:
| State | Description |
|-------|-------------|
| `queued` | Job waiting in queue for execution |
| `initializing` | Setting up training environment |
| `loading_model` | Loading base model and tokenizer |
| `loading_dataset` | Loading and formatting dataset |
| `training` | Active training in progress |
| `saving` | Saving trained model |
| `uploading` | Uploading to HuggingFace Hub |
| `completed` | Training finished successfully |
| `failed` | Training failed with error |
| `stopped` | Job stopped by user request |
| `cancelled` | Job cancelled while in queue |
### Job Priority
Jobs can be submitted with priority levels:
- `low` - Runs after normal and high priority jobs
- `normal` (default) - Standard priority
- `high` - Runs before normal and low priority jobs
### Dataset Sources
Three types of dataset sources are supported:
1. **HuggingFace Hub:** Load datasets from huggingface.co
2. **Local Files:** Load from local JSON/JSONL/CSV/Parquet files
3. **Uploaded Files:** Upload files via API
### Dataset Formats
Supported chat template formats:
- `chatml` - ChatML format (default)
- `llama3` - Llama 3 format
- `mistral` - Mistral format
- `qwen3` - Qwen 3 format (supports thinking mode)
- `tokenizer` - Uses model's native chat template
- `custom` - User-defined custom templates
---
## API Endpoints
### General
#### Get API Information
```http
GET /api
```
Returns basic API information and available endpoints.
**Response:**
```json
{
"name": "Merlina",
"version": "1.0.0",
"description": "Magical Model Training Backend",
"endpoints": {
"POST /train": "Start a new training job",
"GET /status/{job_id}": "Get job status",
"GET /jobs": "List all jobs",
"GET /docs": "Interactive API documentation (Swagger UI)",
"GET /redoc": "API documentation (ReDoc)",
"GET /openapi.json": "OpenAPI specification"
}
}
```
---
#### Get System Statistics
```http
GET /stats
```
Returns database, WebSocket, and queue statistics.
**Response:**
```json
{
"database": {
"total_jobs": 42,
"total_metrics": 15234,
"db_size_mb": 2.5
},
"websockets": {
"total_connections": 3
},
"queue": {
"queued_count": 2,
"running_count": 1,
"max_concurrent": 1,
"available_slots": 0
}
}
```
---
#### List HuggingFace Namespaces
```http
POST /hf/namespaces
```
Lists the namespaces a HuggingFace token can publish to — the token owner's
own account plus every organization they belong to. Use it to pick
`hf_namespace` before training or uploading; a bare `output_name` always
resolves to the personal account, which is what makes org uploads 404.
**Request Body:**
```json
{
"hf_token": "hf_..."
}
```
`hf_token` is optional when `HF_TOKEN` is set in the server's `.env`.
**Response:**
```json
{
"user": "nbeerbower",
"namespaces": [
{"name": "nbeerbower", "type": "user", "full_name": "", "role": "owner", "can_write": true},
{"name": "Schneewolf-Labs", "type": "org", "full_name": "Schneewolf Labs", "role": "admin", "can_write": true}
]
}
```
**Errors:**
- `400` — no token available, or the Hub rejected it
- `502` — the Hub could not be reached
---
### Training Job Management
#### Validate Training Configuration
```http
POST /validate
```
Validates training configuration before starting. Checks GPU availability, VRAM, disk space, model access, and more.
**Request Body:**
```json
{
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"output_name": "my-llama-ft",
"lora_r": 64,
"lora_alpha": 32,
"learning_rate": 5e-6,
"num_epochs": 2,
"batch_size": 1,
"gradient_accumulation_steps": 16,
"max_length": 2048,
"max_prompt_length": 1024,
"beta": 0.1,
"use_4bit": true,
"dataset": {
"source": {
"source_type": "huggingface",
"repo_id": "schneewolflabs/Athanorlite-DPO",
"split": "train"
},
"format": {
"format_type": "chatml"
}
}
}
```
**Response:**
```json
{
"valid": true,
"results": {
"errors": [],
"warnings": [
"Flash Attention 2 not available, falling back to eager attention"
],
"checks": {
"gpu_available": true,
"vram_sufficient": true,
"disk_space_ok": true,
"model_accessible": true,
"dataset_valid": true
}
}
}
```
**Status Codes:**
- `200 OK` - Validation completed
- `400 Bad Request` - Invalid configuration
---
#### Estimate Training VRAM
```http
POST /estimate/vram
```
Architecture-aware VRAM estimate for a training configuration. Reads the model's real `config.json` (local directory or HuggingFace Hub, cache-aware) and returns a per-component breakdown. Does not require a GPU.
**Request Body:** Same as `/validate` (see [TrainingConfig Schema](#trainingconfig))
**Response:**
```json
{
"available": true,
"total_gb": 15.42,
"breakdown_gb": {
"model_weights": 5.4,
"trainable_params": 0.62,
"gradients": 0.62,
"optimizer_states": 0.31,
"activations": 3.05,
"logits_and_loss": 4.92,
"cuda_context": 0.9,
"fragmentation_buffer": 1.1
},
"confidence": "high",
"notes": [
"ORPO runs chosen+rejected together, doubling the effective forward batch to 2."
],
"model_profile": {
"num_params_billions": 8.03,
"hidden_size": 4096,
"num_hidden_layers": 32,
"vocab_size": 128256,
"source": "hub_config",
"params_exact": true
},
"gpu_total_gb": 24.0
}
```
When the model architecture cannot be determined, `available` is `false` and a `reason` string explains why.
`confidence` is `high` (exact parameter count from safetensors metadata), `medium` (parameters derived from config.json dims), or `low` (architecture guessed from the model name).
**Status Codes:**
- `200 OK` - Estimate completed (check `available`)
- `400 Bad Request` - Invalid configuration
- `504 Gateway Timeout` - Estimation timed out
---
#### Create Training Job
```http
POST /train?priority=normal
```
Creates and queues a new training job with automatic validation.
**Query Parameters:**
- `priority` (optional) - Job priority: `low`, `normal` (default), `high`
**Request Body:** Same as `/validate` (see [TrainingConfig Schema](#trainingconfig))
**Response:**
```json
{
"job_id": "job_20250116_143022",
"status": "queued",
"message": "Training spell cast! Job job_20250116_143022 queued at position 1."
}
```
**Status Codes:**
- `200 OK` - Job created successfully
- `400 Bad Request` - Validation failed or invalid configuration
**Example with cURL:**
```bash
curl -X POST "http://localhost:8000/train?priority=high" \
-H "Content-Type: application/json" \
-d @training_config.json
```
---
#### Get Job Status
```http
GET /status/{job_id}
```
Returns current status and progress of a training job.
**Path Parameters:**
- `job_id` - Job identifier (e.g., `job_20250116_143022`)
**Response:**
```json
{
"job_id": "job_20250116_143022",
"status": "training",
"progress": 0.45,
"current_step": 234,
"total_steps": 520,
"loss": 0.234,
"error": null,
"wandb_url": "https://wandb.ai/user/project/runs/abc123",
"queue_position": null,
"queue_state": "running"
}
```
**Status Codes:**
- `200 OK` - Job found
- `404 Not Found` - Job not found
---
#### List All Jobs
```http
GET /jobs
```
Returns a summary of all jobs.
**Response:**
```json
{
"job_20250116_143022": {
"status": "training",
"progress": 0.45
},
"job_20250116_120000": {
"status": "completed",
"progress": 1.0
}
}
```
---
#### Get Job History
```http
GET /jobs/history?status=completed&limit=50&offset=0
```
Returns paginated job history with detailed information. Jobs persist across server restarts.
**Query Parameters:**
- `status` (optional) - Filter by status (e.g., `completed`, `failed`, `training`)
- `limit` (optional, default: 50) - Maximum number of jobs to return
- `offset` (optional, default: 0) - Number of jobs to skip
**Response:**
```json
{
"jobs": [
{
"job_id": "job_20250116_143022",
"status": "completed",
"progress": 1.0,
"created_at": "2025-01-16T14:30:22",
"updated_at": "2025-01-16T15:45:10",
"output_dir": "./models/my-llama-ft",
"error": null
}
],
"count": 1
}
```
---
#### Get Job Metrics
```http
GET /jobs/{job_id}/metrics
```
Returns detailed time-series training metrics for a job.
**Path Parameters:**
- `job_id` - Job identifier
**Response:**
```json
{
"job_id": "job_20250116_143022",
"metrics": [
{
"step": 1,
"loss": 0.845,
"eval_loss": 0.923,
"learning_rate": 5e-6,
"timestamp": "2025-01-16T14:35:00"
},
{
"step": 10,
"loss": 0.654,
"eval_loss": 0.712,
"learning_rate": 4.8e-6,
"timestamp": "2025-01-16T14:37:00"
}
],
"count": 520
}
```
**Status Codes:**
- `200 OK` - Job found
- `404 Not Found` - Job not found
---
#### Stop/Cancel Job
```http
POST /jobs/{job_id}/stop
```
Cancels a queued job or requests graceful stop of a running job.
**Path Parameters:**
- `job_id` - Job identifier
**Response (Queued Job):**
```json
{
"status": "success",
"message": "Job job_20250116_143022 removed from queue",
"job_id": "job_20250116_143022",
"was_queued": true
}
```
**Response (Running Job):**
```json
{
"status": "success",
"message": "Stop request sent to job job_20250116_143022. Training will stop after current step.",
"job_id": "job_20250116_143022",
"was_queued": false
}
```
**Status Codes:**
- `200 OK` - Stop request processed
- `404 Not Found` - Job not found
---
#### Delete Job
```http
DELETE /jobs/{job_id}
```
Deletes a specific job and all associated metrics from the database.
If the job was still waiting in the queue, its queue entry is dropped too, so a
deleted job is never executed later. A running job is not stopped by this
endpoint — use `POST /jobs/{job_id}/stop` first.
**Path Parameters:**
- `job_id` - Job identifier
**Response:**
```json
{
"status": "success",
"message": "Job job_20250116_143022 deleted successfully",
"job_id": "job_20250116_143022",
"removed_from_queue": true
}
```
`removed_from_queue` is `true` when the job was queued and its queue entry was
removed, `false` when it wasn't waiting in the queue.
**Status Codes:**
- `200 OK` - Job deleted
- `404 Not Found` - Job not found
---
#### Clear All Jobs
```http
DELETE /jobs
```
Deletes all jobs and metrics from the database, and clears every queued job
from the queue (running jobs are left alone).
**Response:**
```json
{
"status": "success",
"message": "Cleared all jobs (42 jobs deleted)",
"deleted_count": 42,
"removed_from_queue": 3
}
```
`removed_from_queue` is the number of queued entries that were dropped.
---
### Queue Management
#### Get Queue Status
```http
GET /queue/status
```
Returns overall queue statistics and lists of queued and running jobs.
**Response:**
```json
{
"stats": {
"queued_count": 2,
"running_count": 1,
"max_concurrent": 1,
"available_slots": 0
},
"queued_jobs": [
{
"job_id": "job_20250116_150000",
"priority": 5,
"position": 1
},
{
"job_id": "job_20250116_151000",
"priority": 3,
"position": 2
}
],
"running_jobs": [
{
"job_id": "job_20250116_143022",
"priority": 5
}
]
}
```
---
#### List Queue Jobs
```http
GET /queue/jobs
```
Returns detailed information about queued and running jobs.
**Response:**
```json
{
"queued": [
{
"job_id": "job_20250116_150000",
"priority": 5,
"position": 1
}
],
"running": [
{
"job_id": "job_20250116_143022",
"priority": 5
}
]
}
```
---
### GPU Management
#### List All GPUs
```http
GET /gpu/list
```
Lists all available GPUs with detailed information.
**Response:**
```json
{
"status": "success",
"gpus": [
{
"index": 0,
"name": "NVIDIA GeForce RTX 4090",
"memory_total_mb": 24564,
"memory_free_mb": 23120,
"memory_used_mb": 1444,
"utilization_percent": 5,
"temperature_c": 42,
"power_usage_w": 45.2,
"compute_capability": "8.9"
}
],
"count": 1
}
```
**Response (No CUDA):**
```json
{
"status": "no_cuda",
"message": "CUDA is not available on this system",
"gpus": []
}
```
---
#### Get GPU Information
```http
GET /gpu/{index}
```
Returns detailed information about a specific GPU.
**Path Parameters:**
- `index` - GPU index (0-based)
**Response:**
```json
{
"status": "success",
"gpu": {
"index": 0,
"name": "NVIDIA GeForce RTX 4090",
"memory_total_mb": 24564,
"memory_free_mb": 23120,
"memory_used_mb": 1444,
"utilization_percent": 5,
"temperature_c": 42,
"power_usage_w": 45.2,
"compute_capability": "8.9"
}
}
```
**Status Codes:**
- `200 OK` - GPU found
- `404 Not Found` - GPU not found or CUDA unavailable
---
#### Get Available GPUs
```http
GET /gpu/available?min_free_memory_mb=4000
```
Returns list of GPU indices with sufficient free memory.
**Query Parameters:**
- `min_free_memory_mb` (optional, default: 4000) - Minimum free memory in MB
**Response:**
```json
{
"status": "success",
"available_gpus": [0, 1],
"count": 2,
"min_free_memory_mb": 4000
}
```
---
#### Get Recommended GPU
```http
GET /gpu/recommended
```
Returns the GPU with the most free memory.
**Response:**
```json
{
"status": "success",
"recommended_index": 0,
"gpu": {
"index": 0,
"name": "NVIDIA GeForce RTX 4090",
"memory_total_mb": 24564,
"memory_free_mb": 23120,
"memory_used_mb": 1444,
"utilization_percent": 5,
"temperature_c": 42,
"power_usage_w": 45.2,
"compute_capability": "8.9"
}
}
```
**Status Codes:**
- `200 OK` - GPU found
- `404 Not Found` - No GPUs available
---
### Dataset Management
#### Preview Dataset (Raw)
```http
POST /dataset/preview
```
Previews raw dataset without formatting (up to 10 samples).
**Request Body:**
```json
{
"source": {
"source_type": "huggingface",
"repo_id": "schneewolflabs/Athanorlite-DPO",
"split": "train"
},
"format": {
"format_type": "chatml"
}
}
```
**Response:**
```json
{
"status": "success",
"samples": [
{
"prompt": "What is the capital of France?",
"chosen": "The capital of France is Paris.",
"rejected": "The capital of France is London.",
"system": "You are a helpful assistant."
}
],
"num_samples": 10
}
```
**Status Codes:**
- `200 OK` - Preview successful
- `400 Bad Request` - Invalid dataset configuration
---
#### Preview Dataset (Formatted)
```http
POST /dataset/preview-formatted
```
Previews dataset with formatting applied (up to 5 samples).
**Request Body:** Same as `/dataset/preview` (see [DatasetConfig Schema](#datasetconfig))
**Response:**
```json
{
"status": "success",
"samples": [
{
"prompt": "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n",
"chosen": "The capital of France is Paris.<|im_end|>",
"rejected": "The capital of France is London.<|im_end|>"
}
],
"num_samples": 5
}
```
---
#### Get Dataset Columns
```http
POST /dataset/columns
```
Returns column names and sample data for column mapping.
**Request Body:**
```json
{
"source": {
"source_type": "huggingface",
"repo_id": "schneewolflabs/Athanorlite-DPO",
"split": "train"
}
}
```
**Response:**
```json
{
"status": "success",
"columns": ["system", "prompt", "chosen", "rejected"],
"samples": [
{
"system": "You are a helpful assistant.",
"prompt": "What is the capital of France?",
"chosen": "The capital of France is Paris.",
"rejected": "The capital of France is London."
}
],
"total_rows": 10000
}
```
---
#### Upload Dataset File
```http
POST /dataset/upload-file
```
Uploads a dataset file and returns a dataset ID for later use.
**Request:** Multipart form data with file
**Response:**
```json
{
"status": "success",
"dataset_id": "upload_20250116_143022_a1b2c3d4",
"filename": "my_dataset.jsonl",
"size": 1048576
}
```
**Example with cURL:**
```bash
curl -X POST "http://localhost:8000/dataset/upload-file" \
-F "file=@my_dataset.jsonl"
```
**Status Codes:**
- `200 OK` - Upload successful
- `400 Bad Request` - Upload failed
---
#### List Uploaded Datasets
```http
GET /dataset/uploads
```
Lists all uploaded datasets.
**Response:**
```json
{
"datasets": [
{
"dataset_id": "upload_20250116_143022_a1b2c3d4",
"filename": "my_dataset.jsonl",
"size": 1048576
}
]
}
```
---
### Configuration Management
#### Save Configuration
```http
POST /configs/save
```
Saves a training configuration for later reuse.
**Request Body:**
```json
{
"name": "llama3-8b-default",
"config": {
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"lora_r": 64,
"lora_alpha": 32,
"learning_rate": 5e-6
},
"description": "Default configuration for Llama 3 8B fine-tuning",
"tags": ["llama3", "8b", "default"]
}
```
**Response:**
```json
{
"status": "success",
"message": "Configuration 'llama3-8b-default' saved successfully",
"filepath": "/path/to/data/configs/llama3-8b-default.json"
}
```
**Status Codes:**
- `200 OK` - Configuration saved
- `400 Bad Request` - Invalid configuration
---
#### List Configurations
```http
GET /configs/list?tag=llama3
```
Lists all saved configurations with optional tag filtering.
**Query Parameters:**
- `tag` (optional) - Filter by tag
**Response:**
```json
{
"status": "success",
"configs": [
{
"name": "llama3-8b-default",
"description": "Default configuration for Llama 3 8B fine-tuning",
"tags": ["llama3", "8b", "default"],
"created_at": "2025-01-16T14:30:22",
"updated_at": "2025-01-16T14:30:22"
}
],
"count": 1
}
```
---
#### Get Configuration
```http
GET /configs/{name}?include_metadata=false
```
Loads a saved configuration.
**Path Parameters:**
- `name` - Configuration name
**Query Parameters:**
- `include_metadata` (optional, default: false) - Include metadata in response
**Response (without metadata):**
```json
{
"status": "success",
"config": {
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"lora_r": 64,
"lora_alpha": 32,
"learning_rate": 5e-6
}
}
```
**Response (with metadata):**
```json
{
"status": "success",
"config": {
"name": "llama3-8b-default",
"description": "Default configuration for Llama 3 8B fine-tuning",
"tags": ["llama3", "8b", "default"],
"created_at": "2025-01-16T14:30:22",
"updated_at": "2025-01-16T14:30:22",
"config": {
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"lora_r": 64,
"lora_alpha": 32,
"learning_rate": 5e-6
}
}
}
```
**Status Codes:**
- `200 OK` - Configuration found
- `404 Not Found` - Configuration not found
---
#### Delete Configuration
```http
DELETE /configs/{name}
```
Deletes a saved configuration.
**Path Parameters:**
- `name` - Configuration name
**Response:**
```json
{
"status": "success",
"message": "Configuration 'llama3-8b-default' deleted successfully"
}
```
**Status Codes:**
- `200 OK` - Configuration deleted
- `404 Not Found` - Configuration not found
---
#### Export Configuration
```http
POST /configs/export
```
Exports a configuration to a specific file path.
**Request Body:**
```json
{
"name": "llama3-8b-default",
"output_path": "/path/to/export/config.json"
}
```
**Response:**
```json
{
"status": "success",
"message": "Configuration exported to /path/to/export/config.json",
"filepath": "/path/to/export/config.json"
}
```
**Status Codes:**
- `200 OK` - Configuration exported
- `404 Not Found` - Configuration not found
---
#### Import Configuration
```http
POST /configs/import
```
Imports a configuration from an external file.
**Request Body:**
```json
{
"filepath": "/path/to/import/config.json",
"name": "imported-config"
}
```
**Response:**
```json
{
"status": "success",
"message": "Configuration imported successfully",
"filepath": "/path/to/data/configs/imported-config.json"
}
```
**Status Codes:**
- `200 OK` - Configuration imported
- `404 Not Found` - File not found
---
### Model Management
#### Preload Model Tokenizer
```http
POST /model/preload
```
Preloads a model's tokenizer for dataset preview. Downloads model files and caches the tokenizer.
**Request Body:**
```json
{
"model_name": "meta-llama/Meta-Llama-3-8B-Instruct",
"hf_token": "hf_..."
}
```
**Response:**
```json
{
"status": "success",
"model_name": "meta-llama/Meta-Llama-3-8B-Instruct",
"vocab_size": 128256,
"model_max_length": 8192,
"has_chat_template": true,
"chat_template_preview": "{% set loop_messages = messages %}...",
"pad_token": "<|end_of_text|>",
"eos_token": "<|end_of_text|>",
"bos_token": "<|begin_of_text|>"
}
```
**Status Codes:**
- `200 OK` - Tokenizer loaded
- `400 Bad Request` - Failed to load tokenizer
---
#### List Cached Models
```http
GET /model/cached
```
Lists currently cached model tokenizers.
**Response:**
```json
{
"cached_models": [
"meta-llama/Meta-Llama-3-8B-Instruct",
"mistralai/Mistral-7B-Instruct-v0.2"
],
"count": 2
}
```
---
## WebSocket API
### Connect to Job Updates
```
WebSocket /ws/{job_id}
```
Establishes a WebSocket connection for real-time training updates.
**Path Parameters:**
- `job_id` - Job identifier
**Connection:**
```javascript
const ws = new WebSocket(`ws://localhost:8000/ws/${job_id}`);
ws.onmessage = (event) => {
const data = JSON.parse(event.data);
console.log('Received:', data);
};
ws.onerror = (error) => {
console.error('WebSocket error:', error);
};
ws.onclose = () => {
console.log('WebSocket closed');
};
```
### Message Types
#### Status Update
Sent periodically during training with progress and metrics.
```json
{
"type": "status_update",
"job_id": "job_20250116_143022",
"status": "training",
"progress": 0.45,
"current_step": 234,
"total_steps": 520,
"loss": 0.234,
"eval_loss": 0.256,
"learning_rate": 4.8e-6,
"gpu_memory": 12.5,
"timestamp": 1705413022.123
}
```
#### Metrics Update
Sent at specific training steps with detailed metrics.
```json
{
"type": "metrics",
"job_id": "job_20250116_143022",
"step": 100,
"metrics": {
"loss": 0.234,
"eval_loss": 0.256,
"learning_rate": 4.8e-6,
"grad_norm": 0.15
},
"timestamp": 1705413022.123
}
```
#### Error
Sent when training fails.
```json
{
"type": "error",
"job_id": "job_20250116_143022",
"error": "CUDA out of memory",
"timestamp": 1705413022.123
}
```
#### Completion
Sent when training completes successfully.
```json
{
"type": "completed",
"job_id": "job_20250116_143022",
"output_dir": "./models/my-llama-ft",
"final_metrics": {
"final_loss": 0.123,
"final_eval_loss": 0.145
},
"timestamp": 1705413022.123
}
```
#### Heartbeat
Sent in response to client messages to maintain connection.
```json
{
"type": "heartbeat",
"status": "ok"
}
```
---
## Request/Response Schemas
### TrainingConfig
Complete training configuration schema.
```json
{
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"output_name": "my-llama-ft",
"use_lora": true,
"lora_r": 64,
"lora_alpha": 32,
"lora_dropout": 0.05,
"target_modules": ["up_proj", "down_proj", "gate_proj", "k_proj", "q_proj", "v_proj", "o_proj"],
"learning_rate": 5e-6,
"num_epochs": 2,
"batch_size": 1,
"gradient_accumulation_steps": 16,
"max_length": 2048,
"max_prompt_length": 1024,
"beta": 0.1,
"seed": 42,
"max_grad_norm": 0.3,
"warmup_ratio": 0.05,
"eval_steps": 0.2,
"save_steps": null,
"use_4bit": true,
"use_wandb": true,
"push_to_hub": false,
"merge_lora_before_upload": true,
"hf_hub_private": true,
"hf_namespace": "Schneewolf-Labs",
"hf_token": "hf_...",
"wandb_key": "...",
"shuffle_dataset": true,
"weight_decay": 0.01,
"lr_scheduler_type": "cosine",
"gradient_checkpointing": false,
"logging_steps": 1,
"optimizer_type": "paged_adamw_8bit",
"adam_beta1": 0.9,
"adam_beta2": 0.999,
"adam_epsilon": 1e-8,
"attn_implementation": "auto",
"gpu_ids": [0],
"wandb_project": "merlina-training",
"wandb_run_name": "llama3-ft-v1",
"wandb_tags": ["llama3", "orpo"],
"wandb_notes": "Fine-tuning Llama 3 with ORPO",
"dataset": {
"source": {
"source_type": "huggingface",
"repo_id": "schneewolflabs/Athanorlite-DPO",
"split": "train"
},
"format": {
"format_type": "chatml",
"custom_templates": null,
"enable_thinking": true
},
"model_name": null,
"column_mapping": null,
"test_size": 0.01,
"max_samples": null
}
}
```
### DatasetConfig
Dataset configuration schema.
```json
{
"source": {
"source_type": "huggingface",
"repo_id": "schneewolflabs/Athanorlite-DPO",
"split": "train",
"file_path": null,
"file_format": null,
"dataset_id": null
},
"format": {
"format_type": "chatml",
"custom_templates": null,
"enable_thinking": true
},
"model_name": null,
"column_mapping": null,
"test_size": 0.01,
"max_samples": null
}
```
**Dataset Source Types:**
1. **HuggingFace:**
```json
{
"source_type": "huggingface",
"repo_id": "schneewolflabs/Athanorlite-DPO",
"split": "train"
}
```
2. **Local File:**
```json
{
"source_type": "local_file",
"file_path": "/path/to/dataset.jsonl",
"file_format": "jsonl"
}
```
3. **Uploaded:**
```json
{
"source_type": "upload",
"dataset_id": "upload_20250116_143022_a1b2c3d4",
"file_format": "jsonl"
}
```
**Custom Format Example:**
```json
{
"format_type": "custom",
"custom_templates": {
"system_prefix": "[SYSTEM] ",
"system_suffix": "\n",
"user_prefix": "[USER] ",
"user_suffix": "\n",
"assistant_prefix": "[ASSISTANT] ",
"assistant_suffix": "\n"
}
}
```
---
## Error Handling
### Error Response Format
All errors follow a consistent format:
```json
{
"detail": "Error message or details object"
}
```
### HTTP Status Codes
| Code | Meaning | Description |
|------|---------|-------------|
| 200 | OK | Request successful |
| 400 | Bad Request | Invalid request parameters or validation failed |
| 404 | Not Found | Resource not found (job, config, GPU, etc.) |
| 500 | Internal Server Error | Server error occurred |
### Common Error Scenarios
#### Validation Failed (400)
```json
{
"detail": {
"message": "Training configuration validation failed",
"errors": [
"GPU not available",
"Insufficient VRAM (required: 16GB, available: 8GB)"
],
"warnings": [
"Flash Attention 2 not available"
]
}
}
```
#### Job Not Found (404)
```json
{
"detail": "Job not found"
}
```
#### Configuration Not Found (404)
```json
{
"detail": "Configuration 'my-config' not found"
}
```
#### GPU Not Available (404)
```json
{
"detail": "CUDA is not available"
}
```
---
## Examples
### Complete Training Workflow
```python
import requests
import time
BASE_URL = "http://localhost:8000"
# 1. Validate configuration
config = {
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"output_name": "my-llama-ft",
"lora_r": 64,
"lora_alpha": 32,
"learning_rate": 5e-6,
"num_epochs": 2,
"batch_size": 1,
"gradient_accumulation_steps": 16,
"max_length": 2048,
"max_prompt_length": 1024,
"beta": 0.1,
"use_4bit": True,
"dataset": {
"source": {
"source_type": "huggingface",
"repo_id": "schneewolflabs/Athanorlite-DPO",
"split": "train"
},
"format": {
"format_type": "chatml"
}
}
}
# Validate
response = requests.post(f"{BASE_URL}/validate", json=config)
validation = response.json()
if not validation["valid"]:
print("Validation failed:", validation["results"]["errors"])
exit(1)
print("Validation passed!")
# 2. Create training job
response = requests.post(f"{BASE_URL}/train?priority=high", json=config)
job = response.json()
job_id = job["job_id"]
print(f"Job created: {job_id}")
# 3. Monitor progress
while True:
response = requests.get(f"{BASE_URL}/status/{job_id}")
status = response.json()
print(f"Status: {status['status']}, Progress: {status['progress']:.2%}")
if status["status"] in ["completed", "failed", "stopped"]:
break
time.sleep(5)
# 4. Get final metrics
if status["status"] == "completed":
response = requests.get(f"{BASE_URL}/jobs/{job_id}/metrics")
metrics = response.json()
print(f"Training completed with {metrics['count']} metric points")
else:
print(f"Training {status['status']}: {status.get('error', 'Unknown')}")
```
### WebSocket Monitoring
```python
import asyncio
import websockets
import json
async def monitor_job(job_id):
uri = f"ws://localhost:8000/ws/{job_id}"
async with websockets.connect(uri) as websocket:
print(f"Connected to job {job_id}")
async for message in websocket:
data = json.loads(message)
if data["type"] == "status_update":
print(f"Progress: {data['progress']:.2%}, Loss: {data.get('loss', 'N/A')}")
elif data["type"] == "metrics":
print(f"Step {data['step']}: {data['metrics']}")
elif data["type"] == "completed":
print(f"Training completed! Model saved to: {data['output_dir']}")
break
elif data["type"] == "error":
print(f"Error: {data['error']}")
break
# Run
asyncio.run(monitor_job("job_20250116_143022"))
```
### Upload and Train with Custom Dataset
```python
import requests
BASE_URL = "http://localhost:8000"
# 1. Upload dataset file
with open("my_dataset.jsonl", "rb") as f:
files = {"file": f}
response = requests.post(f"{BASE_URL}/dataset/upload-file", files=files)
upload = response.json()
dataset_id = upload["dataset_id"]
print(f"Dataset uploaded: {dataset_id}")
# 2. Preview the dataset
preview_config = {
"source": {
"source_type": "upload",
"dataset_id": dataset_id,
"file_format": "jsonl"
},
"format": {
"format_type": "chatml"
}
}
response = requests.post(f"{BASE_URL}/dataset/preview", json=preview_config)
preview = response.json()
print(f"Dataset has {preview['num_samples']} samples")
# 3. Train with uploaded dataset
config = {
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"output_name": "custom-ft",
# ... other training parameters ...
"dataset": preview_config
}
response = requests.post(f"{BASE_URL}/train", json=config)
job = response.json()
print(f"Training started: {job['job_id']}")
```
### GPU Selection
```python
import requests
BASE_URL = "http://localhost:8000"
# Get recommended GPU
response = requests.get(f"{BASE_URL}/gpu/recommended")
recommended = response.json()
gpu_index = recommended["recommended_index"]
print(f"Using GPU {gpu_index}: {recommended['gpu']['name']}")
# Configure training to use specific GPU
config = {
# ... other parameters ...
"gpu_ids": [gpu_index]
}
response = requests.post(f"{BASE_URL}/train", json=config)
```
### Configuration Management
```python
import requests
BASE_URL = "http://localhost:8000"
# 1. Save configuration
save_request = {
"name": "llama3-default",
"config": {
"base_model": "meta-llama/Meta-Llama-3-8B-Instruct",
"lora_r": 64,
"lora_alpha": 32,
# ... other parameters ...
},
"description": "Default Llama 3 configuration",
"tags": ["llama3", "default"]
}
response = requests.post(f"{BASE_URL}/configs/save", json=save_request)
print(response.json())
# 2. List configurations
response = requests.get(f"{BASE_URL}/configs/list?tag=llama3")
configs = response.json()
print(f"Found {configs['count']} configurations")
# 3. Load configuration
response = requests.get(f"{BASE_URL}/configs/llama3-default")
config = response.json()["config"]
# 4. Use loaded configuration for training
response = requests.post(f"{BASE_URL}/train", json=config)
```
---
## Rate Limits & Quotas
**Current Version:** No rate limits implemented (v1.1.0)
For production deployments, consider implementing:
- Request rate limiting
- Concurrent job limits per user
- Dataset upload size limits (configurable via `MAX_UPLOAD_SIZE_MB` in `.env`)
---
## Changelog
### Version 1.1.0
- Added persistent job storage with SQLite
- Added WebSocket real-time updates
- Added pre-flight validation
- Added job queue with priority support
- Added GPU management endpoints
- Added configuration management
- Added model tokenizer preloading
### Version 1.0.0
- Initial API release
- Basic training job management
- Dataset loading and formatting
- HuggingFace Hub integration
---
## Support
For issues, questions, or feature requests:
- **GitHub:** https://github.com/Schneewolf-Labs/Merlina
- **Documentation:** See `CLAUDE.md` for developer documentation
- **API Docs:** Visit `/docs` for interactive documentation