Huggingface Trackio

What This Skill Does

This skill provides an experiment tracking library for logging and visualizing ML training metrics, syncing to Hugging Face Spaces for real-time monitoring dashboards. It offers Python API for logging metrics and firing alerts, and CLI for retrieving metrics and alerts. It's designed for ML engineers to monitor and diagnose training runs.

When to Use

• Log metrics during training runs.
• Fire alerts for training diagnostics.
• Retrieve metrics and alerts after/during training.
• Monitor training progress in real-time on Hugging Face Spaces.
• Flag important events in training code.
• Enable autonomous experiment iteration.

Key Features

• Logs metrics using trackio.init(), trackio.log(), and trackio.finish().
• Fires alerts with severity levels using trackio.alert().
• Retrieves metrics and alerts using the CLI.
• Syncs metrics to Hugging Face Spaces for real-time dashboards.
• Integrates with TRL via report_to="trackio".
• Supports webhooks (Slack/Discord) for alert notifications.

Manual Installation

Three Interfaces

When to Use Each

Python API → Logging

Use import trackio in your training scripts to log metrics:

Initialize tracking with trackio.init() Log metrics with trackio.log() or use TRL's report_to="trackio" Finalize with trackio.finish()

Key concept: For remote/cloud training, pass space_id — metrics sync to a Space dashboard so they persist after the instance terminates.

→ See references/logging_metrics.md for setup, TRL integration, and configuration options.

Python API → Alerts

Insert trackio.alert() calls in training code to flag important events — like inserting print statements for debugging, but structured and queryable:

• trackio.alert(title="...", level=trackio.
• AlertLevel.
• WARN) — fire an alert. Three severity levels: INFO, WARN, ERROR Alerts are printed to terminal, stored in the database, shown in the dashboard, and optionally sent to webhooks (Slack/Discord)

Key concept for LLM agents: Alerts are the primary mechanism for autonomous experiment iteration. An agent should insert alerts into training code for diagnostic conditions (loss spikes, NaN gradients, low accuracy, training stalls). Since alerts are printed to the terminal, an agent that is watching the training script's output will see them automatically. For background or detached runs, the agent can poll via CLI instead.

→ See references/alerts.md for the full alerts API, webhook setup, and autonomous agent workflows.

CLI → Retrieving

Use the trackio command to query logged metrics and alerts:

trackio list projects/runs/metrics — discover what's available trackio get project/run/metric — retrieve summaries and values trackio list alerts --project --json — retrieve alerts trackio show — launch the dashboard trackio sync — sync to HF Space

Key concept: Add --json for programmatic output suitable for automation and LLM agents.

→ See references/retrieving_metrics.md for all commands, workflows, and JSON output formats.

Minimal Logging Setup

import trackio

trackio.init(project="my-project", space_id="username/trackio") trackio.log({"loss": 0.1, "accuracy": 0.9}) trackio.log({"loss": 0.09, "accuracy": 0.91}) trackio.finish()

Minimal Retrieval

trackio list projects --json trackio get metric --project my-project --run my-run --metric loss --json

Autonomous ML Experiment Workflow

When running experiments autonomously as an LLM agent, the recommended workflow is:

Set up training with alerts — insert trackio.alert() calls for diagnostic conditions Launch training — run the script in the background Poll for alerts — use trackio list alerts --project --json --since to check for new alerts Read metrics — use trackio get metric ... to inspect specific values Iterate — based on alerts and metrics, stop the run, adjust hyperparameters, and launch a new run

import trackio

trackio.init(project="my-project", config={"lr": 1e-4})

for step in range(num_steps): loss = train_step() trackio.log({"loss": loss, "step": step})

if step > 100 and loss > 5.0:
    trackio.alert(
        title="Loss divergence",
        text=f"Loss {loss:.4f} still high after {step} steps",

• level=trackio.
• AlertLevel.
• ERROR,. ) if step > 0 and abs(loss) < 1e-8: trackio.alert( title="Vanishing loss", text="Loss near zero — possible gradient collapse", level=trackio. AlertLevel. WARN, )

trackio.finish()

Then poll from a separate terminal/process:

trackio list alerts --project my-project --json --since "2025-01-01T00:00:00"Trackio - Experiment Tracking for ML Training

Trackio is an experiment tracking library for logging and visualizing ML training metrics. It syncs to Hugging Face Spaces for real-time monitoring dashboards.

Three Interfaces

When to Use Each

Python API → Logging

Use import trackio in your training scripts to log metrics:

Initialize tracking with trackio.init() Log metrics with trackio.log() or use TRL's report_to="trackio" Finalize with trackio.finish()

Key concept: For remote/cloud training, pass space_id — metrics sync to a Space dashboard so they persist after the instance terminates.

→ See references/logging_metrics.md for setup, TRL integration, and configuration options.

Python API → Alerts

Insert trackio.alert() calls in training code to flag important events — like inserting print statements for debugging, but structured and queryable:

• trackio.alert(title="...", level=trackio.
• AlertLevel.
• WARN) — fire an alert. Three severity levels: INFO, WARN, ERROR Alerts are printed to terminal, stored in the database, shown in the dashboard, and optionally sent to webhooks (Slack/Discord)

Key concept for LLM agents: Alerts are the primary mechanism for autonomous experiment iteration. An agent should insert alerts into training code for diagnostic conditions (loss spikes, NaN gradients, low accuracy, training stalls). Since alerts are printed to the terminal, an agent that is watching the training script's output will see them automatically. For background or detached runs, the agent can poll via CLI instead.

→ See references/alerts.md for the full alerts API, webhook setup, and autonomous agent workflows.

CLI → Retrieving

Use the trackio command to query logged metrics and alerts:

trackio list projects/runs/metrics — discover what's available trackio get project/run/metric — retrieve summaries and values trackio list alerts --project --json — retrieve alerts trackio show — launch the dashboard trackio sync — sync to HF Space

Key concept: Add --json for programmatic output suitable for automation and LLM agents.

→ See references/retrieving_metrics.md for all commands, workflows, and JSON output formats.

Minimal Logging Setup

import trackio

trackio.init(project="my-project", space_id="username/trackio") trackio.log({"loss": 0.1, "accuracy": 0.9}) trackio.log({"loss": 0.09, "accuracy": 0.91}) trackio.finish()

Minimal Retrieval

trackio list projects --json trackio get metric --project m