Langsmith Fetch

What This Skill Does

The Langsmith Fetch skill helps debug LangChain and LangGraph agents by fetching execution traces directly from LangSmith Studio. It allows users to quickly identify errors, analyze tool usage, and review agent performance, all from the terminal. This skill is designed for developers actively building and debugging agents.

When to Use

Debug a failing agent execution trace. Show recent agent activity and traces. Check for errors in agent execution. Analyze agent memory operations. Review agent performance and token usage. Show the tools called during agent execution.

Key Features

Fetches traces from LangSmith Studio. Analyzes traces for errors and failures. Reports tools called during execution. Provides execution times and token usage. Offers quick debugging of recent activity. Enables deep dives into specific traces.

Manual Installation

Manual installationView Full Skill ContentThe complete markdown content that gets installedLangSmith Fetch - Agent Debugging Skill Debug LangChain and LangGraph agents by fetching execution traces directly from LangSmith Studio in your terminal.

When to Use This Skill

Automatically activate when user mentions: 🐛 "Debug my agent" or "What went wrong?" 🔍 "Show me recent traces" or "What happened?" ❌ "Check for errors" or "Why did it fail?" 💾 "Analyze memory operations" or "Check LTM" 📊 "Review agent performance" or "Check token usage" 🔧 "What tools were called?" or "Show execution flow"

Prerequisites

1. Install langsmith-fetch

pip install langsmith-fetch

2. Set Environment Variables

export LANGSMITH_API_KEY="your_langsmith_api_key" export LANGSMITH_PROJECT="your_project_name" Verify setup: echo $LANGSMITH_API_KEY echo $LANGSMITH_PROJECT

Core Workflows

Workflow 1: Quick Debug Recent Activity

When user asks: "What just happened?" or "Debug my agent" Execute: langsmith-fetch traces --last-n-minutes 5 --limit 5 --format pretty Analyze and report: ✅ Number of traces found ⚠️ Any errors or failures 🛠️ Tools that were called ⏱️ Execution times 💰 Token usage Example response format: Found 3 traces in the last 5 minutes: Trace 1: ✅ Success Agent: memento Tools: recall_memories, create_entities Duration: 2.3s Tokens: 1,245 Trace 2: ❌ Error Agent: cypher Error: "Neo4j connection timeout" Duration: 15.1s Failed at: search_nodes tool Trace 3: ✅ Success Agent: memento Tools: store_memory Duration: 1.8s Tokens: 892 💡 Issue found: Trace 2 failed due to Neo4j timeout. Recommend checking database connection.

Workflow 2: Deep Dive Specific Trace

When user provides: Trace ID or says "investigate that error" Execute: langsmith-fetch trace <trace-id> --format json Analyze JSON and report: 🎯 What the agent was trying to do 🛠️ Which tools were called (in order) ✅ Tool results (success/failure) ❌ Error messages (if any) 💡 Root cause analysis 🔧 Suggested fix Example response format: Deep Dive Analysis - Trace abc123 Goal: User asked "Find all projects in Neo4j" Execution Flow: ✅ search_nodes(query: "projects") → Found 24 nodes ❌ get_node_details(node_id: "proj_123") → Error: "Node not found" → This is the failure point ⏹️ Execution stopped Root Cause: The search_nodes tool returned node IDs that no longer exist in the database, possibly due to recent deletions. Suggested Fix: Add error handling in get_node_details tool Filter deleted nodes in search results Update cache invalidation strategy Token Usage: 1,842 tokens ($0.0276) Execution Time: 8.7 seconds

Workflow 3: Export Debug Session

When user says: "Save this session" or "Export traces" Execute:

Create session folder with timestamp

SESSION_DIR="langsmith-debug/session-$(date +%Y%m%d-%H%M%S)" mkdir -p "$SESSION_DIR" Export traces langsmith-fetch traces "$SESSION_DIR/traces" --last-n-minutes 30 --limit 50 --include-metadata Export threads (conversations) langsmith-fetch threads "$SESSION_DIR/threads" --limit 20 Report: ✅ Session exported successfully! Location: langsmith-debug/session-20251224-143022/ Traces: 42 files Threads: 8 files You can now: Review individual trace files Share folder with team Analyze with external tools Archive for future reference Session size: 2.3 MB

Workflow 4: Error Detection

When user asks: "Show me errors" or "What's failing?" Execute:

Fetch recent traces

langsmith-fetch traces --last-n-minutes 30 --limit 50 --format json > recent-traces.json Search for errors grep -i "error|failed|exception" recent-traces.json Analyze and report: 📊 Total errors found ❌ Error types and frequency 🕐 When errors occurred 🎯 Which agents/tools failed 💡 Common patterns Example response format: Error Analysis - Last 30 Minutes Total Traces: 50 Failed Traces: 7 (14% failure rate) Error Breakdown: Neo4j Connection Timeout (4 occurrences)

• Agent: cypher
• Tool: search_nodes
• First occurred: 14:32
• Last occurred: 14:45
• Pattern: Happens during peak load Memory Store Failed (2 occurrences)
• Agent: memento
• Tool: store_memory
• Error: "Pinecone rate limit exceeded"
• Occurred: 14:38, 14:41 Tool Not Found (1 occurrence)
• Agent: sqlcrm
• Attempted tool: "export_report" (doesn't exist)
• Occurred: 14:35 💡 Recommendations: Add retry logic for Neo4j timeouts Implement rate limiting for Pinecone Fix sqlcrm tool configuration

Common Use Cases

Use Case 1: "Agent Not Responding"

User says: "My agent isn't doing anything" Steps: Check if traces exist: langsmith-fetch traces --last-n-minutes 5 --limit 5 If NO traces found:

• Tracing might be disabled
• Check: LANGCHAIN_TRACING_V2=true in environment
• Check: LANGCHAIN_API_KEY is set
• Verify agent actually ran If traces found:
• Review for errors
• Check execution time (hanging?)
• Verify tool calls completed

Use Case 2: "Wrong Tool Called"

User says: "Why did it use the wrong tool?" Steps: Get the specific trace Review available tools at execution time Check agent's reasoning for tool selection Examine tool descriptions/instructions Suggest prompt or tool config improvements

Use Case 3: "Memory Not Working"

User says: "Agent doesn't remember things" Steps: Search for memory operations: langsmith-fetch traces --last-n-minutes 10 --limit 20 --format raw | grep -i "memory|recall|store" Check:

• Were memory tools called?
• Did recall return results?
• Were memories actually stored?
• Are retrieved memories being used?

Use Case 4: "Performance Issues"

User says: "Agent is too slow" Steps: Export with metadata: langsmith-fetch traces ./perf-analysis --last-n-minutes 30 --limit 50 --include-metadata Analyze:

• Execution time per trace
• Tool call latencies
• Token usage (context size)
• Number of iterations
• Slowest operations Identify bottlenecks and suggest optimizations

Output Format Guide

Pretty Format (Default)

langsmith-fetch traces --limit 5 --format pretty Use for: Quick visual inspection, showing to users

JSON Format

langsmith-fetch traces --limit 5 --format json Use for: Detailed analysis, syntax-highlighted review

Raw Format

langsmith-fetch traces --limit 5 --format raw Use for: Piping to other commands, automation

Advanced Features

Time-Based Filtering

After specific timestamp

langsmith-fetch traces --after "2025-12-24T13:00:00Z" --limit 20 Last N minutes (most common) langsmith-fetch traces --last-n-minutes 60 --limit 100

Include Metadata

Get extra context

langsmith-fetch traces --limit 10 --include-metadata Metadata includes: agent type, model, tags, environment

Concurrent Fetching (Faster)

Speed up large exports

langsmith-fetch traces ./output --limit 100 --concurrent 10

Troubleshooting

"No traces found matching criteria"

Possible causes: No agent activity in the timeframe Tracing is disabled Wrong project name API key issues Solutions:

1. Try longer timeframe

langsmith-fetch traces --last-n-minutes 1440 --limit 50