Senior ML Engineer

What This Skill Does

This skill provides tools and templates for deploying machine learning models to production, setting up MLOps pipelines, and integrating LLMs. It's designed for ML engineers who need to automate their model deployment and monitoring processes.

When to Use

• Deploy a model using a Docker container.
• Set up experiment tracking with MLflow.
• Configure a feature store for training data.
• Monitor model latency and error rates.
• Integrate a Retrieval Augmented Generation system.
• Choose the right serving option for a model.

Key Features

• Provides a Dockerfile template for model deployment.
• Offers guidance on choosing serving options like FastAPI and Triton.
• Automates training and deployment pipelines.
• Integrates with feature stores like Feast and Tecton.
• Supports experiment tracking with MLflow and Weights & Biases.
• Offers a model deployment workflow with canary deployments.

Manual Installation

Manual installation View Full Skill Content The complete markdown content that gets installed Senior ML Engineer

Production ML engineering patterns for model deployment, MLOps infrastructure, and LLM integration.

Table of Contents

Model Deployment Workflow

MLOps Pipeline Setup LLM Integration Workflow RAG System Implementation Model Monitoring Reference Documentation Tools

Model Deployment Workflow

Deploy a trained model to production with monitoring:

Export model to standardized format (ONNX, TorchScript, SavedModel) Package model with dependencies in Docker container Deploy to staging environment Run integration tests against staging Deploy canary (5% traffic) to production Monitor latency and error rates for 1 hour Promote to full production if metrics pass Validation: p95 latency < 100ms, error rate < 0.1%

Container Template

FROM python:3.11-slim

COPY requirements.txt . RUN pip install --no-cache-dir -r requirements.txt

COPY model/ /app/model/ COPY src/ /app/src/

HEALTHCHECK CMD curl -f http://localhost:8080/health || exit 1

EXPOSE 8080 CMD ["uvicorn", "src.server:app", "--host", "0.0.0.0", "--port", "8080"]

Serving Options

MLOps Pipeline Setup

Establish automated training and deployment:

Configure feature store (Feast, Tecton) for training data Set up experiment tracking (MLflow, Weights & Biases) Create training pipeline with hyperparameter logging Register model in model registry with version metadata Configure staging deployment triggered by registry events Set up A/B testing infrastructure for model comparison Enable drift monitoring with alerting Validation: New models automatically evaluated against baseline

Feature Store Pattern

from feast import Entity, Feature, FeatureView, File Source user = Entity (name="user_id", value_type=ValueType.INT64)

user_features = Feature View ( name="user_features", entities=["user_id"], ttl=timedelta (days=1), features=[ Feature (name="purchase_count_30d", dtype=ValueType.INT64), Feature (name="avg_order_value", dtype=ValueType.FLOAT), ], online=True, source=File Source (path="data/user_features.parquet"), )

Retraining Triggers

LLM Integration Workflow

Integrate LLM APIs into production applications:

Create provider abstraction layer for vendor flexibility Implement retry logic with exponential backoff Configure fallback to secondary provider Set up token counting and context truncation Add response caching for repeated queries Implement cost tracking per request Add structured output validation with Pydantic Validation: Response parses correctly, cost within budget

Provider Abstraction

from abc import ABC, abstractmethod from tenacity import retry, stop_after_attempt, wait_exponential

class LLMProvider (ABC): @abstractmethod def complete (self, prompt: str, **kwargs) -> str: pass

@retry (stop=stop_after_attempt (3), wait=wait_exponential (min=1, max=10)) def call_llm_with_retry (provider: LLMProvider, prompt: str) -> str: return provider.complete (prompt)

Cost Management

RAG System Implementation

Build retrieval-augmented generation pipeline:

Choose vector database (Pinecone, Qdrant, Weaviate) Select embedding model based on quality/cost tradeoff Implement document chunking strategy Create ingestion pipeline with metadata extraction Build retrieval with query embedding Add reranking for relevance improvement Format context and send to LLM Validation: Response references retrieved context, no hallucinations

Vector Database Selection

Chunking Strategies

Model Monitoring

Monitor production models for drift and degradation:

Set up latency tracking (p50, p95, p99) Configure error rate alerting Implement input data drift detection Track prediction distribution shifts Log ground truth when available Compare model versions with A/B metrics Set up automated retraining triggers Validation: Alerts fire before user-visible degradation

Drift Detection

from scipy.stats import ks_2samp

def detect_drift (reference, current, threshold=0.05): statistic, p_value = ks_2samp (reference, current) return { "drift_detected": p_value < threshold, "ks_statistic": statistic, "p_value": p_value }

Alert Thresholds

Reference Documentation

MLOps Production Patterns

references/mlops_production_patterns.md contains:

Model deployment pipeline with Kubernetes manifests Feature store architecture with Feast examples Model monitoring with drift detection code A/B testing infrastructure with traffic splitting Automated retraining pipeline with MLflow

LLM Integration Guide

references/llm_integration_guide.md contains:

Provider abstraction layer pattern Retry and fallback strategies with tenacity Prompt engineering templates (few-shot, CoT) Token optimization with tiktoken Cost calculation and tracking

RAG System Architecture

references/rag_system_architecture.md contains:

RAG pipeline implementation with code Vector database comparison and integration Chunking strategies (fixed, semantic, recursive) Embedding model selection guide Hybrid search and reranking patterns

Tools

Model Deployment Pipeline

python scripts/model_deployment_pi