Scrape Do Automation

What This Skill Does

This skill automates web scraping tasks using the Composio Scrape Do toolkit via Rube MCP. It's designed for users who need to extract data from websites and integrate it with other systems. The skill requires an active Scrape Do connection and Rube MCP setup.

When to Use

Extract product data from e-commerce sites Monitor competitor pricing Gather news articles and headlines Collect contact information from websites Scrape data for market research Automate data entry tasks

Key Features

Uses Rube MCP for tool execution Requires active Scrape Do connection Employs RUBE_SEARCH_TOOLS for tool discovery Uses RUBE_MANAGE_CONNECTIONS for connection management Supports RUBE_MULTI_EXECUTE_TOOL for workflow execution Provides documentation for the Scrape Do toolkit

Automate Scrape Do operations through Composio's Scrape Do toolkit via Rube MCP.

Toolkit docs: composio.dev/toolkits/scrape_do

Prerequisites

Rube MCP must be connected (RUBE_SEARCH_TOOLS available) Active Scrape Do connection via RUBE_MANAGE_CONNECTIONS with toolkit scrape_do Always call RUBE_SEARCH_TOOLS first to get current tool schemas

Setup

Get Rube MCP: Add https://rube.app/mcp as an MCP server in your client configuration. No API keys needed — just add the endpoint and it works.

Verify Rube MCP is available by confirming RUBE_SEARCH_TOOLS responds Call RUBE_MANAGE_CONNECTIONS with toolkit scrape_do If connection is not ACTIVE, follow the returned auth link to complete setup Confirm connection status shows ACTIVE before running any workflows

Tool Discovery

Always discover available tools before executing workflows:

RUBE_SEARCH_TOOLS queries: [{use_case: "Scrape Do operations", known_fields: ""}] session: {generate_id: true}

This returns available tool slugs, input schemas, recommended execution plans, and known pitfalls.

Core Workflow Pattern

Step 1: Discover Available Tools

RUBE_SEARCH_TOOLS queries: [{use_case: "your specific Scrape Do task"}] session: {id: "existing_session_id"}

Step 2: Check Connection

RUBE_MANAGE_CONNECTIONS toolkits: ["scrape_do"] session_id: "your_session_id"

Step 3: Execute Tools

RUBE_MULTI_EXECUTE_TOOL tools: [{ tool_slug: "TOOL_SLUG_FROM_SEARCH", arguments: {/ schema-compliant args from search results /} }] memory: {} session_id: "your_session_id"

Known Pitfalls

Always search first: Tool schemas change. Never hardcode tool slugs or arguments without calling RUBE_SEARCH_TOOLS Check connection: Verify RUBE_MANAGE_CONNECTIONS shows ACTIVE status before executing tools Schema compliance: Use exact field names and types from the search results Memory parameter: Always include memory in RUBE_MULTI_EXECUTE_TOOL calls, even if empty ({}) Session reuse: Reuse session IDs within a workflow. Generate new ones for new workflows Pagination: Check responses for pagination tokens and continue fetching until complete

Quick Reference

Powered by Composio Scrape Do Automation via Rube MCP

Automate Scrape Do operations through Composio's Scrape Do toolkit via Rube MCP.

Toolkit docs: composio.dev/toolkits/scrape_do

Prerequisites

Rube MCP must be connected (RUBE_SEARCH_TOOLS available) Active Scrape Do connection via RUBE_MANAGE_CONNECTIONS with toolkit scrape_do Always call RUBE_SEARCH_TOOLS first to get current tool schemas

Setup

Get Rube MCP: Add https://rube.app/mcp as an MCP server in your client configuration. No API keys needed — just add the endpoint and it works.

Verify Rube MCP is available by confirming RUBE_SEARCH_TOOLS responds Call RUBE_MANAGE_CONNECTIONS with toolkit scrape_do If connection is not ACTIVE, follow the returned auth link to complete setup Confirm connection status shows ACTIVE before running any workflows

Tool Discovery

Always discover available tools before executing workflows:

RUBE_SEARCH_TOOLS queries: [{use_case: "Scrape Do operations", known_fields: ""}] session: {generate_id: true}

This returns available tool slugs, input schemas, recommended execution plans, and known pitfalls.

Core Workflow Pattern

Step 1: Discover Available Tools

RUBE_SEARCH_TOOLS queries: [{use_case: "your specific Scrape Do task"}] session: {id: "existing_session_id"}

Step 2: Check Connection

RUBE_MANAGE_CONNECTIONS toolkits: ["scrape_do"] session_id: "your_session_id"

Step 3: Execute Tools

RUBE_MULTI_EXECUTE_TOOL tools: [{ tool_slug: "TOOL_SLUG_FROM_SEARCH", arguments: {/ schema-compliant args from search results /} }] memory: {} session_id: "your_session_id"

Known Pitfalls

Always search first: Tool schemas change. Never hardcode tool slugs or arguments without calling RUBE_SEARCH_TOOLS Check connection: Verify RUBE_MANAGE_CONNECTIONS shows ACTIVE status before executing tools Schema compliance: Use exact field names and types from the search results Memory parameter: Always include memory in RUBE_MULTI_EXECUTE_TOOL calls, even if empty ({}) Session reuse: Reuse session IDs within a workflow. Generate new ones for new workflows Pagination: Check responses for pagination tokens and continue fetching until complete

Quick Reference