Olostep for OpenClaw

Give your OpenClaw agent the live web. Scrape any URL to clean markdown, search Google with structured results, crawl entire sites, batch-process up to 10,000 URLs in parallel, and get AI-synthesized answers with citations — all from inside your agent workflow.

This plugin provides 13 skills for high-level agent workflows and a 9-tool MCP server for direct tool access. Handles JS rendering, anti-bot bypass, CAPTCHAs, and residential proxies automatically.

Installation

Install from ClawHub:

clawhub install olostep

Or add the MCP server directly to your OpenClaw configuration:

{
  "mcpServers": {
    "olostep": {
      "command": "npx",
      "args": ["-y", "olostep-mcp"],
      "env": {
        "OLOSTEP_API_KEY": "your-api-key-here"
      }
    }
  }
}

Get your free API key at olostep.com/auth — 500 requests/month, no credit card.

Skills

Core Data Skills

Workflow Skills

MCP Tools

The bundled MCP server (olostep-mcp) exposes 9 tools callable by any MCP-compatible agent:

Real Developer Workflows

Debug an obscure error

Your agent searches for the exact error message across GitHub issues and documentation, scrapes the most relevant threads, and delivers a fix — not a generic suggestion, an actual fix based on what people who hit the same error actually did.

Write code from live docs

AI models hallucinate API parameters and use deprecated methods. The docs-to-code skill scrapes the current documentation and writes code from what is really there. No stale training data, no guessing.

Research before choosing a tool

Comparing ORMs? Evaluating payment providers? The research skill searches multiple sources, scrapes actual pricing and feature pages, and returns a structured comparison with a recommendation and citations.

Ingest an entire docs site for RAG

map → discover all URLs on the docs site
batch → scrape every page in parallel as clean markdown
→ feed into your vector store

Extract structured data at scale

map → find all product/listing URLs
batch → scrape with a pre-built parser for typed JSON
→ pipe into your database, API, or seed files

Migrate to a new framework version

The migrate-code skill scrapes the official migration guide, extracts every breaking change with before/after patterns, and refactors your code based on the real documentation.

SDK Quick Reference

Node.js / TypeScript

npm install olostep

import Olostep from 'olostep';

const client = new Olostep({ apiKey: process.env.OLOSTEP_API_KEY });

// Scrape a page to markdown
const scrape = await client.scrapes.create('https://example.com');
console.log(scrape.markdown_content);

// Batch scrape with custom IDs
const batch = await client.batches.create([
  { url: 'https://example.com/page-1', customId: 'p1' },
  { url: 'https://example.com/page-2', customId: 'p2' },
]);

// Crawl a site
const crawl = await client.crawls.create({
  url: 'https://docs.example.com',
  maxPages: 50,
  maxDepth: 3,
});

// Discover URLs
const map = await client.maps.create({
  url: 'https://example.com',
  topN: 100,
  searchQuery: 'pricing',
});

Python

pip install olostep

from olostep import Olostep

client = Olostep()  # reads OLOSTEP_API_KEY from env

# Scrape
result = client.scrapes.create(url_to_scrape="https://example.com")
print(result.markdown_content)

# Batch
batch = client.batches.create(urls=["https://example.com", "https://example.org"])

# Crawl
crawl = client.crawls.create(start_url="https://docs.example.com", max_pages=50)

# Map
site_map = client.maps.create(url="https://example.com")

# AI Answers with structured output
answer = client.answers.create(
    task="Compare Stripe vs Square pricing",
    json_format={"providers": [{"name": "", "pricing": "", "best_for": ""}]},
)

REST API

Base URL: https://api.olostep.com/v1 Auth: Authorization: Bearer <API_KEY>

Pre-built Parsers

Use with parser to get structured JSON from specific site types:

Framework Integrations

This plugin works with OpenClaw directly. If you are building outside of OpenClaw, Olostep has dedicated SDKs and integrations:

Testing

The plugin ships with two test suites:

Static integrity tests (run on every push and PR):

pip install pytest
pytest tests/test_skill_integrity.py -v

Validates: frontmatter fields, MCP tool references, Python code block syntax, and absence of deprecated parameters across all 13 skill definitions.

Live API smoke tests (run on push to main):

OLOSTEP_API_KEY=your_key pytest tests/test_live_api.py -v --timeout=60

Verifies scrape, map, answers, and batch workflows against the live Olostep API.

Project Structure

olostep-clawhub-plugin/
  .clawhub/
    plugin.json          # Plugin metadata, keywords, logo reference
  .github/
    workflows/
      integrity.yml      # CI: static checks on PR, live tests on push to main
  skills/
    answers/SKILL.md     # AI-powered answers with citations
    batch/SKILL.md       # Batch scrape up to 10k URLs
    crawl/SKILL.md       # Autonomous website crawling
    debug-error/SKILL.md # Error debugging from live web
    docs-to-code/SKILL.md # Write code from scraped docs
    extract-schema/SKILL.md # Structured data extraction
    integrate/SKILL.md   # Auto-detect stack, write SDK integration
    map/SKILL.md         # URL discovery and site mapping
    migrate-code/SKILL.md # Framework migration from docs
    research/SKILL.md    # Multi-source competitive research
    scrape/SKILL.md      # Single-page scraping
    search/SKILL.md      # Web search (3 modes)
    setup/SKILL.md       # API key configuration
  tests/
    conftest.py          # Shared fixtures, known MCP tools, stale params
    test_skill_integrity.py  # Static validation of all SKILL.md files
    test_live_api.py     # Live API smoke tests
  mcp.json               # MCP server configuration
  LICENSE                 # MIT
  README.md

Links

• Olostep Website
• API Documentation
• Get API Key (free, 500 req/month)
• Node.js SDK
• Python SDK
• MCP Server on npm
• OpenClaw Skill (lightweight, skill-only)
• ClawHub Plugin Page

License

MIT — see LICENSE for details.