Scrape Pages

Extract content from any webpage as Markdown, HTML, JSON, links, or screenshots. Scrayle handles JavaScript rendering, anti-bot protection, and content extraction automatically.

Basic Scrape

import { ScrayleClient } from '@scrayle/sdk';

const client = new ScrayleClient();

const result = await client.scrapingCrawl.scrapeUrl(
  'https://example.com',
  { formats: ['markdown'] }
);

console.log(result.data?.markdown);

Output Formats

Multiple Formats

const result = await client.scrapingCrawl.scrapeUrl(
  'https://example.com',
  { formats: ['markdown', 'links', 'screenshot'] }
);

console.log(result.data?.markdown);
console.log(result.data?.links);       // string[]
console.log(result.data?.screenshot);  // base64 string

Scrape Options

Content Filtering

const result = await client.scrapingCrawl.scrapeUrl(
  'https://example.com/article',
  {
    formats: ['markdown'],
    onlyMainContent: true,       // Skip nav/footer/sidebar
    excludeTags: ['img', 'svg'], // Skip images
    waitFor: 2000,               // Wait 2s for dynamic content
  }
);

Async Scraping

Start a scrape without waiting for the result:

// Start the scrape
const { id } = await client.scrapingCrawl.scrape.asyncScrapeUrl(
  'https://example.com',
  { formats: ['markdown'] }
);

// Check status later
const status = await client.scrapingCrawl.scrape.checkScrapeStatus(id);
if (status.status === 'completed') {
  console.log(status.data?.markdown);
}

With AI Extraction

Extract structured data using AI:

const result = await client.scrapingCrawl.scrapeUrl(
  'https://example.com/products',
  {
    formats: ['json'],
    extract: {
      prompt: 'Extract all product names and prices',
      schema: {
        type: 'object',
        properties: {
          products: {
            type: 'array',
            items: {
              type: 'object',
              properties: {
                name: { type: 'string' },
                price: { type: 'string' },
              },
            },
          },
        },
      },
    },
  }
);

console.log(result.data?.json);

Next Steps

• Batch Scraping - Scrape multiple URLs at once
• Crawl Websites - Follow links and scrape entire sites
• Universal Scraper - JavaScript rendering and web unlocking