Scrappey — Web Scraping API Wrapper

The official Node.js client for the Scrappey web scraping API. Render and retrieve fully-loaded web pages, run browser automation, manage persistent sessions, and route requests through residential proxies — all behind a single, typed API.

Responsible use: This wrapper is intended for collecting publicly available data in compliance with applicable laws. See the Disclaimer.

Features

Browser automation: Full browser actions (click, type, scroll, execute JS)
Session management: Persistent sessions with cookie and state management
All HTTP methods: GET, POST, PUT, DELETE, PATCH
Proxy support: Built-in residential proxies with country selection
Screenshots & video: Capture screenshots and record browser sessions
TypeScript support: Full TypeScript declarations included

Pricing

Scrappey uses simple, pay-as-you-go pricing:

From €1 per 1,000 requests (JS rendering + residential proxies)
Up to 200 concurrent requests out of the box
30+ browser automation actions
No monthly commitment — pay only for what you use

How It Works

Scrappey handles the infrastructure of page rendering, proxy routing, and session management behind a single API:

flowchart TB
    subgraph YourApp[Your Application]
        Code[Your Code]
        Wrapper[Scrappey Wrapper]
    end
    
    subgraph ScrappeyAPI[Scrappey API]
        RequestHandler[Request Handler]
        BrowserEngine[Browser Engine]
        ProxyManager[Proxy Manager]
    end
    
    subgraph TargetSite[Target Website]
        Website[Website Content]
    end
    
    Code -->|1. Send Request| Wrapper
    Wrapper -->|2. API Request| RequestHandler
    
    RequestHandler -->|5. Execute| BrowserEngine
    BrowserEngine -->|6. Use Proxy| ProxyManager
    ProxyManager -->|7. Residential IP| Website
    
    Website -->|8. Return Content| BrowserEngine
    BrowserEngine -->|9. Process Response| RequestHandler
    RequestHandler -->|10. Return Data| Wrapper
    Wrapper -->|11. Formatted Response| Code

Request Flow

Your code sends a request through the Scrappey wrapper
Scrappey API receives and processes the request
Browser engine executes browser actions if needed (click, type, scroll)
Proxy manager routes through residential proxies
Target website returns content
Response is formatted and returned to your application

Use Cases

E-commerce: Product data, prices, reviews
Social media: Public profile data, posts, engagement metrics
Search engines: SERP data, rankings
Real estate: Property listings, prices, details
News & content: Articles, headlines, metadata
API testing: Test your own APIs from different IPs
Data aggregation: Collect data from multiple sources

Installation

npm install scrappey-wrapper

Quick Start

const Scrappey = require('scrappey-wrapper');

const scrappey = new Scrappey('YOUR_API_KEY');

// Basic GET request
const response = await scrappey.get({
    url: 'https://example.com'
});

console.log(response.solution.response);   // HTML content
console.log(response.solution.statusCode); // 200

Request Modes

Scrappey supports two request modes:

Mode	Description	Cost	Best For
`browser`	Headless browser (default)	1 + 0.2 balance/request	Complex pages, JS rendering, browser actions
`request`	HTTP library with TLS	0.2 balance/request	Simple requests, speed-critical, cost-sensitive applications

Using Browser Mode (Default)

Browser mode uses a real headless browser, enabling full JavaScript execution and browser actions:

const response = await scrappey.get({
    url: 'https://example.com',
    requestType: 'browser', // Default — can be omitted
    browserActions: [
        { type: 'click', cssSelector: '#button' },
        { type: 'wait', wait: 1000 }
    ]
});

Using Request Mode (Faster & Cheaper)

Request mode uses an HTTP library with TLS fingerprinting — faster and cheaper for simple requests:

const response = await scrappey.get({
    url: 'https://api.example.com/data',
    requestType: 'request' // HTTP mode — 5x cheaper
});

Use request mode for:

API calls and JSON endpoints
Simple HTML pages without heavy JavaScript
High-volume collection where cost matters
Speed-critical applications

Use browser mode for:

Sites with JavaScript-rendered content
Browser actions (click, type, scroll)

Drop-in Replacement for Axios/Fetch

New in v2.0.0: Use Scrappey as a drop-in replacement for axios or fetch. Change your import and all requests automatically route through Scrappey, with rendering, and proxy support.

Axios Drop-in

// Before
import axios from 'axios';
const response = await axios.get('https://example.com');

// After — just change the import
import axios from 'scrappey-wrapper/axios';
axios.defaults.apiKey = 'YOUR_API_KEY';
const response = await axios.get('https://example.com');

All axios methods work:

import axios from 'scrappey-wrapper/axios';

axios.defaults.apiKey = 'YOUR_API_KEY';

// GET request
const response = await axios.get('https://example.com', {
    headers: { 'Authorization': 'Bearer token' },
    params: { page: 1 }
});

// POST request
const response = await axios.post('https://api.example.com/submit', {
    name: 'John',
    email: 'john@example.com'
}, {
    headers: { 'Content-Type': 'application/json' }
});

// All standard axios options work
const response = await axios.get('https://example.com', {
    timeout: 5000,
    proxy: 'http://proxy:port',
    cookies: 'session=abc123',
    responseType: 'json'
});

Response format matches axios:

const response = await axios.get('https://example.com');
console.log(response.data);        // Response body
console.log(response.status);      // HTTP status code
console.log(response.statusText);  // Status text
console.log(response.headers);     // Response headers
console.log(response.scrappey);    // Additional Scrappey data (verified, cookies, etc.)

Fetch Drop-in

// Before
const response = await fetch('https://example.com');
const data = await response.json();

// After
import fetch from 'scrappey-wrapper/fetch';
fetch.configure({ apiKey: 'YOUR_API_KEY' });

const response = await fetch('https://example.com');
const data = await response.json();

All fetch methods work:

import fetch from 'scrappey-wrapper/fetch';

fetch.configure({ apiKey: 'YOUR_API_KEY' });

// GET request
const response = await fetch('https://example.com', {
    headers: { 'Authorization': 'Bearer token' }
});

// POST request
const response = await fetch('https://api.example.com/submit', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ name: 'John', email: 'john@example.com' })
});

// Response methods
const text = await response.text();
const json = await response.json();
const blob = await response.blob();
const arrayBuffer = await response.arrayBuffer();

Migration Guide

From Axios:

Change import: import axios from 'axios' → import axios from 'scrappey-wrapper/axios'
Set API key: axios.defaults.apiKey = 'YOUR_API_KEY'
That's it — your existing code works the same.

From Fetch:

Change import: import fetch from 'node-fetch' → import fetch from 'scrappey-wrapper/fetch'
Configure: fetch.configure({ apiKey: 'YOUR_API_KEY' })
All fetch calls now route through Scrappey.

Configuration

Axios:

import axios from 'scrappey-wrapper/axios';

// Set defaults
axios.defaults.apiKey = 'YOUR_API_KEY';
axios.defaults.premiumProxy = true;
axios.defaults.timeout = 60000;

// Or create a custom instance
const scrappeyAxios = axios.create({
    apiKey: 'YOUR_API_KEY',
    premiumProxy: true
});

Fetch:

import fetch from 'scrappey-wrapper/fetch';

fetch.configure({
    apiKey: 'YOUR_API_KEY',
    premiumProxy: true,
    timeout: 60000
});

Supported Options

All standard axios/fetch options are supported:

headers → customHeaders
data / body → postData
params → URL query string
timeout → timeout
proxy → proxy
cookies → cookies or cookiejar
responseType: 'json' → uses innerText for JSON

Plus Scrappey-specific options:

browserActions, screenshot, video
session, premiumProxy, proxyCountry
And many more

Session Management

Both adapters support Scrappey session management:

// Axios
import axios from 'scrappey-wrapper/axios';
axios.defaults.apiKey = 'YOUR_API_KEY';

const session = await axios.createSession();
const sessionId = session.session;

await axios.get('https://example.com', { session: sessionId });
await axios.destroySession(sessionId);

// Fetch
import fetch from 'scrappey-wrapper/fetch';
fetch.configure({ apiKey: 'YOUR_API_KEY' });

const session = await fetch.createSession();
const sessionId = session.session;

await fetch('https://example.com', { session: sessionId });
await fetch.destroySession(sessionId);

API Reference

Constructor

const scrappey = new Scrappey(apiKey, options);

Parameter	Type	Description
`apiKey`	`string`	Your Scrappey API key (required)
`options.baseUrl`	`string`	Custom API base URL (optional)
`options.timeout`	`number`	Default timeout in ms (default: 300000)

HTTP Methods

GET Request

const response = await scrappey.get({
    url: 'https://example.com',
    session: 'optional-session-id',
    // ... other options
});

POST Request

const response = await scrappey.post({
    url: 'https://api.example.com/submit',
    postData: {
        name: 'John Doe',
        email: 'john@example.com'
    },
    customHeaders: {
        'content-type': 'application/json'
    }
});

PUT, DELETE, PATCH

await scrappey.put({ url, postData, ...options });
await scrappey.delete({ url, ...options });
await scrappey.patch({ url, postData, ...options });

Session Management

// Create a session
const session = await scrappey.createSession({
    proxyCountry: 'UnitedStates',
    // proxy: 'http://user:pass@ip:port'
});
const sessionId = session.session;

// Use the session
await scrappey.get({
    url: 'https://example.com',
    session: sessionId
});

// Check if a session is active
const status = await scrappey.isSessionActive(sessionId);

// List all sessions
const sessions = await scrappey.listSessions(userId);

// Destroy session when done
await scrappey.destroySession(sessionId);

Browser Actions

const response = await scrappey.get({
    url: 'https://example.com/login',
    browserActions: [
        { type: 'wait_for_selector', cssSelector: '#login-form' },
        { type: 'type', cssSelector: '#username', text: 'myuser' },
        { type: 'type', cssSelector: '#password', text: 'mypassword' },
        { type: 'click', cssSelector: '#submit', waitForSelector: '.dashboard' },
        { type: 'execute_js', code: 'document.querySelector(".user-info").innerText' }
    ]
});

// Access JS execution results
console.log(response.solution.javascriptReturn[0]);

Available Actions

Action	Description
`click`	Click on an element
`type`	Type text into an input
`goto`	Navigate to a URL
`wait`	Wait for milliseconds
`wait_for_selector`	Wait for an element to appear
`wait_for_function`	Wait for a JS condition
`wait_for_load_state`	Wait for page load state
`wait_for_cookie`	Wait for a cookie to be set
`execute_js`	Execute JavaScript
`scroll`	Scroll to element or bottom
`hover`	Hover over an element
`keyboard`	Press keyboard keys
`dropdown`	Select a dropdown option
`switch_iframe`	Switch to iframe context
`set_viewport`	Set viewport size
`if`	Conditional actions
`while`	Loop actions

Screenshots & Video

const response = await scrappey.get({
    url: 'https://example.com',
    screenshot: true,
    screenshotWidth: 1920,
    screenshotHeight: 1080,
    video: true
});

const screenshot = response.solution.screenshot; // Base64
const videoUrl = response.solution.videoUrl;

Data Extraction

const response = await scrappey.get({
    url: 'https://example.com',
    cssSelector: '.product-title',
    innerText: true,
    includeLinks: true,
    includeImages: true,
    regex: 'price: \\$([0-9.]+)'
});

Request Interception

const response = await scrappey.get({
    url: 'https://example.com',
    interceptFetchRequest: '/api/data',
    abortOnDetection: ['analytics', 'tracking', 'ads'],
    whitelistedDomains: ['example.com', 'cdn.example.com'],
    blackListedDomains: ['ads.com']
});

console.log(response.solution.interceptFetchRequestResponse);

Configuration Options

Option	Type	Description
`url`	`string`	Target URL (required)
`session`	`string`	Session ID for session reuse
`proxy`	`string`	Proxy string (`http://user:pass@ip:port`)
`proxyCountry`	`string`	Proxy country (e.g., "UnitedStates")
`premiumProxy`	`boolean`	Use premium proxy pool
`mobileProxy`	`boolean`	Use mobile proxy pool
`postData`	`object/string`	POST/PUT/PATCH request data
`customHeaders`	`object`	Custom HTTP headers
`cookies`	`string`	Cookie string to set
`cookiejar`	`array`	Cookie jar array
`localStorage`	`object`	LocalStorage data to set
`browserActions`	`array`	Browser actions to execute
`cssSelector`	`string`	Extract content by CSS selector
`innerText`	`boolean`	Include page text content
`includeLinks`	`boolean`	Include all page links
`includeImages`	`boolean`	Include all page images
`screenshot`	`boolean`	Capture screenshot
`video`	`boolean`	Record video
`pdf`	`boolean`	Generate PDF
`filter`	`array`	Filter response fields
`timeout`	`number`	Request timeout (ms)

Response Structure

{
    solution: {
        verified: true,              // Request verification status
        response: '<html>...',       // HTML content
        statusCode: 200,             // HTTP status code
        currentUrl: 'https://...',   // Final URL after redirects
        userAgent: 'Mozilla/5.0...', // User agent used
        cookies: [...],              // Array of cookies
        cookieString: '...',         // Cookie string
        responseHeaders: {...},      // Response headers
        innerText: '...',            // Page text content
        screenshot: 'base64...',     // Base64 screenshot
        screenshotUrl: 'https://...',// Screenshot URL
        videoUrl: 'https://...',     // Video URL
        javascriptReturn: [...],     // JS execution results
    },
    timeElapsed: 1234,               // Request time (ms)
    data: 'success',                 // 'success' or 'error'
    session: 'session-id',           // Session ID
    error: '...',                    // Error message (if failed)
}

Error Handling

try {
    const response = await scrappey.get({ url: 'https://example.com' });

    if (response.data === 'error') {
        console.error('Error:', response.error);
        // See: https://wiki.scrappey.com/getting-started
    }
} catch (error) {
    console.error('Request failed:', error.message);
}

Examples

Multi-language examples are available in the examples directory:

Node.js: examples/nodejs
TypeScript: examples/typescript
Python: examples/python
Go: examples/go
C#: examples/csharp
PHP: examples/php
Java: examples/java
Ruby: examples/ruby
Rust: examples/rust
cURL: examples/curl

TypeScript

Full TypeScript support is included:

import Scrappey = require('scrappey-wrapper');

const scrappey = new Scrappey('API_KEY');

const response = await scrappey.get({
    url: 'https://example.com',
    browserActions: [
        { type: 'execute_js', code: 'document.title' }
    ]
});

console.log(response.solution.statusCode); // Fully typed

For AI / LLM Agents

Package: scrappey-wrapper (Node.js); install via npm install scrappey-wrapper.
Purpose: Send HTTP requests through the Scrappey API to render and retrieve web pages, run browser automation, manage sessions, and route through residential proxies for collecting publicly available data.
Auth: API key passed to new Scrappey('YOUR_API_KEY'), or axios.defaults.apiKey / fetch.configure({ apiKey }) for the drop-in adapters.
Core call: await scrappey.get({ url }) → returns response.solution.response (HTML) and response.solution.statusCode.
Modes: requestType: 'browser' (default, full rendering) or 'request' (HTTP+TLS, cheaper/faster).
Key options: browserActions, session, premiumProxy, proxyCountry, screenshot, cssSelector.
Drop-in adapters: scrappey-wrapper/axios and scrappey-wrapper/fetch mirror the axios/fetch APIs.
Docs: https://wiki.scrappey.com
Intended use: Collecting publicly available data in line with applicable law and each target site's Terms of Service.

Links

Website: https://scrappey.com
Documentation: https://wiki.scrappey.com
Request Builder: https://app.scrappey.com/#/builder
GitHub: https://github.com/pim97/scrappey.js
NPM: https://www.npmjs.com/package/scrappey-wrapper

License

This project is licensed under the MIT License — see the LICENSE file for details.

Disclaimer

Ensure your web scraping activities comply with each website's Terms of Service and applicable legal regulations. Scrappey is not responsible for misuse of the library. Use it responsibly and respect website policies, robots.txt, and data-privacy laws.

Name		Name	Last commit message	Last commit date
Latest commit History 95 Commits
.github/workflows		.github/workflows
examples		examples
src		src
tests		tests
types		types
.gitignore		.gitignore
LICENSE		LICENSE
axios.js		axios.js
fetch.js		fetch.js
index.js		index.js
package-lock.json		package-lock.json
package.json		package.json
readme.md		readme.md

Folders and files

Latest commit

History

Repository files navigation

Scrappey — Web Scraping API Wrapper

Features

Pricing

How It Works

Request Flow

Use Cases

Installation

Quick Start

Request Modes

Using Browser Mode (Default)

Using Request Mode (Faster & Cheaper)

Drop-in Replacement for Axios/Fetch

Axios Drop-in

Fetch Drop-in

Migration Guide

Configuration

Supported Options

Session Management

API Reference

Constructor

HTTP Methods

GET Request

POST Request

PUT, DELETE, PATCH

Session Management

Browser Actions

Available Actions

Screenshots & Video

Data Extraction

Request Interception

Configuration Options

Response Structure

Error Handling

Examples

TypeScript

For AI / LLM Agents

Links

License

Disclaimer

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases 7

Packages 0

Uh oh!

Uh oh!

Contributors

Uh oh!

Languages

Packages