Search by

pisotskyi-el / evomi-scraper

pisotskyi.el

Laravel wrapper for Evomi Scraper API with DTO support

Package info

gitlab.com/pisotskyi.el/evomi-scraper

pkg:composer/pisotskyi-el/evomi-scraper

Statistics

Installs: 1 731

Dependents: 0

Suggesters: 0

Stars: 0

v1.0.19 2026-10-29 17:05 UTC

README

Laravel wrapper for Evomi Scraper API with full DTO support and easy configuration.

Repository: https://gitlab.com/pisotskyi.el/evomi-scraper

Features

  • ✅ Full Laravel integration with Service Provider and Facade
  • ✅ DTO (Data Transfer Objects) for type-safe requests and responses
  • ✅ Publishable configuration file
  • ✅ Built-in retry mechanism with intelligent error handling
  • ✅ Support for all Evomi Scraper API parameters
  • ✅ Easy to use API with Facade and Dependency Injection
  • ✅ PSR-4 autoloading

Requirements

  • PHP 8.1 or higher
  • Laravel 9.x – 13.x
  • Guzzle HTTP Client 7.x

Installation

Install the package via Composer:

composer require pisotskyi-el/evomi-scraper

Publish Configuration

Publish the configuration file to your Laravel application:

php artisan vendor:publish --tag=evomi-scraper-config

This will create a config/evomi-scraper.php file in your Laravel application.

Environment Configuration

Add your Evomi Scraper API key to your .env file:

EVOMI_SCRAPER_API_KEY=your-api-key-here
EVOMI_SCRAPER_BASE_URL=https://scrape.evomi.com

Usage

Basic Usage

use Pisotskyi\EvomiScraper\Facades\EvomiScraper;

// Simple GET request
$response = EvomiScraper::request('https://example.com')
    ->send();

echo $response->content;
echo $response->statusCode;

Browser Mode

use Pisotskyi\EvomiScraper\Facades\EvomiScraper;

$response = EvomiScraper::request('https://example.com')
    ->asBrowser()
    ->device('android')
    ->country('GB')
    ->sessionRandom() // generate 8-char session id
    ->waitUntil('load')
    ->waitFor(3)      // seconds after page load, 0-30
    ->blockResources() // images, media, fonts
    ->send();

if ($response->isSuccessful()) {
    echo $response->content;
}

Advanced Usage with All Parameters

use Pisotskyi\EvomiScraper\Facades\EvomiScraper;

$response = EvomiScraper::request('https://example.com')
    ->delivery('json')                          // Delivery mode: json (default), raw
    ->asBrowser()                               // Mode: asRequest(), asBrowser(), asAuto()
    ->asPremium()                               // Proxy: asDatacenter(), asPremium()
    ->randomDevice()                            // Device: device('windows'|'macos'|'android'), asWindows(), asMacos(), asAndroid(), randomDevice()
    ->country('US')                             // Country code
    ->state('california')                       // State/province (residential only)
    ->language('en-US,en')                      // Browser languages / Accept-Language
    ->timezone('America/Los_Angeles')           // Browser timezone (browser/auto mode)
    ->format('html')                            // Output format: html, markdown, pdf, screenshot
    ->excludeTags(['nav', 'footer'])            // Remove tags from html/markdown
    ->excludeSelectors(['.ads'])                // Remove elements from html/markdown
    ->screenshot()                              // Capture screenshot (returned as screenshot_uri)
    ->waitUntil('load')                         // Page-ready condition: commit, domcontentloaded, load, networkidle
    ->waitFor(3)                                // Wait after page load (seconds, 0-30)
    ->waitForSelector('.content')               // Wait for CSS selector (browser/auto mode)
    ->blockResources(['image', 'font'])         // Block resource types, true = image, media, font
    ->headers(['Referer' => 'https://google.com']) // Custom headers sent to the target
    ->session('abc12345')                       // Session ID for proxy persistence (6-8 alphanumeric)
    ->async()                                   // Submit task asynchronously
    ->send();

if ($response->isSuccessful()) {
    $html = $response->content;
    
    if ($response->screenshotUri) {
        // Screenshot is stored by Evomi for 30 days
        file_put_contents('screenshot.png', file_get_contents($response->screenshotUri));
    }
}

Page Loading & Timeouts (browser mode)

By default the page counts as loaded on domcontentloaded. If the site redirects with JavaScript or shows an anti-bot challenge after that, Evomi may read the page in the middle of the navigation:

Page.content: Unable to retrieve content because the page is navigating and changing the content.

Wait for a later state, a short pause or a selector of the real content:

EvomiScraper::request($url)
    ->asBrowser()
    ->waitUntil('load')              // all resources loaded
    ->waitFor(3)                     // let JS redirects finish
    ->waitForSelector('.product')    // or wait for an element that exists on the target page
    ->send();

networkidle waits until there is no network activity. Sites with websockets or constant analytics never become idle, so use it only where needed.

Timeout after 60 seconds while loading page means the page did not load in Evomi's time limit. Keep waitFor() small (it is counted in that limit), block heavy resources with blockResources() and use asRequest() / asAuto() when JavaScript is not needed.

JavaScript Automation (browser/auto mode)

$response = EvomiScraper::request('https://example.com')
    ->asBrowser()
    ->jsInstructions([
        ['wait_for' => '.cookie-banner'],
        ['click' => '.accept-cookies'],
        ['fill' => ['#search', 'web scraping']],
        ['wait' => 500], // milliseconds
    ])
    ->executeJs('return document.title;') // result is in metadata['js_result']
    ->send();

AI Enhancement

$response = EvomiScraper::request('https://example.com/article')
    ->asBrowser()
    ->format('markdown')
    ->aiEnhance('markdown', 'Summarize in 3 bullet points') // source: markdown or screenshot
    ->send();

aiPrompt() alone has no effect, AI processing is enabled only by aiEnhance().

Other API Parameters

Any parameter without a builder method can be passed as is. It overrides builder values:

$response = EvomiScraper::request('https://example.com')
    ->param('extract_scheme', [
        ['label' => 'title', 'type' => 'content', 'selector' => 'h1'],
    ])
    ->param('webhook', ['url' => 'https://my-app.test/evomi-webhook'])
    ->send();

Using Dependency Injection

use Pisotskyi\EvomiScraper\EvomiScraperClient;

class MyController extends Controller
{
    public function __construct(
        private EvomiScraperClient $scraper
    ) {}
    
    public function scrape()
    {
        $response = $this->scraper->request('https://example.com')
            ->asBrowser()
            ->country('US')
            ->send();
        
        return response()->json([
            'content' => $response->content,
            'status' => $response->statusCode,
            'credits' => $response->creditsUsed,
        ]);
    }
}

Working with Response

$response = EvomiScraper::request('https://api.example.com/data')
    ->send();

// Check if successful
if ($response->isSuccessful()) {
    echo "Success!";
}

// Get content
$html = $response->content;

// Parse JSON response
$data = $response->json();

// Get headers
$headers = $response->headers;

// Get metadata
$credits = $response->creditsUsed;
$time = $response->executionTime;

Delivery Mode

By default, requests are sent with delivery=json.

// Explicit JSON delivery (default behavior)
$jsonResponse = EvomiScraper::request('https://example.com')
    ->delivery('json')
    ->send();

// Raw delivery (HTML/markdown/image/pdf body depending on content)
$rawResponse = EvomiScraper::request('https://example.com')
    ->delivery('raw')
    ->send();

Async Task Status with Delivery

use Pisotskyi\EvomiScraper\DTO\ScraperRequestDTO;

// JSON status/details (default)
$statusJson = EvomiScraper::getTaskStatus('/api/v1/scraper/tasks/task_abc123');

// Raw final payload when task is completed
$statusRaw = EvomiScraper::getTaskStatus(
    '/api/v1/scraper/tasks/task_abc123',
    ScraperRequestDTO::DELIVERY_RAW
);

Configuration

The config/evomi-scraper.php file contains all available configuration options:

return [
    // Your API key from Evomi dashboard
    'api_key' => env('EVOMI_SCRAPER_API_KEY', ''),
    
    // Base URL for API
    'base_url' => env('EVOMI_SCRAPER_BASE_URL', 'https://scrape.evomi.com'),
    
    // Request timeout in seconds, sync browser requests with waits take over 60 seconds
    'timeout' => env('EVOMI_SCRAPER_TIMEOUT', 120),

    // Connection timeout in seconds
    'connect_timeout' => env('EVOMI_SCRAPER_CONNECT_TIMEOUT', 10),

    // Retries for network errors and 5xx
    'retry' => [
        'enabled' => env('EVOMI_SCRAPER_RETRY_ENABLED', true),
        'max_attempts' => env('EVOMI_SCRAPER_RETRY_MAX_ATTEMPTS', 3),
        'delay_ms' => env('EVOMI_SCRAPER_RETRY_DELAY_MS', 1000),
    ],
];

If you published the config earlier, set EVOMI_SCRAPER_TIMEOUT=120 in .env: the old default was 30 seconds. With 30 seconds the client gave up on browser requests that Evomi was still processing and the retry sent them again.

Retry Mechanism

The package includes an automatic retry mechanism with intelligent error handling:

  • Retries network errors and 5xx responses (default: up to 3 attempts)
  • Does not retry a client timeout after the connection was made: the task keeps running on Evomi side
  • Exponential backoff between retries: 1s, 2s, 4s... (delay_ms)
  • Does not retry client errors (4xx), including 429
  • A sync request that exceeds the Evomi wait limit (HTTP 408) is not an error: the task keeps running, the response is returned with isAsyncPending() === true and metadata['check_url'] for polling

Available Modes

Request Mode

Simple HTTP request. Fast and efficient.

  • Datacenter: 1 credit
  • Premium: 2 credits

Browser Mode

Full browser rendering with JavaScript support.

  • Premium only: 5 credits

Auto Mode

Automatically chooses between request and browser based on the website.

  • Request succeeds: 2 credits
  • Upgraded to browser: 6 credits

Proxy Types

  • datacenter: Fast and cheap, works only with request mode
  • premium (residential): More expensive but works with all modes, better for avoiding detection

Error Handling

use Pisotskyi\EvomiScraper\Exceptions\EvomiScraperException;

try {
    $response = EvomiScraper::request('https://example.com')
        ->asBrowser()
        ->send();
        
    if (!$response->isSuccessful()) {
        echo "HTTP Error: " . $response->statusCode;
    }
} catch (EvomiScraperException $e) {
    echo "Error: " . $e->getMessage();      // e.g. "Evomi API error (HTTP 500): Navigation timeout"
    echo "Code: " . $e->getCode();          // HTTP status, 0 if the API was not reached
    $body = $e->getResponseData();          // decoded error body, e.g. details.validation_errors for 422
    $e->isRetryable();                      // true for network errors and 5xx
    $e->getRetryAfter();                    // seconds to wait after 429 (concurrency limit), null otherwise
}

Async Tasks

$response = EvomiScraper::request('https://example.com')->asBrowser()->async()->send();

if ($response->isAsyncPending()) {
    // Poll every 2-3 seconds, Evomi keeps the result for 10 minutes
    $status = EvomiScraper::getTaskStatus($response->metadata['check_url']);

    // pending / processing / completed / failed
    $status['status'];

    // A failed task is returned as well (Evomi answers it with HTTP 500), the result is deleted after it
    $status['error']['message'] ?? null;
}

Fluent Builder Methods Reference

HTTP Methods

  • get(), post() - Set HTTP method (method param, not described in the current Evomi docs)

Modes

  • asRequest() - Simple HTTP request (fast)
  • asBrowser() - Full browser rendering
  • asAuto() - Automatically choose mode

Proxy Types

  • asDatacenter() - Datacenter proxies (works with request mode only)
  • asPremium() - Premium residential proxies (works with all modes)

Devices

  • device(string $device) - Set device explicitly (windows, macos, android)
  • asWindows(), asMacos(), asAndroid(), randomDevice() - Shortcuts / helper methods
  • Helpers: RequestBuilder::randomDeviceValue() returns random device string

Geo Location

  • country(string $code) - Country code (e.g., 'US', 'GB')
  • state(string $state), city(string $city) - State and city (residential only, e.g. 'new.york', 'berlin')
  • language(string $tags) - Browser languages / Accept-Language (e.g. 'de-CH,de')
  • timezone(string $timezone) - Browser timezone (browser/auto mode)

Content & Format

  • format(string $format) - Output format: 'html', 'markdown', 'pdf', 'screenshot'
  • includeContent(bool $enabled = true) - Include full content in JSON responses
  • excludeTags(array|string $tags), excludeSelectors(array|string $selectors) - Remove elements (html/markdown)
  • screenshot(bool $enabled = true) - Capture screenshot (browser mode), see $response->screenshotUri
  • pdf(bool $enabled = true) - Capture PDF (browser mode), see $response->pdfUri

Page Loading (browser/auto mode)

  • waitUntil(string $condition) - commit, domcontentloaded (API default), load, networkidle
  • waitFor(int $seconds) - Wait after page load, 0-30 seconds
  • waitForSelector(string $selector) - Wait for CSS selector
  • blockResources(bool|array|string $types = true) - Block resource types (true = image, media, font)
  • jsInstructions(array $instructions) - Actions: wait_for, click, fill, wait
  • executeJs(string $script) - Run JavaScript, result in metadata['js_result']

Request Configuration

  • session(string $id) - Session ID for proxy persistence
  • sessionRandom(int $length = 8) - Set random session id (6-8 alnum)
  • async(bool $enabled = true) - Submit task for background processing
  • delivery(string $delivery) - Response delivery mode: json (default), raw
  • headers(array $headers) - Custom HTTP headers sent to the target
  • captureHeaders(bool $enabled = true) - Capture response and cookie headers
  • param(string $key, mixed $value) - Any other API parameter (overrides builder values)

Helpers

  • RequestBuilder::randomSessionId(int $length = 8) - Generate session id (8 chars)
  • RequestBuilder::randomDeviceValue() - Generate random device value

AI Features

  • aiEnhance(string $source = 'markdown', ?string $prompt = null) - Enable AI processing
  • aiPrompt(string $prompt) - Custom AI prompt (works only with aiEnhance())

Execution

  • send() - Build and send the request, returns ScraperResponseDTO

Credits & Pricing

Transparent credit-based pricing:

  • Request + Datacenter: 1 credit
  • Request + Premium: 2 credits
  • Browser + Premium: 5 credits
  • Auto + Premium (request succeeds): 2 credits
  • Auto + Premium (upgraded to browser): 6 credits
  • Screenshot: +1 credit
  • PDF: +1 credit
  • AI Enhancement: +10 credits

Security

⚠️ Important: Never expose your API key in client-side code or public repositories. Always store it in environment variables and use HTTPS for all requests.

Backward Compatibility

The old DTO-based API is still supported for backward compatibility:

use Pisotskyi\EvomiScraper\Facades\EvomiScraper;
use Pisotskyi\EvomiScraper\DTO\ScraperRequestDTO;

// Using DTO directly
$request = new ScraperRequestDTO(
    url: 'https://example.com',
    mode: ScraperRequestDTO::MODE_BROWSER,
    country: 'US'
);

$response = EvomiScraper::scrape($request);

// Simple scraping
$response = EvomiScraper::scrapeUrl('https://example.com');

Support

For API documentation and support, visit Evomi Documentation.

License

MIT License