pisotskyi-el / evomi-scraper
Laravel wrapper for Evomi Scraper API with DTO support
Requires
- php: ^8.1
- guzzlehttp/guzzle: ^7.0
- illuminate/support: ^9.0|^10.0|^11.0|^12.0|^13.0
Requires (Dev)
- orchestra/testbench: ^7.0|^8.0|^9.0|^10.0|^11.0
- phpunit/phpunit: ^9.0|^10.0|^11.0|^12.0
Suggests
None
Provides
None
Conflicts
None
Replaces
None
README
Laravel wrapper for Evomi Scraper API with full DTO support and easy configuration.
Repository: https://gitlab.com/pisotskyi.el/evomi-scraper
Features
- ✅ Full Laravel integration with Service Provider and Facade
- ✅ DTO (Data Transfer Objects) for type-safe requests and responses
- ✅ Publishable configuration file
- ✅ Built-in retry mechanism with intelligent error handling
- ✅ Support for all Evomi Scraper API parameters
- ✅ Easy to use API with Facade and Dependency Injection
- ✅ PSR-4 autoloading
Requirements
- PHP 8.1 or higher
- Laravel 9.x – 13.x
- Guzzle HTTP Client 7.x
Installation
Install the package via Composer:
composer require pisotskyi-el/evomi-scraper
Publish Configuration
Publish the configuration file to your Laravel application:
php artisan vendor:publish --tag=evomi-scraper-config
This will create a config/evomi-scraper.php file in your Laravel application.
Environment Configuration
Add your Evomi Scraper API key to your .env file:
EVOMI_SCRAPER_API_KEY=your-api-key-here
EVOMI_SCRAPER_BASE_URL=https://scrape.evomi.com
Usage
Basic Usage
use Pisotskyi\EvomiScraper\Facades\EvomiScraper;
// Simple GET request
$response = EvomiScraper::request('https://example.com')
->send();
echo $response->content;
echo $response->statusCode;
Browser Mode
use Pisotskyi\EvomiScraper\Facades\EvomiScraper;
$response = EvomiScraper::request('https://example.com')
->asBrowser()
->device('android')
->country('GB')
->sessionRandom() // generate 8-char session id
->waitUntil('load')
->waitFor(3) // seconds after page load, 0-30
->blockResources() // images, media, fonts
->send();
if ($response->isSuccessful()) {
echo $response->content;
}
Advanced Usage with All Parameters
use Pisotskyi\EvomiScraper\Facades\EvomiScraper;
$response = EvomiScraper::request('https://example.com')
->delivery('json') // Delivery mode: json (default), raw
->asBrowser() // Mode: asRequest(), asBrowser(), asAuto()
->asPremium() // Proxy: asDatacenter(), asPremium()
->randomDevice() // Device: device('windows'|'macos'|'android'), asWindows(), asMacos(), asAndroid(), randomDevice()
->country('US') // Country code
->state('california') // State/province (residential only)
->language('en-US,en') // Browser languages / Accept-Language
->timezone('America/Los_Angeles') // Browser timezone (browser/auto mode)
->format('html') // Output format: html, markdown, pdf, screenshot
->excludeTags(['nav', 'footer']) // Remove tags from html/markdown
->excludeSelectors(['.ads']) // Remove elements from html/markdown
->screenshot() // Capture screenshot (returned as screenshot_uri)
->waitUntil('load') // Page-ready condition: commit, domcontentloaded, load, networkidle
->waitFor(3) // Wait after page load (seconds, 0-30)
->waitForSelector('.content') // Wait for CSS selector (browser/auto mode)
->blockResources(['image', 'font']) // Block resource types, true = image, media, font
->headers(['Referer' => 'https://google.com']) // Custom headers sent to the target
->session('abc12345') // Session ID for proxy persistence (6-8 alphanumeric)
->async() // Submit task asynchronously
->send();
if ($response->isSuccessful()) {
$html = $response->content;
if ($response->screenshotUri) {
// Screenshot is stored by Evomi for 30 days
file_put_contents('screenshot.png', file_get_contents($response->screenshotUri));
}
}
Page Loading & Timeouts (browser mode)
By default the page counts as loaded on domcontentloaded. If the site redirects with JavaScript
or shows an anti-bot challenge after that, Evomi may read the page in the middle of the navigation:
Page.content: Unable to retrieve content because the page is navigating and changing the content.
Wait for a later state, a short pause or a selector of the real content:
EvomiScraper::request($url)
->asBrowser()
->waitUntil('load') // all resources loaded
->waitFor(3) // let JS redirects finish
->waitForSelector('.product') // or wait for an element that exists on the target page
->send();
networkidle waits until there is no network activity. Sites with websockets or constant analytics
never become idle, so use it only where needed.
Timeout after 60 seconds while loading page means the page did not load in Evomi's time limit.
Keep waitFor() small (it is counted in that limit), block heavy resources with blockResources()
and use asRequest() / asAuto() when JavaScript is not needed.
JavaScript Automation (browser/auto mode)
$response = EvomiScraper::request('https://example.com')
->asBrowser()
->jsInstructions([
['wait_for' => '.cookie-banner'],
['click' => '.accept-cookies'],
['fill' => ['#search', 'web scraping']],
['wait' => 500], // milliseconds
])
->executeJs('return document.title;') // result is in metadata['js_result']
->send();
AI Enhancement
$response = EvomiScraper::request('https://example.com/article')
->asBrowser()
->format('markdown')
->aiEnhance('markdown', 'Summarize in 3 bullet points') // source: markdown or screenshot
->send();
aiPrompt() alone has no effect, AI processing is enabled only by aiEnhance().
Other API Parameters
Any parameter without a builder method can be passed as is. It overrides builder values:
$response = EvomiScraper::request('https://example.com')
->param('extract_scheme', [
['label' => 'title', 'type' => 'content', 'selector' => 'h1'],
])
->param('webhook', ['url' => 'https://my-app.test/evomi-webhook'])
->send();
Using Dependency Injection
use Pisotskyi\EvomiScraper\EvomiScraperClient;
class MyController extends Controller
{
public function __construct(
private EvomiScraperClient $scraper
) {}
public function scrape()
{
$response = $this->scraper->request('https://example.com')
->asBrowser()
->country('US')
->send();
return response()->json([
'content' => $response->content,
'status' => $response->statusCode,
'credits' => $response->creditsUsed,
]);
}
}
Working with Response
$response = EvomiScraper::request('https://api.example.com/data')
->send();
// Check if successful
if ($response->isSuccessful()) {
echo "Success!";
}
// Get content
$html = $response->content;
// Parse JSON response
$data = $response->json();
// Get headers
$headers = $response->headers;
// Get metadata
$credits = $response->creditsUsed;
$time = $response->executionTime;
Delivery Mode
By default, requests are sent with delivery=json.
// Explicit JSON delivery (default behavior)
$jsonResponse = EvomiScraper::request('https://example.com')
->delivery('json')
->send();
// Raw delivery (HTML/markdown/image/pdf body depending on content)
$rawResponse = EvomiScraper::request('https://example.com')
->delivery('raw')
->send();
Async Task Status with Delivery
use Pisotskyi\EvomiScraper\DTO\ScraperRequestDTO;
// JSON status/details (default)
$statusJson = EvomiScraper::getTaskStatus('/api/v1/scraper/tasks/task_abc123');
// Raw final payload when task is completed
$statusRaw = EvomiScraper::getTaskStatus(
'/api/v1/scraper/tasks/task_abc123',
ScraperRequestDTO::DELIVERY_RAW
);
Configuration
The config/evomi-scraper.php file contains all available configuration options:
return [
// Your API key from Evomi dashboard
'api_key' => env('EVOMI_SCRAPER_API_KEY', ''),
// Base URL for API
'base_url' => env('EVOMI_SCRAPER_BASE_URL', 'https://scrape.evomi.com'),
// Request timeout in seconds, sync browser requests with waits take over 60 seconds
'timeout' => env('EVOMI_SCRAPER_TIMEOUT', 120),
// Connection timeout in seconds
'connect_timeout' => env('EVOMI_SCRAPER_CONNECT_TIMEOUT', 10),
// Retries for network errors and 5xx
'retry' => [
'enabled' => env('EVOMI_SCRAPER_RETRY_ENABLED', true),
'max_attempts' => env('EVOMI_SCRAPER_RETRY_MAX_ATTEMPTS', 3),
'delay_ms' => env('EVOMI_SCRAPER_RETRY_DELAY_MS', 1000),
],
];
If you published the config earlier, set EVOMI_SCRAPER_TIMEOUT=120 in .env: the old default was 30 seconds.
With 30 seconds the client gave up on browser requests that Evomi was still processing and the retry sent them again.
Retry Mechanism
The package includes an automatic retry mechanism with intelligent error handling:
- Retries network errors and 5xx responses (default: up to 3 attempts)
- Does not retry a client timeout after the connection was made: the task keeps running on Evomi side
- Exponential backoff between retries: 1s, 2s, 4s... (
delay_ms) - Does not retry client errors (4xx), including 429
- A sync request that exceeds the Evomi wait limit (HTTP 408) is not an error: the task keeps running,
the response is returned with
isAsyncPending() === trueandmetadata['check_url']for polling
Available Modes
Request Mode
Simple HTTP request. Fast and efficient.
- Datacenter: 1 credit
- Premium: 2 credits
Browser Mode
Full browser rendering with JavaScript support.
- Premium only: 5 credits
Auto Mode
Automatically chooses between request and browser based on the website.
- Request succeeds: 2 credits
- Upgraded to browser: 6 credits
Proxy Types
- datacenter: Fast and cheap, works only with
requestmode - premium (residential): More expensive but works with all modes, better for avoiding detection
Error Handling
use Pisotskyi\EvomiScraper\Exceptions\EvomiScraperException;
try {
$response = EvomiScraper::request('https://example.com')
->asBrowser()
->send();
if (!$response->isSuccessful()) {
echo "HTTP Error: " . $response->statusCode;
}
} catch (EvomiScraperException $e) {
echo "Error: " . $e->getMessage(); // e.g. "Evomi API error (HTTP 500): Navigation timeout"
echo "Code: " . $e->getCode(); // HTTP status, 0 if the API was not reached
$body = $e->getResponseData(); // decoded error body, e.g. details.validation_errors for 422
$e->isRetryable(); // true for network errors and 5xx
$e->getRetryAfter(); // seconds to wait after 429 (concurrency limit), null otherwise
}
Async Tasks
$response = EvomiScraper::request('https://example.com')->asBrowser()->async()->send();
if ($response->isAsyncPending()) {
// Poll every 2-3 seconds, Evomi keeps the result for 10 minutes
$status = EvomiScraper::getTaskStatus($response->metadata['check_url']);
// pending / processing / completed / failed
$status['status'];
// A failed task is returned as well (Evomi answers it with HTTP 500), the result is deleted after it
$status['error']['message'] ?? null;
}
Fluent Builder Methods Reference
HTTP Methods
get(),post()- Set HTTP method (methodparam, not described in the current Evomi docs)
Modes
asRequest()- Simple HTTP request (fast)asBrowser()- Full browser renderingasAuto()- Automatically choose mode
Proxy Types
asDatacenter()- Datacenter proxies (works with request mode only)asPremium()- Premium residential proxies (works with all modes)
Devices
device(string $device)- Set device explicitly (windows,macos,android)asWindows(),asMacos(),asAndroid(),randomDevice()- Shortcuts / helper methods- Helpers:
RequestBuilder::randomDeviceValue()returns random device string
Geo Location
country(string $code)- Country code (e.g., 'US', 'GB')state(string $state),city(string $city)- State and city (residential only, e.g. 'new.york', 'berlin')language(string $tags)- Browser languages / Accept-Language (e.g. 'de-CH,de')timezone(string $timezone)- Browser timezone (browser/auto mode)
Content & Format
format(string $format)- Output format: 'html', 'markdown', 'pdf', 'screenshot'includeContent(bool $enabled = true)- Include full content in JSON responsesexcludeTags(array|string $tags),excludeSelectors(array|string $selectors)- Remove elements (html/markdown)screenshot(bool $enabled = true)- Capture screenshot (browser mode), see$response->screenshotUripdf(bool $enabled = true)- Capture PDF (browser mode), see$response->pdfUri
Page Loading (browser/auto mode)
waitUntil(string $condition)-commit,domcontentloaded(API default),load,networkidlewaitFor(int $seconds)- Wait after page load, 0-30 secondswaitForSelector(string $selector)- Wait for CSS selectorblockResources(bool|array|string $types = true)- Block resource types (true= image, media, font)jsInstructions(array $instructions)- Actions:wait_for,click,fill,waitexecuteJs(string $script)- Run JavaScript, result inmetadata['js_result']
Request Configuration
session(string $id)- Session ID for proxy persistencesessionRandom(int $length = 8)- Set random session id (6-8 alnum)async(bool $enabled = true)- Submit task for background processingdelivery(string $delivery)- Response delivery mode:json(default),rawheaders(array $headers)- Custom HTTP headers sent to the targetcaptureHeaders(bool $enabled = true)- Capture response and cookie headersparam(string $key, mixed $value)- Any other API parameter (overrides builder values)
Helpers
RequestBuilder::randomSessionId(int $length = 8)- Generate session id (8 chars)RequestBuilder::randomDeviceValue()- Generate random device value
AI Features
aiEnhance(string $source = 'markdown', ?string $prompt = null)- Enable AI processingaiPrompt(string $prompt)- Custom AI prompt (works only withaiEnhance())
Execution
send()- Build and send the request, returnsScraperResponseDTO
Credits & Pricing
Transparent credit-based pricing:
- Request + Datacenter: 1 credit
- Request + Premium: 2 credits
- Browser + Premium: 5 credits
- Auto + Premium (request succeeds): 2 credits
- Auto + Premium (upgraded to browser): 6 credits
- Screenshot: +1 credit
- PDF: +1 credit
- AI Enhancement: +10 credits
Security
⚠️ Important: Never expose your API key in client-side code or public repositories. Always store it in environment variables and use HTTPS for all requests.
Backward Compatibility
The old DTO-based API is still supported for backward compatibility:
use Pisotskyi\EvomiScraper\Facades\EvomiScraper;
use Pisotskyi\EvomiScraper\DTO\ScraperRequestDTO;
// Using DTO directly
$request = new ScraperRequestDTO(
url: 'https://example.com',
mode: ScraperRequestDTO::MODE_BROWSER,
country: 'US'
);
$response = EvomiScraper::scrape($request);
// Simple scraping
$response = EvomiScraper::scrapeUrl('https://example.com');
Support
For API documentation and support, visit Evomi Documentation.
License
MIT License