umpirsky/centipede-crawler

Crawls all unique links.

0.1.2 2016-06-13 10:09 UTC

This package is not auto-updated.

Last update: 2024-10-26 17:24:15 UTC


README

symfony upgrade fixertwig gettext extractorwisdomcentipedepermissions handlerextraloadgravatarlocurrocountry listtransliterator

Centipede Crawler Build Status

Crawls all unique links.

Usage

$urls = (new Centipede\Crawler('http://dev.umpirsky.com'))->crawl();

Asynchronous

(new Centipede\Crawler('http://dev.umpirsky.com'))->crawl(function ($url, GuzzleHttp\Message\FutureResponse $response) {
    printf('(%d) %s', $response->getStatusCode(), $url);
});