dimgraycat/pickurl

A simple PHP Web Url Pick.

0.0.8 2016-03-01 14:05 UTC

Requires

• php: >=5.5.0
• fabpot/goutte: 3.1.0
• sabre/uri: 1.0.1

MIT

• dimgraycat

• scraper

README

Requirements

PHP-PickUrl depends on PHP 5.5+ and Goutte 1+ and Guzzle 6+.

Installation

This library can be found on Packagist. The recommended way to install this is through composer.

Edit your composer.json and add:

{
    "require": {
        "dimgraycat/pickurl": "*"
    }
}

And install dependencies:

$ curl -sS https://getcomposer.org/installer | php
$ php composer.phar install

Usage

Create a Pickurl\Spider Client instance:

<?php
$spider = new PickUrl\Spider();
$spider->addHook('before', function($crawler, $crawl_url) {
    print "$crawl_url\n";
})
->addHook('after', function($crawler, $crawl_url, $searched_urls) {
    print_r($searched_urls);
})
->crawl('http://foo.sample.com');

set UserAgent:

$this->setUserAgent("MyCrawler 1.0");

set WaitTime:

// default is 10sec. It can be shortened by setting.
$this->WaitTime(1);

License

PHP-PickUrl is licensed under the MIT license.