jeroen / batching-iterator
Iterator that fetches values in batch
Installs: 3 933
Dependents: 1
Suggesters: 0
Security: 0
Stars: 4
Watchers: 2
Forks: 1
Open Issues: 1
Requires
- php: >=7.0
Requires (Dev)
- mediawiki/mediawiki-codesniffer: ~0.6.0
- ockcyp/covers-validator: ~0.6
- phpunit/phpunit: ~6.0
- squizlabs/php_codesniffer: ~2.5
Replaces
README
Small library providing an Iterator
that batches requests for additional values.
This is useful as a foundation for iterators over data that is in an expensive to
access location, such as a database or a web API.
Class overview
Core interface:
BatchingIterator
- iterator that batches requests via aBatchingFetcher
BatchingFetcher
- interface withfetchNext
method. You will likely need to create an implementation
Utilities:
MultipleBatchingFetcher
- combines a number ofBatchingFetcher
instances into oneInMemoryBatchingFetcher
- adapts anarray
to theBatchingFetcher
interfaceIteratorBasedBatchingFetcher
- adapts anIterator
to theBatchingFetcher
interface
Usage
Create a service that uses an Iterator
.
class TweetImporter { public function importTweets( Iterator $tweets ) { foreach ( $tweets as $tweet ) { $this->tweetStore->saveTweet( $tweet ); } } }
Note how this service only depends on Iterator. It is not aware of how the Iterator provides its
results. You thus decoupled the service from who retrieves the results, and from when this happens.
They could be coming from values already in memory, wrapped in an ArrayIterator
, or be pulled from
a web service as iteration happens. Using an ArrayIterator
is very helpful for testing.
Implement the BatchingFetcher
interface. If you already have a service to retrieve the data, this
can be a simple wrapper.
class BatchingTweetFetcher implements BatchingFetcher { public function fetchNext( $maxFetchCount ) { // Make a call to some external service to fetch $tweets return $tweets; } public function rewind() { // Go back to the first tweet } }
Now you can easily instantiate the service, have the batching optimization, and have all responsibilities nicely decoupled.
class TweetImportCli { public function importTweets() { $tweetIterator = new BatchingIterator( new BatchingTweetFetcher() ); $tweetIterator->setMaxBatchSize( 42 ); $this->tweetImporter->importTweets( $tweetIterator ); } }
Installation
You can use Composer to download and install this package as well as its dependencies.
To add this package as a local, per-project dependency to your project, simply add a
dependency on jeroen/batching-iterator
to your project's composer.json
file.
Here is a minimal example of a composer.json
file that just defines a dependency on
BatchingIterator 3.x:
{ "require": { "jeroen/batching-iterator": "~3.0" } }
Release notes
Version 3.0.0 (2017-05-16)
- Dropped support for PHP 5.x
- Added scalar and return type hints
Version 2.1.2 (2014-11-02)
- The max batch size in
BatchingIterator
now defaults to 10, avoiding usage of the iterator without this value being set.
Version 2.1.1 (2014-08-19)
- Release with package name
jeroen/batching-iterator
instead ofjeroen-de-dauw/batching-iterator
.
Version 2.1 (2014-07-19)
MultipleBatchingFetcher
now accepts an array ofBatchingFetcher
in its constructor
Version 2.0 (2014-07-19)
Breaking changes:
- Added
rewind
method to theBatchingFetcher
interface - Renamed
BatchingIterator\InMemoryBatchingFetcher
toBatchingIterator\Fetchers\InMemoryBatchingFetcher
New features and enhancements:
BatchingIterator
can now be iterated over multiple times- Added
MultipleBatchingFetcher
- Added
IteratorBasedBatchingFetcher
Version 1.0 (2014-07-03)
Initial release with
BatchingIterator
classBatchingFetcher
interfaceInMemoryBatchingFetcher
trivial implementation ofBatchingFetcher