kvazar/crawler

KVAZAR Index Crawler

Maintainers

Details

github.com/kvazar-network/crawler

Source

Issues

Installs: 0

Dependents: 0

Suggesters: 0

Security: 0

Stars: 0

Watchers: 1

Forks: 0

Open Issues: 1

Type:project

dev-main 2025-03-31 17:25 UTC

Requires

MIT db7ed825168e9cdc530456f2649f1840fa31431d

This package is auto-updated.

Last update: 2025-03-31 17:25:33 UTC

README

Kevacoin blockchain crawler with full-text index storage in the shared Manticore database (based on Sphinx)

Compatible

Install

  • apt install git composer manticore php-fpm php-curl php-mbstring php-pdo php-bcmath
  • git clone https://github.com/kvazar-network/crawler.git
  • cd crawler
  • composer update

Setup

cp example/config.json config.json

Manticore

To prevent data lose on server failures, change default binlog flush strategy to binlog_flush = 1

Kevacoin

Remote or local kevacoind connection required as the data source for this crawler.

Create new system user: useradd -m kevacoin then copy kevacoind compiled into the profile root; make sure kevacoind binary has executable permissions: chmod +x kevacoind

To accept RPC connections from crawler, create kevacoin.conf:

#/home/kevacoin/.kevacoin/kevacoin.conf
rpcuser=USERNAME
rpcpassword=PASSWORD
  • make sure config.json corresponds rpcuser and rpcpassword values!

System

manticore.service

This unit auto-created on install Manticore from the official repository

kevacoin.service

#/etc/systemd/system/kevacoin.service
[Unit]
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=kevacoin
Group=kevacoin
ExecStart=/home/kevacoin/kevacoind -server

[Install]
WantedBy=multi-user.target

crawler.service

#/etc/systemd/system/crawler.service
[Unit]
After=network-online.target manticore.service kevacoin.service
Wants=network-online.target manticore.service kevacoin.service

[Service]
Type=simple
User=kevacoin
Group=kevacoin
ExecStart=/usr/bin/php /home/kevacoin/crawler/src/index.php > /dev/null 2>&1
StandardOutput=null
StandardError=null

[Install]
WantedBy=multi-user.target

Apply changes

  • systemctl daemon-reload
  • systemctl enable kevacoin manticore crawler
  • systemctl start kevacoin manticore crawler
  • systemctl status kevacoin manticore crawler

CLI

Crawler works on background as the systemd process; if you want to operate it manually, use following CLI:

  • php src/index.php 60 - crawl, check for new blocks every n seconds (60 by default)
  • php src/index.php drop - truncate index
  • php src/index.php optimize - optimize index