Packages from t1gor
-
PHP
t1gor/robots-txt-parser
PHP class to parse robots.txt rules according to Google, Yandex, W3C and The Web Robots Pages specifications.
PHP
PHP class to parse robots.txt rules according to Google, Yandex, W3C and The Web Robots Pages specifications.