oihana / php-robots
The Oihana PHP Robots library
Requires
- php: >=8.4
- oihana/php-commands: dev-main
- oihana/php-core: dev-main
- oihana/php-files: dev-main
- oihana/php-schema: dev-main
- oihana/php-standards: dev-main
- oihana/php-system: dev-main
Requires (Dev)
- mikey179/vfsstream: ^1.6
- nunomaduro/collision: ^8.8
- phpdocumentor/shim: ^3.8
- phpunit/phpunit: ^12
README
A PHP library to create and manage robots.txt
files, built on top of the oihana-php-commands library.
✨ Features
- Create or remove a project's robots.txt from the CLI
- Optional custom path via -f|--file (absolute or relative)
- Clear console support via -c|--clear
- Config-driven defaults (config.toml)
A robots.txt file is a simple text file placed on a website’s root directory to give instructions to web crawlers (like search engine bots) about which pages or sections of the site should or shouldn’t be crawled and indexed. It helps control search engine access and manage site visibility.
📦 Installation
Requires PHP 8.4+
Install via Composer:
composer require oihana/php-robots
🚀 Usage
The package provides a Symfony Console command named command:robots with actions:
- create: Generate a
robots.txt
file - remove: Delete a
robots.txt
file
Examples:
Create a robots.txt
using defaults
bin/console command:robots create
Create a robots.txt
at a custom path
bin/console command:robots create --path /var/www/my-website/htdocs/robots.txt
or
bin/console command:robots create --p /var/www/my-website/htdocs/robots.txt
Remove the default robots.txt
bin/console command:robots remove
Remove a robots.txt at a custom path
bin/console command:robots remove --path /var/www/my-website/htdocs/robots.txt
Clear the console before running
bin/console command:robots create --clear
Options
Option | ShortCut | Description |
---|---|---|
--clear | -c | Clear the console before running |
--path | -p | The directory path of the 'robots.txt' file |
Notes:
- If
--path
is a relative path, it is resolved against the current working directory. - On creation/removal, parent directory existence and permissions are validated.
⚙️ Configuration
You can set defaults in config/config.toml under the [robots] section:
[robots] path = "/path/to/your/project/htdocs" overwrite = true permissions = 0o644 owner = "www-data" group = "www-data" content = ''' User-agent: * Disallow: / '''
🧪 Programmatic usage
You can instantiate and configure the command in PHP if needed:
use DI\Container; use oihana\robots\commands\RobotsCommand; $container = new Container(); $command = new RobotsCommand ( null, // let kernel resolve the name $container, [ 'robots' => [ 'path' => '/var/www/my-website/htdocs', 'content' => "User-agent: *\nDisallow: /private/" ] ] );
🔚 Exit codes
- 0 Success
- 1 Failure (invalid action, IO failure, etc.)
❓ Troubleshooting
- Ensure the parent directory for the
robots.txt
is writable (especially when using a custom--path
option). - When using relative paths with
--path
, they are resolved from the current working directory (pwd). - For more details, see the inline documentation in RobotsCommand.php.
✅ Running Unit Tests
To run all tests:
composer test
🧾 License
This project is licensed under the Mozilla Public License 2.0 (MPL-2.0).
👤 About the author
- Author: Marc ALCARAZ (aka eKameleon)
- Mail: marc@ooop.fr
- Website: http://www.ooop.fr