php-extended/php-lexer-object

A library that implements the php-extended/php-lexer-interface interface library.

8.0.9 2025-10-10 15:25 UTC

Requires

• php: >=8.1
• php-extended/php-charset-object: ^8
• php-extended/php-lexer-interface: ^8

Requires (Dev)

• php-extended/placeholder-phpunit: ^9

MIT

• Anastaszor

• php
• lexer
• implementation

README

A library that implements the php-extended/php-lexer-interface interface library.

Installation

The installation of this library is made via composer and the autoloading of all classes of this library is made through their autoloader.

• Download composer.phar from their website.
• Then run the following command to install this library as dependency :
• php composer.phar php-extended/php-lexer-object ^8

Basic Usage

For example, to create a lexer that parses digits, you shoud do the following : (This will parse the equivalent of the regex ^\d+.\d+$ ; ignoring all the "junk" characters that may be anywhere in the stream (junk = not dot, not digit)

use PhpExtended\Lexer\LexerInterface;
use PhpExtended\Lexer\LexerFactory;
use PhpExtended\Lexer\LexerConfiguration;

class CONSTS {
	const DIGIT = 1;
	const DOT   = 2;
	const DIGITWDOT = 3;
	const DIGITWTRAIL = 4;
}

$factory = new LexerFactory();
$config = new LexerConfiguration();
// parses all digits as class digit
$config->addMappings(LexerInterface::CLASS_DIGIT, CONSTS::DIGIT);
// parses all dots as class dot
$config->addMappings('.', CONSTS::DOT);
// a digit followed by a digit is still a digit
$config->addMerging(CONSTS::DIGIT, CONSTS::DIGIT, CONSTS::DIGIT);
// a digit followed by a dot is a digit-dot complex
$config->addMerging(CONSTS::DIGIT, CONSTS::DOT, CONSTS::DIGITWDOT);
// a digit-dot complex followed by a digit is a digit-with-trail complex
$config->addMerging(CONSTS::DIGITWDOT, CONSTS::DIGIT, CONSTS::DIGITWTRAIL);
// a digit-with-trail complex followed by a digit is still a digit-with-trail
$config->addMerging(CONSTS::DIGITWTRAIL, CONSTS::DIGIT, CONSTS::DIGITWTRAIL);

$data = '04972ifp.fhsà9300fd'; // the data 04972.9300 is hidden here
$lexer = $factory->createFromString($data, $config);
foreach($lexer as $lexeme)
{
	echo "\t".$lexeme."\n";
//        Code :   1      Line :   1      Column :   0    Data : 04972
//        Code :  -2      Line :   1      Column :   5    Data : ifp
//        Code :   2      Line :   1      Column :   8    Data : .
//        Code :  -2      Line :   1      Column :   9    Data : fhsà
//        Code :   1      Line :   1      Column :  14    Data : 9300
//        Code :  -2      Line :   1      Column :  18    Data : fd
}

$lexer = new CodeFilterLexer($lexer);
foreach($lexer as $lexeme)
{
	echo "\t".$lexeme."\n";
//        Code :   1      Line :   1      Column :   0    Data : 04972
//        Code :   2      Line :   1      Column :   8    Data : .
//        Code :   1      Line :   1      Column :  14    Data : 9300

}

$lexer = new LexerLexer($lexer, $config);
foreach($lexer as $lexeme)
{
	echo "\t".$lexeme."\n";
//        Code :   4      Line :   1      Column :   0    Data : 04972.9300
}

License

MIT (See license file).