stevebauman / hypertext
The best HTML to text transformer
Installs: 93 398
Dependents: 1
Suggesters: 0
Security: 0
Stars: 147
Watchers: 3
Forks: 5
Open Issues: 0
Requires
- ezyang/htmlpurifier: ^4.16
Requires (Dev)
- pestphp/pest: ^2.24
- spatie/ray: ^1.39
Suggests
- ext-dom: Required for filtering HTML.
- ext-libxml: Required for filtering HTML.
README
A PHP HTML to pure text transformer that beautifully handles various and malformed HTML.
Hypertext is excellent at pulling text content out of any HTML based document and automatically:
- Removes CSS
- Removes scripts
- Removes headers
- Removes non-HTML based content
- Preserves spacing
- Preserves links (optional)
- Preserves new lines (optional)
It is directed at using the output in LLM related tasks, such as prompts and embeddings.
Installation
composer require stevebauman/hypertext
Usage
use Stevebauman\Hypertext\Transformer; $transformer = new Transformer(); // (Optional) Filter out specific elements by their XPath. $transformer->filter("//*[@id='some-element']"); // (Optional) Retain new line characters. $transformer->keepNewLines(); // (Optional) Retain anchor tags and their href attribute. $transformer->keepLinks(); $text = $transformer->toText($html);
Example
For larger examples, please view the tests/Fixtures directory.
Input:
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8"> <meta name="viewport" content="width=device-width, initial-scale=1.0"> <title>My Blog</title> </head> <body> <h1>Welcome to My Blog</h1> <p>This is a paragraph of text on my webpage.</p> <a href="https://blog.com/posts">Click here</a> to view my posts. </body> </html>
Output (Pure Text):
echo (new Transformer)->toText($html);
Welcome to My Blog This is a paragraph of text on my webpage. Click here to view my posts.
Output (Keep New Lines):
echo (new Transformer)->keepNewLines()->toText($html);
Welcome to My Blog
This is a paragraph of text on my webpage.
Click here to view my posts.
Output (Keep Links):
echo (new Transformer)->keepLinks()->toText($html);
Welcome to My Blog This is a paragraph of text on my webpage. <a href="https://blog.com/posts">Click Here</a> to view my posts.
Output (Keep Both):
echo (new Transformer) ->keepLinks() ->keepNewLines() ->toText($html);
Welcome to My Blog
This is a paragraph of text on my webpage.
<a href="https://blog.com/posts">Click Here</a> to view my posts.