README

Adds a frontend search to TYPO3 that queries a YaCy peer.

Version 6.0, TYPO3 13.4 and 14.3, PHP 8.2+
Version 5.2 serves TYPO3 11.5 and 12.4; older cores live on the typo3-11, *-8.7, *-7.6 and *-6.2 branches.

Upgrading from 5.x

The plugins are now content types of their own. TYPO3 13.4 deprecated the list content element with its list_type sub types and v14 removes them, so a plugin registered the old way stops working.

Existing content elements must be migrated: records written by 5.x say CType="list" with list_type="yacy_search" and stop rendering otherwise. The identifiers themselves are unchanged, so TypoScript, page TSconfig and backend layouts referring to yacy_search keep working.

Run this once, after updating:

UPDATE tt_content SET CType = list_type, list_type = ''
WHERE CType = 'list' AND list_type IN ('yacy_search', 'yacy_suggest');

Backend user groups that allow the plugin need yacy_search added to their "Explicitly allow" list for tt_content.CType; the entry under tt_content.list_type no longer has any effect.

The extension deliberately ships no upgrade wizard. TYPO3 14 moved AbstractListTypeToCTypeUpdate and the UpgradeWizard attribute from EXT:install to EXT:core, and 14.3 no longer provides the old class names at all. A wizard class can therefore only extend the base class of one of the two supported cores, and excluding it from dependency injection - the only way to keep TYPO3 14 from loading it - would also stop TYPO3 13 registering it. The SQL above does the same work on both versions.

If you render the plugin yourself, tt_content.list.20.yacy_search becomes tt_content.yacy_search.

Two things change in the frontend for installs that never touched them:

filter now defaults to 1. It used to show a pair of image/text radio buttons inside the search form and was off by default; it now shows the document kind tabs described below. Set it back to 0 to keep the results unfiltered and the tabs hidden.
advancedSettings is gone. The constant had been declared since 5.x and was read by nothing - no template and no PHP class ever referenced it - so removing it changes no behaviour. A setup that assigns it can drop the line; an unused constant is harmless if it stays.
The radio buttons are gone from SearchForm.html. An overridden copy of that partial keeps working, but it will still render the old radios and will not carry the current kind through a new search - the current template has a hidden contentDom field where the radios used to be.

Loading the TypoScript

From TYPO3 13 on the extension ships a site set. Add it to the site's config.yaml:

# config/sites/<identifier>/config.yaml
dependencies:
  - eike/yacy

The static template ("Yacy Search") still works and can be included the traditional way instead. Either path loads the same files from Configuration/TypoScript/, which you can also @import directly.

Configuration

TypoScript constants under plugin.tx_yacy.settings:

Constant	Meaning
`protocol`, `domain`, `port`	How to reach the peer from PHP
`interface`	`yacysearch.json` or `yacysearch.rss`
`resultPage`	PID the search form submits to
`collection`	Restricts the query to a yacy collection
`itemsPerPage`	Page size of the result list
`filter`	Shows the document kind tabs above the results (default on)
`contentDomains`	Which kinds to offer, comma separated and in that order
`facets`	Which filter categories to show, comma separated and in that order
`facetElementLimit`	Values shown per facet (default `10`)
`facetsCollapsed`	Start with the categories folded away (default off)
`facetPanelCollapsed`	Start with the whole filter block put away (default off)
`suggestPageType`	Page type of the suggestion endpoint (default `1589`)
`suggestMinLength`	Characters needed before suggestions are fetched

The templateRootPath, partialRootPath and layoutRootPath constants take the usual Fluid override paths.

The extension setting debug (Admin Tools > Settings) makes the request URL visible in the frontend and rethrows peer errors instead of swallowing them. Without it a peer that is unreachable, or that answers with something other than the interface promises, degrades to an empty result list and a log entry.

Filtering the results

The peer returns filter categories alongside every result set - the same ones its own interface shows - and can be asked for one kind of document at a time. Both are rendered above the result list, and both are also listed as removable chips in an "active filters" bar, so a visitor can always get back out of a search they narrowed.

Facets

A facet value is applied by appending its modifier to the query, exactly as the peer's own interface does: clicking "github.com" under Domain turns typo3 into typo3 site:github.com. That has consequences worth knowing:

the narrowed search is visible and editable in the search field,
it survives pagination without any extra state,
removing it drops that one term and leaves the rest of the query alone.

facets names the categories to show and the order to show them in; an empty value shows everything the peer sent, in its order. A peer answers a common query with a hundred hosts, so facetElementLimit caps the values per category.

plugin.tx_yacy.settings {
    facets = protocols,hosts,filetype,language,authors,topics
    facetElementLimit = 10
}

The filters fold away at two levels, both plain <details> elements: each category on its own, and the whole block behind a single "Filter results" toggle. Neither needs JavaScript, both are keyboard operable, and both degrade to an open list if the stylesheet never loads.

plugin.tx_yacy.settings {
    facetsCollapsed = 0
    facetPanelCollapsed = 0
}

The two are independent: collapsing the panel says nothing about the categories inside it. A category holding an applied value is always open whatever facetsCollapsed says, otherwise nothing on screen explains why the result set is short — the panel has no such rule, because applied filters stay listed in the chips bar above it either way. Neither state is remembered across page loads.

Names the peer did not send are skipped rather than rendered empty. A value that is already part of the query is shown as plain text instead of a link - the peer marks it by negating the modifier it sends back, and the extension strips that sign so the remove link carries the term that is actually in the query.

Facets reach the templates from both interfaces. The RSS repository parses <yacy:navigation> since 6.0; before that the data arrived only through JSON.

Document kinds

contentdom is a request parameter rather than a query term, so it cannot be applied or removed the way a facet value is - but to a visitor it is one more filter, and it is rendered as tabs next to them.

plugin.tx_yacy.settings {
    filter = 1
    contentDomains = all,text,image,audio,video,app
}

all is the extension's name for leaving the parameter off. The other five are what yacy accepts; anything else in the list is dropped rather than passed on to the peer, as is a contentDom arriving from the form. Trim the list to what your peer actually has - a peer that has crawled only text pages answers an audio search with nothing, and an empty tab is worse than no tab. Fewer than two usable entries renders no tabs at all.

The chosen kind travels with the search form as a hidden field, so typing a new query keeps it rather than silently dropping back to searching everything.

Suggestions

The search field fetches suggestions from a same-origin TYPO3 page type which proxies the peer's suggest.json. The browser never contacts the peer: a site served over HTTPS cannot call a plain-HTTP peer, and the peer is a foreign origin. Resources/Public/JavaScript/Suggest.js is a dependency-free ES module.

The site configuration needs a route enhancer for the page type, otherwise TYPO3's routing answers the endpoint with a 404:

# config/sites/<identifier>/config.yaml
routeEnhancers:
  PageTypeSuffix:
    type: PageType
    default: ''
    index: index
    map:
      yacy-suggest.json: 1589

The search term travels as the plain query parameter tx_yacy_suggest_q, which ext_localconf.php adds to FE.cacheHash.excludedParameters - without that, PageArgumentValidator rejects the request as an un-hashed dynamic argument.

Suggest.js is included through page.includeJSFooter with excludeFromConcatenation and disableCompression. Asset bundlers - the Bootstrap Package among them - otherwise merge it into a bundle and drop the type="module" attribute, which moves it into the head and runs it before the search form exists.

Running the tests

The extension carries a standalone, container based test setup - it does not need a TYPO3 installation around it. Everything runs through Build/Scripts/runTests.sh, which is the same runner the TYPO3 core uses.

# once, and after changing composer.json: install into .Build/
./Build/Scripts/runTests.sh -s composerUpdateMax

./Build/Scripts/runTests.sh                 # unit tests (the default)
./Build/Scripts/runTests.sh -s functional   # functional tests, on sqlite
./Build/Scripts/runTests.sh -s cgl -n       # code style, dry run
./Build/Scripts/runTests.sh -h              # every available suite

Useful variations:

./Build/Scripts/runTests.sh -p 8.4                              # another PHP version
./Build/Scripts/runTests.sh -s functional -d mariadb -i 10.11   # another DBMS
./Build/Scripts/runTests.sh -x                                  # with xdebug on port 9003
./Build/Scripts/runTests.sh -- --filter getPeerUrl              # pass options to phpunit
./Build/Scripts/runTests.sh -b docker                           # force docker over podman

Testing the other supported TYPO3 version

-t selects the core for the install step; the suites then use whatever sits in .Build. Both supported cores run the full unit and functional suites:

./Build/Scripts/runTests.sh -t 14.3 -s composerUpdateMax
./Build/Scripts/runTests.sh -s unit
./Build/Scripts/runTests.sh -s functional

./Build/Scripts/runTests.sh -t 13.4 -s composerUpdateMax        # back to the default

TYPO3	PHP	testing-framework	Fluid
13.4	8.2 - 8.4	9.6	4.x
14.3	8.2 - 8.4	9.6	5.x

composerUpdateMin/Max wipe .Build/vendor first. Switching between the two cores otherwise leaves an autoloader pointing at files of the previous dependency set, which fails with a missing symfony/translation include.

-s rector applies the TYPO3 and PHP upgrade rules from Build/rector/config.php; CI runs it with -n so a pending rule fails the build instead of rewriting code.

Extending the result list

Other extensions can enrich or filter results by listening to Eike\Yacy\Event\BeforeReturnResultsEvent, which every search repository dispatches right before handing the result back:

# Configuration/Services.yaml
MyVendor\MyExt\EventListener\MyListener:
  tags:
    - name: event.listener
      identifier: 'my-ext/yacy-results'
      event: Eike\Yacy\Event\BeforeReturnResultsEvent

The event carries the Demand, the current page number and the mutable SearchResult. See EXT:yacy_peoplesearch for a worked example.

Since 5.2 the event is dispatched by the RSS repository as well, not only the JSON one, and it receives the page actually being rendered rather than always 1.

getJson() and setJson() from 5.1 keep working and operate on the same data, presented in the shape of the yacysearch.json payload. A listener written against 5.1 that manipulates $json['channels'][0]['items'] needs no change:

// 5.1 style, still supported
$json = $event->getJson();
array_unshift($json['channels'][0]['items'], $item);
$event->setJson($json);

// 5.2 style
$event->getResult()->prependItem($item);

eike / yacy

Maintainers

Package info

Statistics

Security