Here and there bits and pieces are coming together in the YaCy code that are not yet visible but will be assembled (hopefully) this year to something useful. A lot of it might be experimental, but I don’t want it to be mystical, I want that it is transparent. Therefore I am posting this here now even if there is nothing “ready to be clicked” in the code, just to introduce this topic as an AMA for this topic.
I started a “going public” of AI in YaCy with my patreon posting last week: https://www.patreon.com/posts/ai-done-right-143053426
It describes only principles, not functions. The place to describe possible function of AI in YaCy is now here!
We actually have a lot of code in YaCy that belong to those “bits and pieces” right now already in the YaCy code:
- a MCP server, which is simply a standard-implementation for a tooling-server catalog that describes that this server (that YaCy server) is a tool for web search. It works and can be integrated in any workflow that has a MCP-client, like N8N. The commit is here: https://github.com/yacy/yacy_search_server/commit/f41415a299ec5ac0f3797ee6f6a68faebbdc3940
- a RAG Proxy (RAG: Retrieval Augmented Generation), that is a OpenAI-Compatible chat server which takes the latest prompt, turns it into a search request, takes the search results and integrates it into the chat history as “plugged-in knowledge” (there are several methods to do that). This is very rudimentary but it is there since commit https://github.com/yacy/yacy_search_server/commit/13fbff0bffc78aa4676e92eb0ff95e5a9e98872f - it must be enhanced a lot to have configuration which actual LLM backend has to be used. It also does not support streaming right now and the “stuffing” method must be replaced by tooling - I am working on that.
- A full implementation of a Transformer in commit https://github.com/yacy/yacy_search_server/commit/b0a94aba8f337b12f7775ce2d4d90041cfa2f39b - this is huge, since it integrates a CPU-based LLM Engine right into YaCy which can use a LLM without an external service. This originates from work from Andrej Karpathy who implemented this for OpenAI and from work of Georgi Gerganov who implemented llama.cpp, the engine in Ollama; I took the java version from ttps://github.com/mukel/llama3.java, separated the files and transcibed it from java 23 to java 11 which required to rewrite parts of the matrix multiplication methods. On a fast machine, it computes about 15 tokens per second and this is an importan fall-back if a user does not want an external LLM engine or one is not available.
- a configuration page for LLMs. That is a web page that will be the main switchboard of the AI Lab, it’s just not linked yet. If you want to try it out, its here: http://localhost:8090/LLMSelection_p.html
All of that can only be assembled if we have a strong configuration function which defines a usage matrix between LLM Engines, Models and Usage. That is now almost ready and we can assemble things we have into something, that will for now only be available to the YaCy administrator. So the following things are future functions:
- a chat interface which integrates YaCy search results in the chat results (similar to “Chat with Web Search” in other AI chats)
- a search response tldr-generator (that thing that now appears at every search result i.e. at G**gle) - we first must do experiments with that to find out if we actually like it
- a Map-Reduce - like, LLM-driven Text Collection Generator (please give me time to explain that, it looks like a excel table but with llm functions. Its an experiment, not something I can show now. It not even has a proper name.)
- synthetic text generator for index enrichment. This can be used to enhance ranking or to enhance RAG functions. It is a function that runs during crawling and it may extract information that the user wants to find.
- automatic translation: we can use this to assist the translation of the YaCy interface. It would require manual work but it would be a great way to vastly expand the number of YaCy interface translations.
- truth checker: I know that the word “truth” is terrible (in some way - there is no truth in logic, only consistency), but it can be replaced by anything and everything that the user wants to be checked; then that check result becomes part of the local index and becomes a search facet. Ok, lets rename it simply to “tagger”.
- Any kind of “lets generate this” and “lets combine that”.
So don’t be scared, its a playground. We can play until we find out whats useful.