Would like to understand

YaCy
1

Hello everyone,

I am very interested in the YaCy project. But, i can’t understand the system architecture/design. Moreover, I installed it on my windows but was not able to run it… Can someone point me towards some documentation or youtube videos to understand the code and the architecture ? Im a engineering students and i did small python projects, but i am not used to reviewing githubs projects.

One of my question is what happens when a query is sent ? Several nodes receives the query and sent their result, and then the client agregate it ? What is a node here, can it just be someone’s personal computer being on ? Is the whole index being tracked?

I would love to contribute to the project. Thanks in advance

1 Like
2

Welcome @FredoDoBronx!

Great to see an engineering student interested in understanding and contributing to YaCy.

Your idea of how a search works is already quite close.

A YaCy installation is a peer (node). A peer can simply be someone’s personal computer or server running YaCy and connected to the YaCy network. There isn’t one central server containing the complete search index.

When you perform a search, your own peer searches its local index and can also send the query to other YaCy peers. Those peers search the index data available to them and return results. Your peer then combines the results and presents them to you.

So conceptually you can think of it as:

                         Your YaCy Peer
                              |
                    Search: "example query"
                              |
             +----------------+----------------+
             |                |                |
         Local Index       Peer A           Peer B
             |                |                |
          results          results          results
             |                |                |
             +----------------+----------------+
                              |
                       combine / rank
                              |
                         Search Results

The distributed index is one of the interesting parts of YaCy. No individual peer needs to contain the entire web index.

If you’re interested in the code, don’t worry about understanding the whole GitHub project first. YaCy is a large Java project and it can look intimidating when you first open it.

I would start by getting YaCy running on Windows, performing some searches and crawls, and watching what happens in the log. Once you’ve seen the crawler, local search and remote search operating, the corresponding Java code becomes much easier to understand.

For your Windows problem, tell us what happens when you try to start YaCy. If possible, post the error or console output and your Java version:

java -version

There are people here who can help you get the peer running.

And please don’t be put off by only having experience with small Python projects. Looking through an established project like YaCy is a very good way to learn.

Welcome to YaCy!

1 Like