Hi, just out of curiosity:
do you just experiment with YaCy, or anyone uses it for some real project, or getting really useful results? Is anyone using YaCy for some public-availiable web service and with what success?
Time to time, I browse the other peers and wonder, what are the stories behind them. Some are really huge, like ‘sixcooler’ or ‘megaindex’. Some seem narrow-domain oriented, like ‘rechtslupe’, apparently focused on german law. Some language-specific, like french ‘elnath’. In the search query log, I often get queries in german, russian and sometimes french.
What hardware do you use, and are there some technical tricks to keep YaCy running and effective? Does anyone integrate YaCy in other software or using some tailor-made frontends, scripts, etc?
Thanks for sharing!
Приветствую всех!
Моему узлу исполнился один год, я стартовал его вначале февраля 2022.
YaCy находится на удаленном виртуальном сервере, довольно слабой конфигурации: оплатить мощный ресурс денег было мало.
Зачем я занялся всем этим? Если кратко - недовольство существующими поисковыми системами.
В России основными являются Google и Yandex. Это очень крупные системы. Они давно прошли тот период, когда они зависели от посетителей. Сейчас они больше не стараются сделать поиск удобным и эффективным для людей. Я столкнулся с тем, что некоторые интересные и важные сайты отсутствуют в их поисковой выдаче. Эти крупные поисковики показывают в основном тех, кто им заплатит.
Кроме того, в Google сейчас практически невозможен точный поиск, Google выдает не то, что запрошено, даже в кавычках. Это очень плохо!
И этим, к несчастью, грешит и YaCy, надеюсь, что авторы это поправят и улучшат поиск точной фразы.
Yandex весь увешан рекламою, как бродячий пес паршою, и это было бы еще пол-беды. Но последнее время он начал непрерывно требовать доказательства что ты не робот. Буквально невозможно работать с ним. (И я знаю, почему он так: я заблокировал способы слежки за собою - и вот поэтому он выкидывает капчу. А я не хочу, чтоб за мной следили.)
Нет признаков, что ситуация с такими крупными поисковиками начнет улучшаться. Я думаю, что через некоторое время они придут к полному своему абсурду и станут практически непригодными для использования.
Это дает YaCy неожиданный шанс роста востребованности. Хотя YaCy сделана для такой конкуренции откровенно слабо, но я не обнаружил ни одной другой независимой альтернативы.
На настоящий момент YaCy раскручена по русскоязычному Интернету чрезвычайно слабо. В передаче по DHT домен .RU занимает около 0.6%
По этим причинам я решил открыть узел с задачей индексирования именно русскоязычных ресурсов.
Сейчас кроме моего я вижу только три русскоязычных узла.
В процессе попыток настройки и запуска YaCy я понял, что это - капля в море, для такой амбициозной задачи. Но деньги и усилия уже вложены - и я продолжаю начатое.
Общие впечатления от YaCy.
По-моему, не дотягивает даже до альфы. В программе множество ошибок и недочетов. Для того, чтобы заставить YaCy устойчиво функционировать, мне пришлось написать две собственные утилиты, компенсирующие основные недочеты YaCy. Только с помощью этих утилит мой узел продолжает функционировать.
Для этой, всего лишь начальной, наладки без всяких руководств от авторов, понадобилось больше полугода настойчивых усилий.
Для массового запуска YaCy в таком виде абсолютно непригодна, так что без серьезных доработок надеяться на расширение ее популярности не приходится.
Кроме того, многие хостеры ненавидят пиринговый характер YaCy и не позволяют такое использование, что дополнительно создает проблему.
Эффективность.
Мой узел постоянно доступен, но на него никто не заходит, потому что найти там очень мало что можно.
Так что эффективность на практике равна нулю.
Не хотел никого обидеть, просто написал правду, как есть.
Since our announcement (Many encyclopedias now YaCy-searchable at encyclone.org), the Knowledge Standards Foundation has continued to add encyclopedias. We run a very small index, though. Hardware is minimal. Minimal work configuring YaCy, too.
Great, thanks! Would you translate for us?
I am experimenting. I want to use it for personal purpose. My original idea was to let YaCy index all the websites I browse. This way I want to build a search index costumised to my personal needs.
My first approach was YaCy’s http proxy. However, I realised http is phasing out and mostly replaced by https already. As https is encrypted, the YaCy proxy cannot index https pages.
My new idea now is a YaCy browser plugin that could forward https pages to YaCy for indexing. Is there such a browser plugin available or does someone had thoughts into the same direction?
recently i found this firefox plugin for greasemonkey. didn’t try, and last commit is from 2015, but could be the way. or a direction, at least.
Here is a translation based on Google Translate:
Greetings to all!
My node is one year old, I started it at the beginning of February 2022.
YaCy is located on a remote virtual server, with a rather weak configuration: there was not enough money for more powerful resources.
Why did I do all this? In short - dissatisfaction with existing search engines.
In Russia, the main ones are Google and Yandex. These are very large systems. For a long time, they do not to need to attract their visitors anymore. Now they no longer try to make search convenient and efficient for people. I have found that some interesting and important sites are missing from their search results. These big search engines show mostly those who will pay them.
In addition, exact search is almost impossible in Google now, Google gives out not what is requested, even in quotation marks. This is very bad!
In this respect, YaCy also fails, unfortunately. I hope that the authors will correct this and improve the search for the exact phrase.
Yandex is clutted with advertising, like a stray dog with scab, and that would be half the trouble. But lately, it has been continuously demanding proof that you are not a robot. It is literally impossible to work with it. (And I know why it is like this: I have blocked ways to track me - and that’s why it throws out a captcha. And I don’t want to be followed.)
There are no signs that the situation with such large search engines would improve. I think that after a while, they will come to their complete absurdity and become practically unusable.
This gives YaCy an unexpected chance to grow in popularity. Although, frankly spoken, YaCy is poorly made for such competition, I have not found any other independent alternative.
At the moment, YaCy is extremely poorly promoted on the Russian-speaking Internet. In DHT transmission, the .RU domain occupies about 0.6%
For these reasons, I decided to open a node with the task of indexing specifically Russian-language resources.
Now, in addition to mine, I see only three Russian-language nodes.
In the process of trying to set up and run YaCy, I realized that this is a drop in the bucket for such an ambitious task. But money and efforts have already been invested - and I continue what I started.
General impressions of YaCy:
In my opinion, it has not even reached the alpha phase. The program has many errors and shortcomings. In order to get YaCy to run smoothly, I had to write two of my own utilities to compensate for YaCy’s major shortcomings. It is only with the help of these utilities that my node continues to function.
For this, just initial, adjustment without any manuals from the authors, it took more than six months of persistent effort.
YaCy in this form is absolutely unsuitable for a mass launch, so there is no reason to hope for an expansion of its popularity without serious improvements.
Also, many hosts hate the peer-to-peer nature of YaCy and don’t allow such usage, which is a further problem.
Efficiency:
My site is constantly available, but no one visits it, because there is very little to find there.
So the efficiency in practice is zero.
Didn’t mean to offend anyone, just wrote the truth as it is.
Hi Sviatoslav,
I agree with your opinion about the big search engines. Google is very obviously getting worse. Bing seems the same. It looks like they are playing arround with their algorithem. Google is absolutely not reliable anymore. We need something we can rely on.
You have mentioned two utilities you have written by yourself to run YaCy in a stable way. Could you share and explain these utilities? Are they helpful/needed for YaCy in general or just for your specific configuration?
I’d be also curious to see these utils!
Me personally, I was struggling with YaCy getting stuck every few days, and wrote a cron script to restart it every day. YaCy works continuously since then.
Hi all!
I think it will work out better if any of you use the translation service to translate directly into the language you need,
than I will translate first into English, and from it you into your own.
https://translate.yandex.com/?utm_source=main_stripe_big&source_lang=ru&target_lang=en
I posted about the first utility here:
Работа Яси на слабом ресурсе - #3 by Sviatoslav - a post from Dec 2022.
I don’t mind sharing this utility for free, but the forum doesn’t allow attachments.
If any of you can place it in a permanently public place, I will be happy to provide it.
The second utility supports my individual peername (when it turned out that it could be lost), and therefore would not be useful for general use.