Business Idea - How about offer personalized Yacy search URL for a small price per month?

YaCy
1

what if the people who design Yacy also offered for maybe $5-$10/month the ability for a user to add the websites they want indexed every month (up to say 50 or so?) and then Yacy.net has specific servers and does all the indexing and technical work and the user gets a specific, private URL to search thru? This way i dont have to set up all this stuff on my own computer and since you guys are the experts you will know how to make it work well. Maybe a minimum charge of $50 per year to set up a small personalized search engine?

I would be interested in this and i suspect others would be as well

1 Like
2

It might be an interesting project, sort of just for fun, in my spare time.

My “Kiosk” idea is similar, though oriented towards the needs of a town or village rather than an individual.

I like learning, and I could certainly learn something in the process of customizing YaCy to meet someones personal specifications, and probably this might give the developers some idea for making YaCy more “user friendly”.

So my first question would be, how, or in what way would someone interested in this project want to submit URL’s for inclusion in this personal search engine?

Indexing 50 websites a month for $5 to $10 / month seems excruciatingly limited and expensive, considering YaCy is essentially a free/by donation project, and running on your own computer could easily crawl/index hundreds of websites/minute.

I’d recommend making a donation directly to the project: https://www.patreon.com/0rb1t3r

Aside from that. How are the URL’s that a user wishes to have included in his personal search engine to get from the mind of the user to the technician whose job it is to implement this.

As of now, the basic crawler used for indexing a site with YaCy, installed on a computer looks like this:

crawler
crawler.png1366Ă—768 91.8 KB

The “advanced” crawler is a bit more complicated:

It looks like this:

advanced_crawler_1
advanced_crawler_1.png1366Ă—768 100 KB

advanced_crawler_2
advanced_crawler_2.png1366Ă—768 102 KB

advanced_crawler_3
advanced_crawler_3.png1366Ă—768 94.6 KB

advanced_crawler_4
advanced_crawler_4.png1366Ă—768 97.8 KB

advanced_crawler_5
advanced_crawler_5.png1366Ă—768 92.8 KB

It offers a few additional options for customizing the crawl, and is able to handle a LIST of URL’s simultaneously.

There are additional pages where further customization can be made.

For indexing semantics:

semantics_1
semantics_1.png1366Ă—768 91 KB

semantics_2
semantics_2.png1366Ă—768 99.2 KB

semantics_3
semantics_3.png1366Ă—768 92.8 KB

And then there is ranking of the results:

ranking_1
ranking_1.png1366Ă—768 104 KB

ranking_2
ranking_2.png1366Ă—768 103 KB

ranking_3
ranking_3.png1366Ă—768 105 KB

I’ll admit, it can all be a bit daunting and I’ve personally barely begun to explore all the options for tweaking the behavior of the indexer. I’m not certain that this covers ALL the options available, there may be more, but in general, the default setting seem to get the job done.

Actually, Im just now discovering that there is actually much more. several tabs I did not know existed or never opened. My oh my.

I think I could take a dozen more screen shots, but enough for now.

Anyway, do you think that people would want to add the URLs themselves via some interface on a website somewhere, or just send them via email or what would be the preferred method?

1 Like
3

It is possible to schedule the specified website to be crawled/re-indexed on a regular schedule, which could be useful for sites with content that changes frequently, forums, blogs, news sites, employment etc.

Snooping around in the administration area, it seems I haven’t even covered half of the settings that are available.

4

i was thinking that a person might have an account, just log into a website and paste in a bunch of URLs. You might be right though that he average person could figure out how to do this on their own.

5

Did I say that? I thought I was trying to argue the opposite, how complicated it can be.

Anyway, I think that is an excellent idea. I’d actually like it if something were built right into web browsers, Similar to a one click bookmarking.

Screenshot_20210111-060214_crop_61_resize_54

Click the icon to spider this site.

Something like that.

6

This is a great idea. I think, the key issue is about marketing and design. My suggestion would be:

  1. Create a simple “yacy.net supporters” website with a fancy name.
  2. The website should have only the text search field and an small about, language and support link below. And in the top right corner the supporter name if a supporter is logged in.
  3. If someone has supported the website, addition a progress bar is shown that shows the money spend, the money used and the number of pages crawled.
  4. Beside the progress bar is a link, where the supporter can set priorities by adding an URL, changing the order of crawled urls or switch off an URL.

Should I make a design suggestion? Is there anyone who would do the technical setup?

1 Like