YaCy Support GPT (ChatGPT Assistant)

I created a ChatGPT “GPT” to be able to support YaCy users:

This GPT has a copy of all public YaCy information:

  • the complete source code
  • a text extract of the old wiki (sql dump)
  • a text extract of the old YaCy blog (sql dump)
  • a text extract of the old forums (sql dump)
  • a text extract of the new forums (this one)
  • the YaCy home page
  • the full (almost) 20 year long commit history (commit messages only)

All dumps have been cleaned from personal information.

I tried this using a part of the thread dump from an issue in When peer is overloaded with a search queries. Thread dump. · Issue #645 · yacy/yacy_search_server · GitHub and it gave me valuable information:

Maybe this works because it is not even a YaCy problem, more a Solr problem. However, all the simple questions (how to install YaCy, how can I do this and that) can also be answered by this GPT.

Now, I don’t know if anyone of you have a OpenAI account or want to have one. For me this is a very good proof of concept and a motivation to implement the same functionality inside YaCy as well. Using the latest proof-of-concept of adding RAG in YaCy (see Added a RAG Proxy for AI Chat with YaCy · yacy/yacy_search_server@13fbff0 · GitHub) we could work on a YaCy Expert using YaCy.

The YaCy GPT is public, so please try it out and tell me if this is useful for you!

4 Likes

Error it Requires ChatGPT Plus?

Well, unfortunately yes. I did not know this! I hope that OpenAI will change their policy.
However, I can’t imagine a future where we will not be able to provide such a function ourself!

1 Like

Wow, interesting!

And, Oh The Oracle’s Master, could you ask the Great Oraculum for solving this issue? My prayers go only in this direction. May the consciousness be with you!

Well, I tried. The GPT responded with a very very long answer. One would have to go into detail and find how much hallucination is in the answer, but partly it looks good:

1 Like

I just got an email from OpenAI: GPTs are now available for all users, not only ChatGPT Plus:

Hi there,

We’re reaching out because you’ve built a GPT and shared it in the GPT store.

We recently announced that all ChatGPT Free users would soon be able to use GPTs in the GPT Store, in line with our mission to make AI technology accessible to all. Today, GPTs in the Store are now available to all ChatGPT users, allowing more than 100 million weekly active users to experience your GPT and the others in the store.

GPTs now use our newest flagship model, GPT-4o. Given the possibility of increased usage and behavioral changes with a new model, we recommend reviewing and testing your GPT to ensure it continues to perform as intended.

Thank you for building GPTs and introducing new ways to learn, have fun, and be more productive with ChatGPT.

– The ChatGPT Team

Here is the blog post: https://openai.com/index/gpt-4o-and-more-tools-to-chatgpt-free/

1 Like

I have tried it again today and I think It’s totally awesome.

1 Like

Hi, did the change of that document cache really change performance for anyone?
I found since solr 9.0 the load got quite high and the performance was super low.
In the meantime I found this was becuse the autowarm count for the filter cache got set to 128, which results in autowarming after each commit into the index, which is almost all the time…
Setting it beack to ‘0’ made the the performance being ok again.

And the patch for setting the docCache looks really ugly to me, while there is a clean config in solrconfig.xml.
@Orbiter I hesitate to revert your commit, but it really should :slight_smile:

1 Like

I admit the change was ugly but looked like the only thing that would force an effect. I do not know if the solrconfig is actually read by the code because we have an embedded solr. It’s also scary that my changes are actually reviewed in that intensity, however thats a good thing. So what should we do?

I’m pretty sure the solrconfig.xml is read.
Reducing the the autwarmcount really made a change - I’ll commit a bit later this day…

2 Likes

Definitely not a halucination, seems reasonate. However, based on what I previously wrote in the forum and github issues.
Do you think it’s possible to implement?

As a new user, I read over the forum, git etc.

When I came across this post I tried it out and wow great experience. I would recommend anyone that has a question to give it a go.

@sixcooler, do you use a standalone solr?

HI @okybaca ,
nope - I implemented the update solr servlet part ‘under the hood’.
One Instance get /solr/collection1/update at DATA/web.xml commented out
and others connect to that host just like to an standalone solr.

But be warned: there is no Authorization other than limitation to requests from intranet.

1 Like

Oh, I see, thanks!

I noticed, you run 5 biggest nodes in the network, so I’m interested in architecture. So you have one solr-running YaCy instance and the remaining ones are on the same machine, sharing the index?

I noticed also your search is quite fast. How do you do that? I use a standalone solr, but there is still a struggle with speed for me.