Under “Double-Check Rules”, I’ve set: “Treat documents that are loaded 1 Days ago as stale and load them again. If they are younger, they are ignored.”
My crawl has taken longer than one day. Does YaCy ignore this during the initial crawl, or will it keep crawling the pages forever, since crawl time is longer than the stale time?
Not sure with that, can you run an experiment to find out, how it is?
It appears they get re-indexed. I’ve been periodically checking the “First Seen” date on crawled pages, and today I finally found two that were crawled two weeks ago. Even today, most of them are still new, so I guess I’ll be starting the crawls fresh with a longer stale time.