Is there an easy way for crawling the local filesystem (folder, subfolders, files) without having to provide a sitemap or html-based index file?
An out-of-the-box solution would be greatly appreciated.
Kind regards,
Chris
Yes. You must be in Intranet mode, then start urls in the shape of file:// are possible.
I.e. if you are on a linux file system and you want to crawl /home/admin/ then the start url would be
file:///home/admin/
(triple “/” !)
However, this results in mixed experience when clicking on a search result depending on the browser. Some understand the file url, some not. In most cases they cause a Download of the document (from local file system) what is not what you probably want. There had been browser extensions to change the behavior, but it has been a long time ago since I tried that.
Hi, I want to crawl my local copy of the Gutenberg 16, with all subdirectories.
I tried the local search of file:////home/rob/dataspace/gutenberg-edition16/
and also the Expert Crawling with increased depth (5), but no success; only 4 documents are found. How can I start the search of the complete directory sub-tree?