When I start crawling, the 12 GB of free space disappear, the disk shows full, and crawling stops

YaCy
1

Disk if full, but after clear cache and restart i see 12 GB free space. And then i start crawling and boom, no free space after 1-3 minutes.

2

With limited information, I cannot claim to know why you are seeing this problem. But if I were you, I would look at your disk settings. Here are some settings that are working for me.

/Performance_p.html

disk
disk1042Ă—366 38.5 KB

3

Is it possible that you are crawling sites with lots of .ISO/.zip/.rar files?
Have a look at “Loader Queue” in the crawling monitor to see if this is the case.

4

It doesn’t matter now. I ordered new VPS with 2TB storage space.

1 Like
5

you could diagnose that using du (disk usage) command line tool, for example:

cd yacy/DATA/
du -hs *
512B	ARCHIVE
512B	CLASSIFICATION
8.1M	DICTIONARIES
3.6G	HTCACHE
 93K	HTDOCS
 71G	INDEX
 48K	LISTS
 17M	LOG
5.0K	RELEASE
 48K	SETTINGS
 81K	SKINS
1.5K	SURROGATES
155M	WORK
cd INDEX/freeworld
du -hs *
 14M	NETWORK
 76M	QUEUES
 71G	SEGMENTS

to see, what is ocuppying the space