Hi community,
since a few days, I have the following errors in Yacy logs:
W 2023/10/04 06:04:12 net.yacy.cora.federate.solr.connector.SolrServerConnector * Exception writing document id TlAkphpOk_ejTlAkphpOk_ej31eff16c to the index; possible analysis error: number of documents in the index cannot exceed 2147483519 IDs=[TlAkphpOk_ejTlAkphpOk_ej31eff16c, TlAkphpOk_ejAjdlAZJnAEZjed046e09, TlAkphpOk_ejJIPUfZJnAEZjed046e09, TlAkphpOk_ejJrpaGlpOk_ej00000000, TlAkphpOk_ejOc8GdvpOk_ejd9d1dd0c, TlAkphpOk_ejDXuIUvpOk_ejffe7f4a6, TlAkphpOk_ej8p3dUvpOk_ej5ccfc659, TlAkphpOk_ejXUuemvpOk_ej4b7fd73a, TlAkphpOk_ejOIYKZvpOk_ej48e3b38b, TlAkphpOk_ejexzY0vpOk_ej04b2f17c, TlAkphpOk_ejoIjPjvpOk_ejd980f38b, TlAkphpOk_ejEg9FAvpOk_ej852f37f9, TlAkphpOk_ejlEhHPvpOk_ejd7dce7dc, TlAkphpOk_eji-VuOvpOk_ejf9d0661d, TlAkphpOk_ejHBZYnsYa79Z5b0194d5a, TlAkphpOk_ejkAHaXlpOk_ej4981d7c6, TlAkphpOk_ejiXDs1zK388wgcd2d2e5f, TlAkphpOk_ejrsK1ylpOk_ej5ccfc659, TlAkphpOk_ejwB0FNWpOk_ej9e89708d, TlAkphpOk_ejAmMfCWpOk_ejb4535d44, TlAkphpOk_ejlHPZaWpOk_ej8ccf87df, TlAkphpOk_ejuGqUAWpOk_ej6ef8d0cc, TlAkphpOk_ejASBMGvpOk_ej2ba75c02, TlAkphpOk_ejYELYbWpOk_ejd2185596, TlAkphpOk_ejcqtMIvpOk_ej0672a839, TlAkphpOk_ejbBBFVvpOk_ejdf28064c, TlAkphpOk_ejixRwEvpOk_ej8f19e9a9, TlAkphpOk_ejBixErvpOk_ej1ca1c21d, TlAkphpOk_ej8xOB6gpOk_ej41f6d2a4, TlAkphpOk_ejOcQMfvpOk_ejcf580237, TlAkphpOk_ej_m6dcWpOk_ej6114a785, TlAkphpOk_ejKp75UvpOk_ejeca285a1, TlAkphpOk_ejqMFAYlpOk_ej5d4912eb, TlAkphpOk_ejrhkG-lpOk_ej01855b2d, TlAkphpOk_ejWJj3MlpOk_ej1601604f, TlAkphpOk_ejYE-EYlpOk_ej4b7fd73a, TlAkphpOk_ejCH6dKlpOk_ejd980f38b, TlAkphpOk_ejcRxFUlpOk_ej8ccf87df, TlAkphpOk_ejXdEWqlpOk_ej0f964362, TlAkphpOk_ejk9E40lpOk_ej16bc5fa6, TlAkphpOk_ej2YzNQlpOk_ejfd5ae7f9, TlAkphpOk_ejjiwwflpOk_ej486e2c30, TlAkphpOk_ejfC_TelpOk_ej88680f23, TlAkphpOk_ej_j8ddlpOk_ejc5b068cb, TlAkphpOk_ejQu1v0opOk_ej57c9493a, TlAkphpOk_ejefR8EopOk_ejdae64def, TlAkphpOk_ejmuLfZlpOk_eja1b1ec8d, TlAkphpOk_ejx6SiWgpOk_ej6b86482f, TlAkphpOk_ejsMF9lgpOk_ejf9bc5a4b, TlAkphpOk_ejDtiPJMpOk_ej38557d8d, TlAkphpOk_ejuWVPkMpOk_ej4fdf9175, TlAkphpOk_ejEd94nMpOk_ejb889e0d9, TlAkphpOk_ejS--_xlpOk_ej95402fe5, TlAkphpOk_ejvCRPQlpOk_ej046c7fc4, TlAkphpOk_ejRYVxAgpOk_ej0462ff49, TlAkphpOk_ejNx0KHlypSIe400000000, TlAkphpOk_ej3sISFsBGIES400000000, TlAkphpOk_ej7Av9ShpOk_ej8fc2e054, TlAkphpOk_ejFykb_fpOk_ej00000000, TlAkphpOk_ejSO_AUfpOk_ej00000000, TlAkphpOk_ejDvp2WfpOk_ej00000000, TlAkphpOk_ejt7aMjASsUd8D1c1fff18, TlAkphpOk_ejvZrXl1vGUUug00000000, TlAkphpOk_ej-CV5wENC-JqQf4ada124, TlAkphpOk_ejHCe5WKB3IKGB1f0e978c, TlAkphpOk_ejqamW2esD5tABa2f76b0f, TlAkphpOk_ejrE1YQATmFmnA78c25ab3, TlAkphpOk_ejyet9SA52zHMAf6c8c1e2, TlAkphpOk_ejECoC3fpOk_ej00000000, TlAkphpOk_ejNE75ZvpOk_ej00000000, TlAkphpOk_ejJvt_WhpOk_ej00000000, TlAkphpOk_ejL8x3ohpOk_ej00000000, TlAkphpOk_ejwPB-yvpOk_ej00000000, TlAkphpOk_ejJLauUhpOk_ej00000000, TlAkphpOk_ej8rts6hpOk_ej00000000, TlAkphpOk_ej374dDhpOk_ej00000000, TlAkphpOk_ejdLElehpOk_ej00000000, TlAkphpOk_ejEYTDrhpOk_ej00000000, TlAkphpOk_ejIr5TyhpOk_ej00000000, TlAkphpOk_ejuNKoThpOk_ej07bd9b3e, TlAkphpOk_ej69gaYc9A0pF4d3c11831, TlAkphpOk_ejfMz94hpOk_ejc4655bc3, TlAkphpOk_ej3BQ79hpOk_ejc4655bc3, TlAkphpOk_ej6Uqf6vpOk_ejbd967178, TlAkphpOk_ejmLO6pvpOk_ejb2d42da4, TlAkphpOk_ejD4RtPvpOk_ej11b5ef21, TlAkphpOk_ej3VNF1vpOk_ej68f81df8, TlAkphpOk_ejDN6mTvpOk_ej98c4465c, TlAkphpOk_ejSSRFivpOk_ej1c9b16b4, TlAkphpOk_ejU6mmuvpOk_ej5eae2e4b, TlAkphpOk_ejw6AwU7pOk_ej66a7dd06, TlAkphpOk_ejQo8fKvpOk_ej70e6a274, TlAkphpOk_ejg57e6vpOk_ejb8e2cc65, TlAkphpOk_ejAM7paPpOk_ej00000000, TlAkphpOk_ejoOcYzvpOk_ej37eb7ec8, TlAkphpOk_ejOBUmgvpOk_ej0ec62036, TlAkphpOk_ejZAHYevpOk_ej74de673f, TlAkphpOk_ejbmDM1vpOk_ejee70d4f7, TlAkphpOk_ejeO-Lf7pOk_ej524391a5, TlAkphpOk_ejD0JIj7pOk_ejdddf29ca, TlAkphpOk_ej7QeUkhpOk_ejfc160573, TlAkphpOk_ej7afhIhpOk_ej5d1b63fd, TlAkphpOk_ejHuUCPvpOk_ej00000000, TlAkphpOk_ejbrtTGypOk_ej3da9c0f7, TlAkphpOk_ejhevBrypOk_ej6dab1e88, TlAkphpOk_ejrjbUmypOk_ejcc6c771c, TlAkphpOk_ejhjA7qypOk_ejed15aa88, TlAkphpOk_ejwNxpCypOk_ejfdf25d34, TlAkphpOk_ejyx3wZypOk_ej95c1b575, TlAkphpOk_ejYXdcjPpOk_ejd8ebfcb7, TlAkphpOk_ejT6JDBMpOk_ej13cf8b05, TlAkphpOk_ej0FDR-MpOk_ej04d1cbcb, TlAkphpOk_ejBybmqoMPDumja1a7c0db, TlAkphpOk_ejjgfUZlMPDumj44946131, TlAkphpOk_ejtnHeFlMPDumjd8ebc5fe, TlAkphpOk_ejDfOsUl6JQkag750b256d, TlAkphpOk_ej4MM7rlQcH5Jhfc1e7727, TlAkphpOk_ej62JLWuMPDumj2dcf9317]
W 2023/10/04 06:04:12 SOLR * failed to send https://example.com/ to solr: org.apache.solr.common.SolrException: Exception writing document id TlAkphpOk_ejTlAkphpOk_ej31eff16c to the index; possible analysis error: **number of documents in the index cannot exceed 2147483519**
I 2023/10/04 06:04:13 org.apache.solr.update.processor.LogUpdateProcessorFactory [webgraph] webapp=null path=/update params={}{} 0 0
E 2023/10/04 06:04:13 org.apache.solr.handler.RequestHandlerBase org.apache.solr.common.SolrException: Exception writing document id TlAkphpOk_ejTlAkphpOk_ej31eff16c to the index; possible analysis error: **number of documents in the index cannot exceed 2147483519**
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:249)
at org.apache.solr.update.processor.RunUpdateProcessorFactory$RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:73)
at org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:55)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:263)
at org.apache.solr.update.processor.DistributedUpdateProcessor.doVersionAdd(DistributedUpdateProcessor.java:502)
at org.apache.solr.update.processor.DistributedUpdateProcessor.lambda$versionAdd$0(DistributedUpdateProcessor.java:343)
at org.apache.solr.update.VersionBucket.runWithLock(VersionBucket.java:50)
at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:343)
at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:229)
at org.apache.solr.update.processor.LogUpdateProcessorFactory$LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:106)
at org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:110)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:343)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readIterator(JavaBinUpdateRequestCodec.java:291)
at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:338)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:283)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$StreamingCodec.readNamedList(JavaBinUpdateRequestCodec.java:244)
at org.apache.solr.common.util.JavaBinCodec.readObject(JavaBinCodec.java:303)
at org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:283)
at org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:196)
at org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:131)
at org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:122)
at org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:70)
at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)
at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:82)
at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:216)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:2637)
at org.apache.solr.client.solrj.embedded.EmbeddedSolrServer.request(EmbeddedSolrServer.java:227)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:214)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:106)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:71)
at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:85)
at net.yacy.cora.federate.solr.connector.SolrServerConnector.add(SolrServerConnector.java:260)
at net.yacy.cora.federate.solr.connector.MirrorSolrConnector.add(MirrorSolrConnector.java:210)
at net.yacy.search.index.Fulltext.putEdges(Fulltext.java:386)
at net.yacy.search.index.Segment.storeDocument(Segment.java:676)
at net.yacy.search.Switchboard.storeDocumentIndex(Switchboard.java:3505)
at net.yacy.search.Switchboard.storeDocumentIndex(Switchboard.java:3419)
at net.yacy.search.Switchboard.lambda$new$0(Switchboard.java:1057)
at net.yacy.kelondro.workflow.InstantBlockingThread.job(InstantBlockingThread.java:72)
at net.yacy.kelondro.workflow.AbstractBlockingThread.run(AbstractBlockingThread.java:82)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.lang.IllegalArgumentException: **number of documents in the index cannot exceed 2147483519**
at org.apache.lucene.index.DocumentsWriterPerThread.reserveOneDoc(DocumentsWriterPerThread.java:186)
at org.apache.lucene.index.DocumentsWriterPerThread.updateDocuments(DocumentsWriterPerThread.java:207)
at org.apache.lucene.index.DocumentsWriter.updateDocuments(DocumentsWriter.java:415)
at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1471)
at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1464)
at org.apache.solr.update.DirectUpdateHandler2.updateDocOrDocValues(DirectUpdateHandler2.java:967)
at org.apache.solr.update.DirectUpdateHandler2.doNormalUpdate(DirectUpdateHandler2.java:342)
at org.apache.solr.update.DirectUpdateHandler2.addDoc0(DirectUpdateHandler2.java:294)
at org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:241)
... 44 more
Is there a workaround for this? Or can the limit of 2147483519 be increased?
Thanks in advance
Iso