No relevant resource is found in the selected language.

This site uses cookies. By continuing to browse the site you are agreeing to our use of cookies. Read our privacy policy>Search

Reminder

To have a better experience, please upgrade your IE browser.

upgrade

FAQ – Data Occasionally Fails to Be Imported to the Solr Due to Insufficient Random Ports on FusionInsight

Publication Date:  2019-04-12 Views:  56 Downloads:  0
Issue Description

An error is reported when an application imports data to Solr.

2018-04-10 13:36:56,587 [consumer_5] ERROR    - Request to collection benchmarktest failed due to (400)

org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at

https://122.21.173.47:21110/solr/benchmarktest_shard2_replica1: Exception writing document id 896546283 to the index; possible analysis error., retry? 0 Exception in thread "consumer_5" org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException: Error from server at

https://122.21.173.47:21110/solr/benchmarktest_shard2_replica1: Exception writing document id 896546283 to the index; possible analysis error.

 

      at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:643)

      at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:992)

      at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:881)

      at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:817)

      at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:135)

      at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:107)

      at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:72)

      at org.apache.solr.client.solrj.SolrClient.add(SolrClient.java:86)

      at target.solr.SolrImportConsumer.accept(SolrImportConsumer.java:34)

      at target.solr.SolrImportConsumer.accept(SolrImportConsumer.java:1)

      at benchmark.ConsumerThread.process(ConsumerThread.java:46)

      at benchmark.ConsumerThread.run(ConsumerThread.java:33)

 Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://122.21.173.47:21110/solr/benchmarktest_shard2_replica1: Exception writing document id 896546283 to the index; possible analysis error.
Solution

1. Based on the error information, you can view the specific node and determine the error on the solrserver3 node corresponding to the port 21110.

2. Check the solrserver3 log, and the error information shows in the following figure.


3. The log shows that Solr fails to connect to HDFS DataNode because the operating system does not have sufficient random ports. As a result, data occasionally fails to be imported to Solr.

4. Add the following information to the /etc/sysctl.conf file on each Datanode:
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_timestamps = 1

5. Run the sysctl -p command to make the configurations take effect.

END