mongo_connector breaks down when trying to import more than 34000 records about mongo-connector HOT 4 CLOSED

I just tested this and was able to insert 50,000 documents fine, so I'm guessing it's a problem with your solr config. Can you also test with this alternative solr doc manager and tell me if you get the same results? https://github.com/thomascirca/mongo-connector-solr-doc-manager/blob/master/solr_doc_manager.py

from mongo-connector.

I tested with alternative solr doc manager and it works (after modification).

I cloned alternative solr_doc_manager repository and installed it as shown in readme.

Modification:
in alternative solr_doc_manager.py I've changed this code

from mongo_connector.util import verify_url, retry_until_ok

to this

from util import verify_url, retry_until_ok

because of the error:

INFO - Beginning Mongo Connector
Traceback (most recent call last):
  File "mongo_connector.py", line 474, in <module>
    main()
  File "mongo_connector.py", line 459, in main
    auth_username=options.admin_name)
  File "mongo_connector.py", line 53, in __init__
    doc_manager = imp.load_source('DocManager', doc_manager)
  File "/opt/solr/dm/solr_doc_manager.py", line 32, in <module>
    from mongo_connector.util import verify_url, retry_until_ok
ImportError: No module named util

It imports too slow, but correct. I've changed auto_commit to false in upsert method of alternative solr_doc_manager, but when import finished, I had to do a manually request http://localhost:8983/solr/update/?commit=true. And over 52200 records were in solr. Cool!

from mongo-connector.

@buskar Yes sorry about that import- that was assuming a different version of mongo-connector (which should be released soon). glad it worked for you! Can I close the issue?

from mongo-connector.

Thank you for your help.

from mongo-connector.