not generating index urls correctly? about searchkick HOT 26 CLOSED

I was also battling with getting my indexes to build in production. The trick that did it for me was to explicitly set the RACK_ENV=production in the rake task as follows:

bundle exec rake RAILS_ENV=production RACK_ENV=production searchkick:reindex:all

from searchkick.

Talked to @nz from Bonsai and he confirmed that the URL is showing up with $RACK_ENV not being inperolated correctly in the requests to the index.

Also confirmed that Organization.index_name returns organizations_production in console

from searchkick.

Update: Decided to hack it up and this made it work for me:
mehulkar@37c5c72.

Sanity check: is that ENV['RACK_ENV'] variable available in models?

from searchkick.

The ENV variable should be available everywhere, and the fact that Organization.index_name is organizations_production seems to confirm this. I've been in contact with Bonsai about an intermittent bug with aliases, which may be the cause of this error.

In the Rails console, try:

puts ENV["RACK_ENV"]
Tire.configure{ logger STDOUT }
Organization.reindex

And paste the relevant output.

from searchkick.

It looks like reindexing is working fine. On Organization.reindex:

# 2013-08-11 19:12:25:872 [CREATE] ("organizations_production_20130811191224")
#
curl -X POST http://clustername.us-east-1.bonsai.io/organizations_production_20130811191224 -d '{"settings":{"analysis":{"analyzer":{"searchkick_keyword":{"type":"custom","tokenizer":"keyword","filter":["lowercase","snowball"]},"default_index":{"type":"custom","tokenizer":"standard","filter":["standard","lowercase","asciifolding","stop","searchkick_index_shingle","snowball"]},"searchkick_search":{"type":"custom","tokenizer":"standard","filter":["standard","lowercase","asciifolding","stop","searchkick_search_shingle","snowball"]},"searchkick_search2":{"type":"custom","tokenizer":"standard","filter":["standard","lowercase","asciifolding","stop","snowball"]},"searchkick_autocomplete_index":{"type":"custom","tokenizer":"searchkick_autocomplete_ngram","filter":["lowercase","asciifolding"]},"searchkick_autocomplete_search":{"type":"custom","tokenizer":"keyword","filter":["lowercase","asciifolding"]},"searchkick_suggest_index":{"type":"custom","tokenizer":"standard","filter":["lowercase","asciifolding","searchkick_suggest_shingle"]}},"filter":{"searchkick_index_shingle":{"type":"shingle","token_separator":""},"searchkick_search_shingle":{"type":"shingle","token_separator":"","output_unigrams":false,"output_unigrams_if_no_shingles":true},"searchkick_suggest_shingle":{"type":"shingle","max_shingle_size":5}},"tokenizer":{"searchkick_autocomplete_ngram":{"type":"edgeNGram","min_gram":1,"max_gram":50}}}},"mappings":{"organization":{"properties":{},"dynamic_templates":[{"string_template":{"match":"*","match_mapping_type":"string","mapping":{"type":"multi_field","fields":{"{name}":{"type":"string","index":"not_analyzed"},"analyzed":{"type":"string","index":"analyzed"}}}}}]}}}'

[A bunch of db queries]

# 2013-08-11 19:12:35:032 [_bulk] ("organizations_production_20130811191224")
#
curl -X POST "http://clustername.us-east-1.bonsai.io/organizations_production_20130811191224/_bulk" --data-binary '... data omitted ...'

# 2013-08-11 19:12:35:033 [200]

# 2013-08-11 19:12:35:047 [_aliases]
#
curl "http://clustername.us-east-1.bonsai.io/_aliases"

# 2013-08-11 19:12:35:047 [NilClass]

# 2013-08-11 19:12:35:683 [_aliases]
#
curl -X POST "http://clustername.us-east-1.bonsai.io/_aliases" -d '{"actions":[{"add":{"index":"organizations_production_20130811191224","alias":"organizations_production"}},{"remove":{"index":"organizations_production_20130809023625","alias":"organizations_production"}}]}'

Running search again after reindexing gives me the same

Tire::Search::SearchRequestFailed (404 : {"error":"Index not found, and lazy index creation is disabled. Please contact [email protected] for further assistance.","status":404}):

from searchkick.

Glad reindexing is working. How did you solve it? I believe the 404 is due to a bug with aliases on Bonsai.

from searchkick.

Reindexing was working before as well.

Before, the request URL wasn't being interpolated correctly so we had $RACK_ENV in the url. Now, the URL for requests to Bonsai seems correct, but I'm still getting a 404. I switched back to using my fork and search works in production. It doesn't work in development and this is definitely a temporary solution though. cc @nz

from searchkick.

Index and alias creation is pretty solid on Bonsai over the last week and some change.

Included in the earlier gist, logs on our end are definitely receiving a literal "$RACK_ENV" in the request path for searches. Those are causing 404s.

I'm at a loss as to why that would happen for searches through the controller, and yet work fine in the Rails console. Nor do I see anything in the source to indicate a bug in searchkick.

Color me stumped on this one.

—Nick

from searchkick.

@mehulkar it seems like ENV["RACK_ENV"] is set to $RACK_ENV on your system. Can you confirm this isn't the case?

puts ENV["RACK_ENV"]

from searchkick.

I can confirm that ENV["RACK_ENV"] is production. When I switched back to the upstream version of searchkick and deployed, the gem was curling the correct URL (not with the the $RACK_ENV literal). But I was still getting the 404.

Now I'm back on my fork, and the problem is again magically fixed. I'm going to investigate this again by deploying to a staging server tomorrow and get back to you. Don't want to keep pushing broken code to production :/

from searchkick.

I would also check to make sure ENV["RACK_ENV"] is production on your web server since it could be different from the Rails console.

from searchkick.

Hey Mehul, any luck?

from searchkick.

@ankane I've been meaning to push to a staging app on Heroku and test again, but haven't had the time. For now, I'm still using my fork with the hack and it is working. Feel free to close this issue, I'm not sure when I'll be able to test this for sure since I'm not working on the same app right at this moment. Sorry for the delay!

from searchkick.

Thanks for letting me know :) I added fallback RAILS_ENV if RACK_ENV is not set 1ca5a1f and just released a new version.

from searchkick.

Ah very nice. I'll update this weekend and report back.

from searchkick.

Is this issue still happening ? I had to set RACK_ENV with the latest release.

from searchkick.

You'll need to include RACK_ENV=production if your profile does not set it.

from searchkick.

Yes @ankane I have to include RACK_ENV=staging... It's working, thank you. But, I think this could be a fallback to RAILS_ENV.

from searchkick.

It does https://github.com/ankane/searchkick/blob/master/lib/searchkick/model.rb#L6

from searchkick.

Cleaning up issues - let me know if this is still a problem.

from searchkick.

@ankane, I have finally gotten around to this again, and it is still an issue for me:

2013-11-12T02:57:04.417282+00:00 app[web.1]: [REQUEST FAILED] curl -X GET 'http://7i0gred8:[email protected]/organizations_$RACK_ENV/_search?from=0&size=100000&pretty' -d '{"query":{"dis_max":{"queries":[{"multi_match":{"fields":["_all"],"query":"south asian","use_dis_max":false,"operator":"and","boost":10,"analyzer":"searchkick_search"}},{"multi_match":{"fields":["_all"],"query":"south asian","use_dis_max":false,"operator":"and","boost":10,"analyzer":"searchkick_search2"}},{"multi_match":{"fields":["_all"],"query":"south asian","use_dis_max":false,"operator":"and","fuzziness":1,"max_expansions":3,"analyzer":"searchkick_search"}},{"multi_match":{"fields":["_all"],"query":"south asian","use_dis_max":false,"operator":"and","fuzziness":1,"max_expansions":3,"analyzer":"searchkick_search2"}}]}},"size":100000,"from":0,"fields":[]}'

2013-11-12T02:57:04.420804+00:00 app[web.1]: 
2013-11-12T02:57:04.420804+00:00 app[web.1]: RuntimeError (Index missing - run Organization.reindex):
2013-11-12T02:57:04.420804+00:00 app[web.1]:   app/models/organization.rb:69:in `filter_and_search_by_query'
2013-11-12T02:57:04.420804+00:00 app[web.1]:   app/controllers/api/v1/organizations_controller.rb:44:in `search'
2013-11-12T02:57:04.420804+00:00 app[web.1]: 
2013-11-12T02:57:04.420804+00:00 app[web.1]: 

I ran this like @jonaphin suggested:

heroku run rake RAILS_ENV=production RACK_ENV=production searchkick:reindex:all

This was deployed on a brand new heroku app with searchkick v0.3.3

I must be missing a step somewhere.

In heroku console, I can verify that both ENV['RACK_ENV'] and ENV['RAILS_ENV'] are production

from searchkick.

On second look, I am very confused.

I logged ENV['RACK_ENV'] and it ouputs to $RACK_ENV. This is wrong.

But when I look on heroku config, RACK_ENV is set to 'production'

Not sure if this is a heroku configuration or what?

from searchkick.

SOLVED: I updated my Procfile to use RAILS_ENV instead of RACK_ENV. I'm not entirely sure yet what this means, but it solved my issue:

from searchkick.

Your Procfile set the environment to $RACK_ENV instead of production due to the slash at the beginning (this has been the problem all along).

from searchkick.

No idea how/why that was there. Thanks again for the help @ankane!

from searchkick.

No problem, not an easy error to spot.

from searchkick.