Elastic search implementation

ojophilip
Ojo Philip Odeniyi
• 22 Aug

I was going through the documentation and I noticed that you recommended using elastic search rather than the pre-configured woosh as search engine. Pls sir, how do I go about this? Do I need to write new codes for it? Awaiting your response. Thanks very much.

nitely
Esteban Castro Borsani

You need to install it, and then configure it. Refer to the elastic website to learn how to install it.

Whoosh may work just fine for you though. It depends on the size of your forum (number of topics and replies) and active users. If the search gets slow you can always switch to elastic search later.

ojophilip
Ojo Philip Odeniyi
• 22 Aug

Thanks for the prompt response☺

ojophilip
Ojo Philip Odeniyi

Hey bro, default search feature not working, just displaying there are no results. Can you pls suggest why this is happening? Dependencies have been installed. Thanks

nitely
Esteban Castro Borsani
• 1 Sep

There are a few ways to keep the search index updated: for low traffic sites you can add HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor' to your django settings; another way is to run the manage.py update_index --age=1 command periodically using a cron-job (if you run it every hour, set --age=1; if you run every 2 hours, set --age=2, etc) ; another way is to use celery or some other task manager to update the index.

I should've added this to the docs, sorry.

ojophilip
Ojo Philip Odeniyi
• 1 Sep

Thanks, I will try this out. I would have loved to help in doing some things but I'm relatively new in programming

ojophilip
Ojo Philip Odeniyi
• 1 Sep

@nitely said:
There are a few ways to keep the search index updated: for low traffic sites you can add HAYSTACK_SIGNAL_PROCESSOR = 'haystack.signals.RealtimeSignalProcessor' to your django settings; another way is to run the manage.py update_index --age=1 command periodically using a cron-job (if you run it every hour, set --age=1; if you run every 2 hours, set --age=2, etc) ; another way is to use celery or some other task manager to update the index.

I should've added this to the docs, sorry.

Well I've added that into the setting and also run the index command which showed indexing 0 topics. I ran my server but still showed there are no search results. Pls, what else do you suggest?

nitely
Esteban Castro Borsani

Did you restart the server after adding the setting? That setting should work for new topics / comments, but it won't index old ones. Same for the update_index command, unless you remove the --age parameter. Try running manage.py update_index.

Another thing you can try is to create a new topic and check whether it shows in the search results.

If everything fails, try running manage.py rebuild_index that'll clear the index and index every topic.

ojophilip
Ojo Philip Odeniyi

I didn't restart the server at first but when it didn't work, I had to stop the server inorder to run the update_index and afterwards restart it.
OK, I will try this and get back to you. Thanks

ojophilip
Ojo Philip Odeniyi
• 2 Sep

It's working fine now. Thanks very much😊😊😊. But pls, how can this be run automatically in production?

nitely
Esteban Castro Borsani

I don't know which option worked for you. Honestly, all of the above should have just worked. Ok, the settings does require a server restart, but after that it should show results for new topics.

The update_index should have worked fine without the "age" parameter, and with the "age" parameter it should index topics created within the "age" time span.

So, assuming the setting works, you don't need anything else in production. If you prefer the update_index command, then you've to setup a cron-job or use a task manager that runs the command (ex: celery).

There's one caveat, the setting only works with single server instances. If you have more than one server then it won't work well, because it will only update the index for the server that received the request.

There may be something else going on as well.

ojophilip
Ojo Philip Odeniyi
• 2 Sep

Thanks for the reply, I've taken note of those option and will work on it. But it's working well in development now

nitely
Esteban Castro Borsani
• 13 Sep

FWIW, I just released Spirit v0.9.0 and it contains support to run the search index update as an asynchronous task. It requires either Huey or Celery. I included some docs as well.

ojophilip
Ojo Philip Odeniyi
• 18 Sep

@nitely,
Wow! Thanks very much will check it up

ojophilip
Ojo Philip Odeniyi

However I'm getting this error in the logger when I deployed my app to pythonanywhere. Myproject.spirit_cache does not exist can you pls help me with it?

The domain is ojophilip.pythonanywhere.com

nitely
Esteban Castro Borsani

Follow the Spirit install instructions. Did you run python manage.py spiritinstall? Is manage.py using the same settings as wsgi.py or whatever you are using to run the server?

ojophilip
Ojo Philip Odeniyi

@nitely, I didn't run spirit command in production, I just ran manage.py makemigrations and manage.py migrate. I will try running the command.

Should I remove spirit_topic_poll from install apps as it gave error during migration in production? I had to fake its migration

Also, I saw something online like running manage.py createcachetable for db caches. Will spirit install command do all these?

ojophilip
Ojo Philip Odeniyi
• 16h

Thanks very much, I will still need more when I finally want to deploy to digital ocean. Thanks once more for responses

nitely
Esteban Castro Borsani
• 16h

Should I remove spirit_topic_poll from install apps as it gave error during migration in production, I had to fake its migration

spirit.topic.poll was removed ages ago in Spirit 0.6, not sure why would you have it installed.

Also, I saw something online like running manage.py createcacheable for db caches. Will spirit install command do all these?

Yes.

ojophilip
Ojo Philip Odeniyi
• 14h

@nitely, thanks I will do that and get back to you