Is it possible to pre-load passages from csv? about spago HOT 8 CLOSED

Thanks for sharing your project. It's really ambitious!

I've been extremely busy these days so you'll have to wait a few more days for the text-similarity demo.

Ps. about Go, I feel it as if it was custom made for me, that is the exact opposite of my feelings towards Python 🤫

from spago.

Dear Sven,

Yes, it's pretty immediate to use spaGO to get the goal you want.

Between tonight and tomorrow, I'll add an example on spaGO for a tiny FAQ search using a text similarity technique to semantically compare a user's question with a "canonical" FAQ question.

I'll let you know as soon as I'm done here.

In detail, I will use the BERT encoder to load LaBSE (Language-agnostic BERT Sentence Embedding) model.

If you don't know it, it's a powerful tool originally proposed by Google (2020) that encodes texts in a shared embedding space. It allows to determine how 'close' two pieces of text are both in surface and meaning. You can even compare texts in different languages, resulting in an inherently multilingual FAQ system!

Text similarity can be combined with Question-Answering techniques (SQuAD2 models and alike) to extract detailed answers from passages as well.

Spoiler
I'm working on a project called SuperFAQ (https://www.superfaq.ai/). SuperFAQ does exactly this, but thought for production use, relying on a high-performance vector similarity database, HTTP/gRPC API, and much more. We'll open-source it at SpecializedGeneralist as soon as it works well.

I tried to upload on SuperFAQ some questions for fun from go-faq.
I often change datasets (I'll put the FAQ on COVID-19 again soon), so hurry up if you want to try it :)

Please note that not all the questions were uploaded because of time-out errors or something else that I haven't investigated now. SuperFAQ is just born, we are not even at the alpha version but I think that's enough to get the idea!

from spago.

That is pretty impressive. I tried SuperFAQ it works really well :-)
Although I can not just share the project just yet as it is not on a state I am satisfied with I want to just paste an image here so you can get an impression of what I am doing.
STT/TTS (it would be amazing if you'd consider contributing your voice too - Italy needs to be heard!!!)

I will check it tomorrow. I did not know about LaBSE till now. Much thanks for your kindness

Your go is quite good actually! You mentioned you started go with this project on youtube :)

from spago.

totally understandable. If we could just do what we want he :)
Ps as the saying goes: if you do not like programming you should try python ahahaha.
Have a nice day today!

from spago.

I think spaGO is ready for a release tag too so one can reliably manage it through go mod :)

from spago.

@go-dockly feel free to open a separate issue for that (release tag/versioning) so we can jointly discuss with other members.

Thanks

from spago.

@go-dockly I've made a repository with an example that's close to your question on the occasion of my talk at GopherCon Europe 2021!

You can look at this link and ask any additional questions or close this issue if you wish.

from spago.

Ciao Matteo
Thank you for getting back to me on this!
I tried out the text similarity example and indeed it works really well - bit creepy even how good.
Blows Bayes right out the window ;)
Pity I did not see it earlier but good things come to those who wait.
I picked up the old project and it is in a pretty good state.
I wonder what to do with it now. I will get back to you on this but the issue can certainly be closed :)

from spago.