deanishe / alfred-index-demo Goto Github PK

So, I'm well on my way in my ground-up rewrite of ZotQuery. I have completely rewritten the data model, and I'm now using sqlite FTS as the search backend. You were right, the speed difference is noticeable. There is, however, one drawback. I cannot search with an ASCII string (e.g. metaphysique) and get a result for a Unicode string (e.g. métaphysique). I know that sqlite can store Unicode (I see that you store data in Unicode here), but is there any way to mimic Alfred-Workflow's folding functionality with sqlite?

It seems to me that there is really only one option, and that would be to "unidecode" to ASCII before storing the text in the FTS database. That way, everything is in ASCII in the database, so ASCII searches would work. However, this would then conversely kill the ability to search using Unicode queries. Is there any other possible solution? Should I force one way over the other? If so, which one?

PS. I am using "Unicode" merely as a standin for all characters above 128. I am aware that ASCII is a character encoding standard, and Unicode is a character representation.