Use fast access to postings depending on label matcher type about tsdb HOT 6 CLOSED

Looks like this is still relevant? Can I take this?

from tsdb.

Sure, that would be great.
I think we converged towards adding new matcher types for sets and prefixes to prometheus/tsdb and then add separate functions to actually disassemble a regex and creating set and prefix matcher types out of those.

from tsdb.

So I just got to this... I can see the first part (direct posting lookup for equal matcher) is done. What are "set matchers" supposed to be exactly? Can you point me to the discussion?

Also I assume you mean some simple heuristic for regex prefix extraction? e.g. only optimize regex or the form "^abc" or something like that? I tried to read up on regex index search optimization but couldn't find much info...

from tsdb.

Hey, so it was decided that we should not add optimisations to the regexp-matcher but rather create new matchers and have prometheus parse its queries and send in the correct matcher kind into tsdb. For example if ^abc was given to prometheus, then it will send in NotMatcher(abc).

So one thing you can do is to add a prefix matcher like the Equal and Regexp matchers.

For example, the prefix matcher with the prefix: abc will match the strings abcd, abccc and so on.

Once this matcher is added, we can actually optimise the queries that are like {path~="/users"} in prometheus to detect prefix matching and send it down to tsdb.

Let me know if anything is unclear or if you have any doubts while implementing.

from tsdb.

Should this still be open? Looks like the prefix matcher was added.

from tsdb.

Yes I think can be closed now.

from tsdb.