Allow fast access to most recent sample about tsdb HOT 6 CLOSED

With atomicity and staleness handling, which I hope to get in with the new storage for 2.0, I can see this requiring more involved logic which makes considerations around the current implementation largely moot.

Let's get something working, and then benchmark.

from tsdb.

Federation does work modulo #10 which can cause duplicate entries for overlapping selectors right now. So yes, this is is a low priority issue for now.

from tsdb.

Just two thoughts at this point (not much time to dig deeper):

• The fast access path for the last sample was not only motivated by federation but also by the fact that rule evaluation are often just accessing the last sample of a series.

• Given that allocation churn of the query layer seems a problem right now, I could imagine that a low-alloc access path to the last sample would help here (note that many advanced setups have a huge amount of rule evaluations going on all the time). (This is based on naïve code reading if how the iteration for for federation works. Might be all wrong.)

from tsdb.

The fast access path for the last sample was not only motivated by federation but also by the fact that rule evaluation are often just accessing the last sample of a series.

To my knowledge rule evaluations are just regular queries and do not make us of the particular storage call. The query engine does not use it internally either.
I'd argue that almost all rule evaluations that are relevant in cost, use time aggregation anyway and it doesn't apply.

Given that allocation churn of the query layer seems a problem right now, I could imagine that a low-alloc access path to the last sample would help here (note that many advanced setups have a huge amount of rule evaluations going on all the time). (This is based on naïve code reading if how the iteration for for federation works. Might be all wrong.)

Not so much of a problem. Just way higher than it would need to be. The allocs don't relate so much to access but to what the QE allocates itself. v2 also allocates internally when being queried, but that's not happening in v3.

from tsdb.

Storage v2 transparently returns an "instant" series iterator for an instant query with a timestamp beyond the end of a series.

But yes, range queries obviously cannot benefit from that.

from tsdb.

As per reports federation actually works faster already and there's currently no need for adding complexity of the fast access.

from tsdb.