Simple Global Secondary Index Query Support about meteor HOT 8 CLOSED

Interesting, I didn't know that there is no uniqueness guarantee on GSI. Changing the signature of retrieve to Stream[F, T] is a bit weird since SimpleTable and local version of SecondarySimpleIndex use F[Option[T]] and it fits a bit better in those cases. IMHO, the hierarchy should be:

sealed abstract class SimpleIndex[F[_]: Async, P: Encoder] {
  ...
  def retrieve[T: Decoder](
    query: Query[P, Nothing],
    consistentRead: Boolean
  )(implicit RT: RaiseThrowable[F]): F[Option[T]]
}

case class SimpleLSI[F[_]: Async, P: Encoder](
  tableName: String,
  indexName: String,
  partitionKeyDef: KeyDef[P],
  jClient: DynamoDbAsyncClient
) extends SimpleIndex[F, P] {
  ...
}

case class SimpleGSI[F[_]: Async, P: Encoder](
  tableName: String,
  indexName: String,
  partitionKeyDef: KeyDef[P],
  jClient: DynamoDbAsyncClient
) extends SimpleIndex[F, P] {
  def retrieveAll[T: Decoder](
    query: Query[P, Nothing],
    consistentRead: Boolean
  )(implicit RT: RaiseThrowable[F]): Stream[F, T]
}

What I'm proposing is to replace SecondarySimpleIndex with SimpleLSI and SimpleGSI so that retrieve's signature doesn't change and retrieveAll can be introduced to SimpleGSI. WDYT?

from meteor.

Is a SimpleLSI actually possible? I believe an LSI has to have a sort key.

from meteor.

I could see either side for only having retrieve return a stream vs have a retrieve/retrieveAll. Allowing a retrieve that returns F[Option[T]] that might actually arbitrarily select the last element (and if accidentally really mis-used, only after streaming through many records -- perhaps the implementation should change to be .head.compile.last...). But it's also true that someone may have a GSI with a guaranteed-to-be-unique partition key (I have such use-cases myself) and that retrieve would read more clearly than having to add .compile.last.

from meteor.

Is a SimpleLSI actually possible? I believe an LSI has to have a sort key.

that is a good point, I think I've found a fix for this. I moved SecondarySimpleIndex to extends CompositeIndex in this branch, the signature of retrieve fits better there, the only downside is naming. WDYT? https://github.com/d2a4u/meteor/pull/234/files#diff-06c5a4f5c02eebaa484b12addf3b0efc009b69d1e93124501aac37d7dbf31183R59

from meteor.

Seems ok to me, let's me do what I need:

byAuthorIndex.retrieve[Book](
        Query(partitionKey = "Jules Verne"),
        limit = 100
      )

The hacky SecondaryCompositeIndex in my original post has the nicer ergonomics of not needing to wrap in a query though which would be nice for SecondarySimpleIndex:

byAuthorIndexFixed.retrieve[Book](
        "Jules Verne",  limit = 100
      )

from meteor.

it is a overloading method so the signature has been changed to this

from meteor.

Looks good to me!

from meteor.

This is now fixed in latest version: v1.0.14. It should have been bumped to 1.1.0 but there seems to be an issue with CI versioning bump. Apologies.

from meteor.