cognitedata / cognite-sdk-scala Goto Github PK

Add support for reading from these two endpoints
https://docs.cognite.com/api/v1/#operation/get3DNodes
https://docs.cognite.com/api/v1/#operation/get3DNodeAncestors

We should have something similar to https://cognite-docs.readthedocs-hosted.com/projects/cognite-sdk-python/en/latest/cognite.html#upload-a-file-or-directory

We can either make it similar to https://cognite-docs.readthedocs-hosted.com/projects/cognite-sdk-python/en/latest/ or go for something in the style of https://typelevel.org/cats/typeclasses.html

https://doc.cognitedata.com/api/v1/#operation/downloadLinks

Time series may not have names, and so the name field should be optional for time series related case classes.

val client = new GenericClient[IO, Nothing]("cdp-spark-datasource")
client.threeDAssetMappings(modelId.toLong, revisionId.toLong).read().unsafeRunSync()

Attempting to read 3D Asset mappings of an existing modelId + revisionId throws a decoding error:

Attempt to decode value on failed cursor: DownField(treeIndex),DownArray,DownField(items)
DecodingFailure(Attempt to decode value on failed cursor, List(DownField(treeIndex), DownArray, DownField(items)))

Some requests have limits, like max 1000 items per delete request on assets. The SDK user shouldn't have to care about those, and we should handle it for them in the SDK itself.

As a simpler alternative we could also throw an exception on the client side if a request that is too large for the API is made, but that's hardly better than just leaving the decision to the API.

PS.

A challenge if we take care of this on the API side is that the return type of delete would then be a sequence of responses, instead of a single response. We may have to add some cats-effect to summarize the response into a single one (either a single success response, or a single failed response).

However, that may be considered a separate issue, and returning a list of responses that is usually not inspected anyway is acceptable.

cognitedata/cdp-spark-datasource#308

We should allow users to query aggregates in addition to the raw data points.

There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.

Error type: Cannot find preset's package (local>cognitedata/renovate-config)

Write some introductory documentation similar to https://cognite-docs.readthedocs-hosted.com/projects/cognite-sdk-python/en/latest/cognite.html#

Timeseries data points support protobuf in v1 (not yet documented, but it's been implemented), and we should be using it.

See https://github.com/cognitedata/protobuf-files/blob/master/v1/timeseries/data_point_insertion_request.proto etc.

client.dataPoints.queryAggregatesById(
      3385857257491234L,
      1509490000001L,
      1510358399999L,
      "1d",
      Seq("min")
    ).unsafeRunSync()

This will return the counts from publicdata. Using count seems to return nothing. I have not checked other aggregates than min, max, sum and count

See https://github.com/cognitedata/cdp-spark-datasource/blob/da6363ab16c7da625ba689831411cd987ad647e4/src/main/scala/com/cognite/spark/datasource/CdpConnector.scala#L57 for an example.

Rather it uses the project name returned by the login call. This works fine for api key authorization, but when using a bearer token, it sets the project to root, which can be confusing.

Not possible to specify limit to dataPoints-queries.

I can't see any docs on how to install the SDK (is it available through any package management systems)?

It's annoying to have to create Auth and SttpBackend explicitly for the simple use case. Find a way to make the user experience smoother, by providing default values.

It should be possible to override them.

Some, or maybe only Files, of our resource only support creating a single item per request. Currently we handle this with a runtime check on the length of the items passed to create, but it would be better if we handled this in a separate createOne method: it would remove the need for the runtime check, and it would allow the method to return the single response without wrapping it in a sequence/array.

Add a CreateOne trait for this method, which can easily be implemented in terms of create for resource types that support it (wrapping and unwrapping the single item in a sequence and using create), and have Files implement that trait directly (but not the Create trait).

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• chore(deps): update dependency org.scalatest:scalatest to v3.2.18
• chore(deps): update jettytestversion to v9.4.54.v20240208 (org.eclipse.jetty:jetty-server, org.eclipse.jetty:jetty-servlet)
• chore(deps): update dependency ch.epfl.scala:sbt-scalafix to v0.12.1
• chore(deps): update dependency commons-io:commons-io to v2.16.1
• chore(deps): update dependency org.typelevel:typelevel-scalafix to v0.3.1
• chore(deps): update dependency sbt/sbt to v1.10.0
• chore(deps): update dependency scalafmt to v3.8.2
• chore(deps): update actions/checkout action to v4
• chore(deps): update codecov/codecov-action action to v4
• chore(deps): update dependency com.google.protobuf:protobuf-java to v4
• 🔐 Create all rate-limited PRs at once 🔐

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• chore(deps): update dependency com.jsuereth:sbt-pgp to v2

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• chore(deps): update dependency com.google.protobuf:protobuf-java to v3.25.3
• chore(deps): update dependency com.thesamet.scalapb:compilerplugin to v0.11.17
• chore(deps): update dependency org.tpolecat:natchez-core to v0.3.5
• chore(deps): update dependency scala to v2.13.14
• chore(deps): update catseffectversion to v3.5.4 (org.typelevel:cats-effect, org.typelevel:cats-effect-laws, org.typelevel:cats-effect-testkit)
• chore(deps): update dependency org.wartremover:sbt-wartremover to v3.1.6
• chore(deps): update fs2version to v3.10.2 (co.fs2:fs2-core, co.fs2:fs2-io)
• chore(deps): update sttpversion to v3.9.7 (com.softwaremill.sttp.client3:async-http-client-backend-cats, com.softwaremill.sttp.client3:circe, com.softwaremill.sttp.client3:core)
• chore(deps): update jettytestversion (major) (org.eclipse.jetty:jetty-server, org.eclipse.jetty:jetty-servlet)
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

Testing updates is a bit tedious, so we don't always test all different updates available.

They could be automated by using reflection on the update and/or read class to generate updates attempting to set each properties to different valid values (and set to null when allowed), and automatically compare results with expectations.