aiven-open / kio Goto Github PK

See #92 (comment) where this is being addressed. To close this ticket, we should also remove the read_raw_bytes() function.

We should not have to use # ruff: noqa: A003 in schema. Instead, for names that collide with a builtin name, we should do something like this:

type_: i32 = field(metadata={"kafka_type": "i32", "alias": "type"})

Fields with mapKey should be represented as a mapping.

Consider parsing into MagicStack/immutables.

The Java tester is currently disabled for a few entities that aren't passing, see link to exclusion. We should address these issues and make sure the test is enabled for all entities.

Entities with records

Records support has been intentionally deferred until most other things is working as expected. Records are not defined as a schema, and has separate encoding, so it requires some more manual parsing and serializing implementations. There's also the possibility that we don't want to eagerly parse records, but rather build some lazy functionality to allow good performance when used in proxy functionality that doesn't have the need to inspect records.

• ProduceRequest
• FetchResponse
• FetchSnapshotResponse

Tagged fields issue

• FetchRequest #102

Should not output tagged field if its value equals to default. The tagged field in this case is a nested entity which is why this is behaving differently than other tagged fields. To deal with this efficiently we need to avoid instantiating entities at serialization-time, therefore just like with primitives, the default should be defined in the model. The replica_state field needs to change into:

replica_state: ReplicaState = field(
    metadata={"tag": 1},
    default=ReplicaState(),
)

This needs some fancy detection, as it's only possible when all fields of the nested model in turn have defaults (this is why only a single model in the entire protocol schema excersises this behavior).

Nullable bug

Nullable assignment field. This is a bug in the code generation, the field should be None-able and have None as default.

• ConsumerGroupHeartbeatResponse #101.

Generated entity round-trip tests and Java compat tests have a subtle but impactful difference: @settings(max_examples=1). This results in the Java tests taking significantly longer to run, and the round-trip tests are run with much fewer variations. We should remove this explicit setting from the test cases, and instead tune to find some appropriate values that can be set in environment or passed to the CLI invocations.

https://github.com/Aiven-Open/kio/blob/6e1eac6cbdd1bffca9d67ab3efe88ae0c560cac9/codegen/generate_tests.py#L68C1-L68C26

Since #37 starts treating UUIDs as None, fields should be typed as UUID | None.

The library should provide some way to map between request and response, such that given a request model, it's possible to retrieve its corresponding response model.

One way would be to add a field to requests, but that would introduce a runtime dependency between request and response modules, which seems like a good idea to avoid.

Another way is to provide a function that traverses the module tree, this seems easy to implement but a bit ugly ...

With constrained primitive types in place, it's now probably unnecessary to use field(metadata=...) for most cases (records is the only exception that comes to mind). Would be nice to simplify this before initial release. I.e. most of the types can now simply be unambiguously introspected by simply looking at the type annotation.