(Taken from confluentinc/kafka-tutorials#937 over at the Kafka Tutorials repo. The same reasoning applies.)

The current kafka tutorials user experience is sub optimal:

• User must install Docker, ccloud CLI, and sometimes other tools
• User must download several GB of docker images
• User must do things like exec into containers in order to use tools installed within

Ideally the user experience would be:

• Click a link to open a remote dev environment with all prerequisites baked in.
• Focus entirely on the learning objective

This more ideal user experience is possible with gitpod workspaces or GitHub Codespaces. In fact, I've already "gitpodified" a kafka tutorial here!

Enabling gitpod would require configuring a .gitpod.yml file in the repo root. Gitpod doesn't yet support multiple .gitpod.yml files, but as a workaround, we could use environment variables via URL (docs and example) to automatically change into the proper code directory on launch. This would mean we'd need one workspace to be able to run any tutorial which should be fine since everything is on docker. If something's not on docker, we could just install it to the workspace, or alias it with docker underneath (e.g. how the az CLI is aliased in the kafka docker playground project here)

When running initial 'build', testing phase (during aggregate test?) seems to be having an issue finding the lib for rocksdb. When I check, there is indeed no file (or similar named file) in the directory.

Following instructions here

Have done a lot of error searching but am struggling to find a solution. Any guidance or suggestions, fixes or similar most welcome.

command:-

./gradlew build

returns output :-

Task :test FAILED

io.confluent.developer.aggregate.solution.StreamsAggregateTest > shouldAggregateRecords FAILED
java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/f3/0y1cbfts4fg4tc9m7yylqpvr0000gp/T/librocksdbjni14255294656738842507.jnilib
at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2633)
at java.base/java.lang.Runtime.load0(Runtime.java:768)
at java.base/java.lang.System.load(System.java:1837)
at org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:78)
at org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:56)
at org.rocksdb.RocksDB.loadLibrary(RocksDB.java:64)
at org.rocksdb.RocksDB.(RocksDB.java:35)
at org.rocksdb.DBOptions.(DBOptions.java:21)
at org.apache.kafka.streams.state.internals.RocksDBStore.openDB(RocksDBStore.java:129)
at org.apache.kafka.streams.state.internals.RocksDBStore.init(RocksDBStore.java:254)
at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:55)
at org.apache.kafka.streams.state.internals.ChangeLoggingKeyValueBytesStore.init(ChangeLoggingKeyValueBytesStore.java:55)
at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:55)
at org.apache.kafka.streams.state.internals.CachingKeyValueStore.init(CachingKeyValueStore.java:75)
at org.apache.kafka.streams.state.internals.WrappedStateStore.init(WrappedStateStore.java:55)
at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.lambda$init$1(MeteredKeyValueStore.java:122)
at org.apache.kafka.streams.processor.internals.metrics.StreamsMetricsImpl.maybeMeasureLatency(StreamsMetricsImpl.java:884)
at org.apache.kafka.streams.state.internals.MeteredKeyValueStore.init(MeteredKeyValueStore.java:122)
at org.apache.kafka.streams.processor.internals.ProcessorStateManager.registerStateStores(ProcessorStateManager.java:201)
at org.apache.kafka.streams.processor.internals.StateManagerUtil.registerStateStores(StateManagerUtil.java:103)
at org.apache.kafka.streams.processor.internals.StreamTask.initializeIfNeeded(StreamTask.java:216)
at org.apache.kafka.streams.TopologyTestDriver.setupTask(TopologyTestDriver.java:537)
at org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:385)
at org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:313)
at org.apache.kafka.streams.TopologyTestDriver.(TopologyTestDriver.java:272)
at io.confluent.developer.aggregate.solution.StreamsAggregateTest.shouldAggregateRecords(StreamsAggregateTest.java:54)

there are many problems in ApplianceOrder, ElectronicOrder, User.java and may be more. For eg these are compilation errors when I am trying to run the joins example. This is in the kafka-streams module. Kindly fix this

/usr/bin/env /usr/lib/jvm/java-11-openjdk-amd64/bin/java -agentlib:jdwp=transport=dt_socket,server=n,suspend=y,address=localhost:39733 @/tmp/cp_4s59uz02yuziby1lbgkafmh34.argfile io.confluent.developer.joins.TopicLoader
Exception in thread "main" java.lang.Error: Unresolved compilation problems:
The type ApplianceOrder must implement the inherited abstract method SpecificRecordBase.getSchema()
The package org.apache.avro is accessible from more than one module: , org.apache.avro
The package org.apache.avro is accessible from more than one module: , org.apache.avro
The package org.apache.avro is accessible from more than one module: , org.apache.avro
SCHEMA$ cannot be resolved to a variable
SCHEMA$ cannot be resolved to a variable
SCHEMA$ cannot be resolved to a variable
The package org.apache.avro is accessible from more than one module: , org.apache.avro
SCHEMA$ cannot be resolved to a variable
SCHEMA$ cannot be resolved to a variable
SCHEMA$ cannot be resolved to a variable
The package org.apache.avro is accessible from more than one module: , org.apache.avro
The package org.apache.avro is accessible from more than one module: , org.apache.avro
SCHEMA$ cannot be resolved to a variable
SCHEMA$ cannot be resolved to a variable
The package org.apache.avro is accessible from more than one module: , org.apache.avro

    at io.confluent.developer.avro.ApplianceOrder.<init>(ApplianceOrder.java:16)
    at io.confluent.developer.joins.TopicLoader.runProducer(TopicLoader.java:56)
    at io.confluent.developer.joins.TopicLoader.main(TopicLoader.java:25)

This is happening in vs code

Running ./gradlew build causes errors on a M1 (ARM) mac, and the KTable and StreamsJoin examples don't work, apparently because Kafka Streams depends on RocksDB, and the version Kafka Streams 2.8.0 uses doesn't work on ARM macs. Updating the Kafka Streams dependencies in build.gradle to 3.5.0 seems to fix the problem.