maif / thoth Goto Github PK
View Code? Open in Web Editor NEWEvent sourcing in java with vavr, akka stream and vertx reactive PG driver
Home Page: https://MAIF.github.io/thoth/
License: Apache License 2.0
Event sourcing in java with vavr, akka stream and vertx reactive PG driver
Home Page: https://MAIF.github.io/thoth/
License: Apache License 2.0
replace java 15 enabled preview by java 16 in sample project
The replay event mecanism should use the postgresql "select for update" to replay event in order to avoid multi publication to kafka.
Right now when there is an error building eventually consistent projection, offset won't be comitted and consumption will be stuck.
We should add another option for error handling.
In its current state, scribe only allow modification of a single entity with one command.
However there is some event sourcing use case when modification of multiple entity at once is necessary (e.g. bank accounts).
Therefore it would be a good feature to have.
Some rough ideas:
replace command entityId by a list
allow command handler to return a list of event with differents entityIds
A different ObjectMapper is used to publish events in Kafka at first try and to write events in database.
This can cause bugs when republishing events in kafka if first publication failed, since the two are not compatible, for instance on vavr lists.
I think the best way to do it would be to instantiate and use some kind of singleton ObjectMapper, that is exposed for modification.
TableNames should have a schema field
Provide a default config for durability in kafka producer
Create a kafka consumer implementing resilience :
A Technology Compatibilty Kit would be helpfull to ensure every implementations of scribre-core are corrects.
This would be a great help to #3.
Hello, have you planned to add a spring integration (like Izanami), i'm not thinking about some magic annotations, but just an auto-configure that create some beans and make the integration straight forward.
I've created a demo project for my company and the integration is quite easy, but a little auto-configuration would make it easier and add a first "good impression" ๐
Current projection mecanism allow to consume events that will happen in the future, however, there is nothing in the doc explaining how to catch up with past events.
Therefore we should add a documentation section explaining how to catch up by streaming journal content.
Some projections may benefit from the aggregate.
It could avoid useless journal reads in some cases
Hi @larousso and @ptitFicus
This project looks valuable. I have few queries relating to it.
a. Why have you decided not to use CDC-Debezium and how are you able to achieve it without using CDC.. pretty interesting? Documentation says: "It keeps trying writing to Kafka until succeeded", won't it impact performance?
b. What is the role of akka in this project? Can we do away with it because it is BSL now.
c. While writing to Kafka, Avro/protobuf formats have not been used which are really fast. Any reason for it? Is it possible to integrate Thoth with this gRPC/REST proxy for Kafka?
d. Does it support these: Optimistic Concurrency, (multiple users writing events to the same table at the same time) Event Replay (right from beginning for Audits etc.), Snapshots, topic compaction (delete/overwrite a message record for a given key), and Event Versioning?
e. What is the difference between Non blocking JOOQ and Standard JOOQ/Kafka implementations. In which case (use cases), shall we use which implementation?
f. Is there no need for Kafka-proxy
Thanks
In some scenarios, commands should not be run concurrently.
There is several possible needs :
There is also several way to handle concurrent command :
At the moment, it's very complicated to wire all the components, it could be really good to have a step builder that help for this.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.