Giter VIP home page Giter VIP logo

debezium.github.io's Introduction

Build Status License Developer chat Google Group

Introduction

This is the source code for the Debezium website. This is based on templates created by the Red Hat upstream community using Jekyll.

For publishing the Debezium reference documentation, the Antora tool is used, which produces the documentation based on AsciiDoc files in different branches of the Debezium main code repository. The rendered HTML pages are added as-is to the website generated with Jekyll. Please see ANTORA.md to learn more.

License

Contents of this repository are available as open source software under Apache License Version 2.0.

System Requirements

We use Docker to build the site. Be sure you have a recent version of the Docker Engine or Docker Machine.

Getting Started

1. Get the site source code

Use Git to clone the Debezium website Git repository and change into that directory:

$ git clone https://github.com/debezium/debezium.github.io.git
$ cd debezium.github.io

If you plan to submit changes, fork the Git repository on GitHub and then add your fork as a remote:

$ git remote rename origin upstream
$ git remote add origin https://github.com/<you>/debezium.github.io.git

Then check out the develop branch and get the latest. If you're going to make changes, create a topic branch and make the changes there.

2. Start the development webserver

There are two recommended ways for previewing the website locally, either via the container environment that's also used on CI for publishing the website, or via a Jekyll install on your machine. The latter is a bit quicker, but requires Ruby/Jekyll to be set up correctly, whereas you get this "for free" via the container image.

2.1 Using the container image

In a new terminal initialized with the Docker host environment, start a Docker container that has the build environment for our website:

$ docker run --privileged -it --rm -p 4000:4000 -e LC_ALL=C.UTF-8 -e LANG=C.UTF-8 -v $(pwd):/site --name website-builder debezium/website-builder bash

This command tells Docker to start a container using the debezium/website-builder image (downloading it if necessary) with an interactive terminal (via -it flag) to the container so that you will see the output of the process running in the container. The --rm flag will remove the container when it stops, while the -p 4000 flag maps the container's 4000 port to the same port on the Docker host (which is the local machine on Linux or the virtual machine if running Boot2Docker or Docker Machine on OS X and Windows). The -v $(pwd):/site option mounts your current working directory (where the website's code is located) into the /site directory within the container.

Next, in the shell in the container, run the following commands to update and then (re)install all of the Ruby libraries required by the website:

jekyll@49d06009e1fa:/site$ bundle update
jekyll@49d06009e1fa:/site$ bundle install

This should only need to be performed once. After the libraries are installed, we can then build the site from the code so you can preview it in a browser:

jekyll@49d06009e1fa:/site$ rake clean preview

The site generation process is integrated with Antora. Antora generates the version-specific documentation from the main Debezium GitHub repository, and is invoked prior to Jekyll for inclusion in the website. The default mode using the above commands will fetch the Debezium codebase from GitHub. If you wish to generate the Debezium documentation using your local Debezium git repository, see the next section. For more specifics on the Antora build, please see ANTORA.md

2.2 Using the container image - (generate Debezium docs from local repo)

The default process (outlined in the previous section) will generate the Debezium version-specific documentation from the Debezium repository on GitHub. However you may want to view or edit the documentation found in your local Debezium git repository. In this case, the steps are slightly different.

First, you must have a local clone of the Debezium repository.

Then start the Docker container:

$ docker run --privileged -it --rm -p 4000:4000 -e LC_ALL=C.UTF-8 -e LANG=C.UTF-8 -v $(pwd):/site -v ~/<PATH_TO_REPOS>/debezium:/debezium --name website-builder debezium/website-builder bash

Note the addition of the volume mapping -v ~/<PATH_TO_REPO_DIR>/debezium:/debezium - ( replace <PATH_TO_REPO_DIR> with the actual location on your system ).

Next, in the shell in the container, run the following commands to update and then (re)install all of the Ruby libraries required by the website:

jekyll@49d06009e1fa:/site$ bundle update
jekyll@49d06009e1fa:/site$ bundle install

This should only need to be performed once. After the libraries are installed, we can then build the site from the code so you can preview it in a browser:

jekyll@49d06009e1fa:/site$ rake clean author preview

The additional author arg above sets the environment to use playbook_author.yml instead of playbook.yml for Antora, which then builds the Debezium docs from the local repo. For more specifics on the Antora build, please see ANTORA.md

2.3 Using a local Ruby and Jekyll installation

Run the following steps once to set up your local Jekyll installation.

  • Install RVM, the Ruby version manager (see here why to use RVM and Bundler).

  • Install Ruby:

      rvm install ruby-2.7.2
    
  • Create a gemset for all the dependencies to be installed:

      rvm ruby-2.7.2 do rvm gemset create debezium.io
    
  • Create a file named .rvmrc in this directory (debezium.github.io) with the following contents:

  • Change out of this directory and back in again, it should display a message like this:

      cd .. && cd debezium.github.io
      Using /Users/gunnar/.rvm/gems/ruby-2.7.2 with gemset debezium.io
    
  • Install all dependencies

      bundle update
      bundle install
    

With this set-up in place, you can preview the website by running Jekyll like so:

bundle exec jekyll serve --livereload --incremental

3. View the site

Point your browser to http://localhost:4000 to view the site. You may notice some delay during development, since the site is generated somewhat lazily.

4. Edit the site

Use any development tools on your local machine to edit the source files for the site. Jekyll will detect the changes and may regenerate the corresponding static file(s).

If you have to change the Gemfile to use different libraries, you will need to let the container download the new versions. The simplest way to do this is to stop the container (using CTRL-C), use rm -rf bundler to remove the directory where the gem files are stored, and then restart the container. This ensures that you're always using the exact files that are specified in the Gemfile.lock file.

5. Commit changes

Use Git on your local machine to commit the changes to the site's codebase to your topic branch, and then create a pull request.

6. PR Preview the website

As soon as you passes the pull request a GitHub action generates the link for preview the change in surge.sh. When a pull request is closed the surge preview instance will be torn down.

7. Publish the website

Review the pull request and merge onto the develop branch. The GitHub Actions will then build the develop branch and, if successful, store the generated site in the gh-pages branch and publish to GitHub Pages.

debezium.github.io's People

Contributors

adelasofia avatar amitbhave avatar ani-sha avatar avelanarius avatar cheyenneweaver avatar criccomini avatar crim avatar dependabot[bot] avatar emmanuelbernard avatar gunnarmorling avatar hashhar avatar hpgrahsl avatar indraraj avatar insectengine avatar ismailsimsek avatar jcechace avatar jpechane avatar lightguard avatar matzew avatar mdrillin avatar mfvitale avatar naros avatar pimpelsang avatar renatomefi avatar rhauch avatar rk3rn3r avatar scholzj avatar uidoyen avatar unibrew avatar vjuranek avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

debezium.github.io's Issues

Issue with creating topics in capital letters for multiple tables while using debezium source connectors

Hi, I am trying to push my source postgres cdc changes to target oracle tables through kafka and that's why using debezium kafka source connector(docker image debezium/connect:1.8) as a source connector and kafka jdbc sink connector as a sink connector. along with kafka broker. Now, I want the topic names to be created in capital letters as otherwise we face other issues while creating the tables in oracle through the jdbc sink connector. As the postgres tables are by default created in small letters , I am using the below configurations in my source connector to create the topic in capital letters(hardcoding) and the below is working fine. Now, I want this to be done for multiple tables, "table.include.list": prodfxiapp1.requests,prodfxiapp1.trade,prodfxiapp1.leg. Is it possible to do it through the config changes? If so, what should be the changes?

"name": "source-connector",
"config": {
"connector.class": "io.debezium.connector.postgresql.PostgresConnector",
"tasks.max": "1",
"database.hostname": "",
"database.port": "
",
"database.user": "",
"database.password": "
",
"database.dbname": "",
"database.server.name": "
",
"table.include.list": "prodfxiapp1.requests",
"transforms": "topicRename",
"transforms.topicRename.type": "org.apache.kafka.connect.transforms.RegexRouter",
"transforms.topicRename.regex": "(.*)",
"transforms.topicRename.replacement": "REQUESTS"
}

Disqus comments aren't loaded

I pushed 9540910 as a work-around, hard-coding the base URL into the Disqus include for now.

That way, the comment widgets are loaded again, but ideally we won't have the base URL in any template/include. With Awestruct, the base URL was part of the environment-specific configuration and could be referenced in templates as a variable then. Is something similar doable for Jekyll?

@mdrillin @indraraj @uidoyen

Feature Request: Connectors for sybase ASE

Hello, My organization is using Sybase ASE 16 SP04 for storing all the transaction and reference data. We have a need to capture and stream the changed data fields on specific data tables as events to the downstream systems. I looked into your documentation, but I don't see Sybase in the support databases list. Do you have a plan to support Sybase database?

Note: I have seen few threads in stackoverflow looking for Sybase ASE connectors for similar need.

Let me know when can you support this and be part of the roadmap.

Regards,
Nazimuddin
Enterprise Architect - Carelon Inc (a Elevance Health company)

Feature Request: Connectors for sybase ASE

Hello, My organization is using Sybase ASE 16 SP04 for storing all the transaction and reference data. We have a need to capture and stream the changed data fields on specific data tables as events to the downstream systems. I looked into your documentation, but I don't see Sybase in the support databases list. Do you have a plan to support Sybase database?

Note: I have seen few threads in stackoverflow looking for Sybase ASE connectors for similar need.

Let me know when can you support this and be part of the roadmap.

Regards,
Nazimuddin
Enterprise Architect - Carelon Inc (a Elevance Health company)

Have consistent hover style in top-level menu

When hovering over the Debezium logo, that tile gets a bit darker. That's not the case when hovering over any of the menu items ("Blog", "Community", etc.). This should be done consistently. The hover state should be a bit lighter than the active state when being e.g. on the blog URL.

Missing step in kube (openshift) documentation: "connect" deployment

Hello,
I'm following the steps from that documentation https://debezium.io/documentation/reference/operations/openshift.html (adapting to my minikube environment)
It seems to me that a step is missing (or I am misunderstanding something), in step "3.c", it is described how to create an image for connect named ${DOCKER_ORG}/connect-debezium, and shortly after stating we should see a pod

debezium-connect-3-4sdjr                   1/1       Running       0          1m

... without describing how that pod is created, so I guess there's a missing step to create a deployment based on the built docker image, no?

Cache Jekyll dependencies

I suppose simplest would be to actually do call bundle update and bundle install in the Dockerfile of the builder image, so that the actual build itself should be quicker.

@uidoyen

Set up CI and publication for migrated website

In order to preview how the website looks like in the course of the migration, set up a GH Action which builds and publishes the website to somerepo.github.io. In the future, we should use this to publish a staged preview of the website before pushing changes to the live website.

Add latest posts side bar

An area for improvement of the blog is discoverability of earlier posts. Right now, you only can go to older posts via the "Older" link at the bottom of the list page. To mitigate that, we added the "Featured Posts" section on the left some time ago, which is driven by the "Featured" tag assigned to posts. But it'd be nice to have some sort of "archive" side bar, as done on many blogs. Is here something like that in Jekyll we could use?

@mdrillin @indraraj @uidoyen

bash:: command not found

I created a docker-compose file and run it. It works fine but when I get inside the containers no command works. even apt-get or apt, it gives only command not found error. I dont know why?

in mac os intel.
docker version: 20.10.13

version: '2'
services:
  zookeeper:
    image: debezium/zookeeper
    ports:
      - 2181:2181
      - 2888:2888
      - 3888:3888
  kafka:
    image: debezium/kafka
    ports:
      - 9092:9092
    environment:
      - ZOOKEEPER_CONNECT=zookeeper:2181
  postgres:
    container_name: postgresql
    image: debezium/postgres:11
    ports:
      - 5432:5432
    environment:
      - POSTGRES_DB=debezium
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=password
  connect:
    image: debezium/connect
    ports:
      - 8083:8083
      - 5005:5005
    environment:
      - BOOTSTRAP_SERVERS=kafka:9092
      - GROUP_ID=1
      - CONFIG_STORAGE_TOPIC=my_connect_configs
      - OFFSET_STORAGE_TOPIC=my_connect_offsets
      - STATUS_STORAGE_TOPIC=my_source_connect_statuses
CONTAINER ID   IMAGE                  COMMAND                  CREATED          STATUS         PORTS                                                                    NAMES
a34e34516bdf   debezium/kafka         "/docker-entrypoint.…"   9 minutes ago    Up 8 minutes   0.0.0.0:9092->9092/tcp                                                   docker-kafka-1
1500945fd17c   debezium/connect       "/docker-entrypoint.…"   21 minutes ago   Up 8 minutes   0.0.0.0:5005->5005/tcp, 0.0.0.0:8083->8083/tcp, 9092/tcp                 docker-connect-1
d475c9993513   debezium/zookeeper     "/docker-entrypoint.…"   21 minutes ago   Up 8 minutes   0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 0.0.0.0:3888->3888/tcp   docker-zookeeper-1
58896bc0547b   debezium/postgres:11   "docker-entrypoint.s…"   4 hours ago      Up 8 minutes   0.0.0.0:5432->5432/tcp                                                   postgresql

Clarify license

README and license.txt say Apache v2, but badge in README says CC.

Minor inconsistencies around docs/releases versions

On https://debezium.io/releases/, we show older to newer for the current series, but then newer to older under "See older releases". We should consistently do the latter, i.e. switch the order for the current releases.

Also, in the menu for Documentation and Releases on the left, we have two different green badges for "Stable" and "Latest Stables", but we only show "Stable" on the actual https://debezium.io/releases/ and https://debezium.io/documentation/ pages themselves. Let's add the "Latest Stable" distinction there, too.

docs redirects not working

Before we moved to Antora, there only was a single version published for the documentation, under the /docs path. With the old site, there used to be redirects for URLs like these:

https://debezium.io/docs/tutorial/
https://debezium.io/docs/online-resources/
https://debezium.io/docs/faq/
https://debezium.io/docs/connectors/postgresql/

They got lost in the transition to Jekyll. We should double-check with the old redirect configuration and add back all that are missing, so to keep any links out in the wild intact.

@mdrillin @uidoyen @indraraj

Update instructions in README

It still speaks of Awestruct but should describe how to build the site with Jekyll. This should also touch on how to enable a specific environment (e.g. "production").

@uidoyen, would you like to grab this one?

Add support for Google Analytics

As discussed with @uidoyen; I'd expect Jekyll has built-in support for GA and we just need to configure our tracking id somewhere. If so, the explicit tracker snippet should be removed from the Jekyll counterpart of the template linked below. The GA tracking should only be enabled for the production website.

Incorrect label on prior stable releases

On the release details page, earlier releases show a label of 'latest stable' - even though they are not the latest stable. Should just show 'stable' except for the real 'latest stable'

releaseLabel

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.