Giter VIP home page Giter VIP logo

redisha's Introduction

REDIS HIGH AVAILABILITY AND REPLICATION

This project mainly focuses on achieving HA and Replication on Kubernetes platform. I tried to follow all the necessary best practices unless and until it makes our workflow more complicated. Since our primary goal is to achieve HA.

CLIENT

I made a most simple and bare-bone client on nodejs to ensure our connection to Redis and for testing purposes. Since Redis is a key-value pair in-memory database (now it supports Json, Hash, etc) and just to make things simple I am setting a key and sending the response back to the client after fetching the key from the database.

For More info on client side and Kubernetes implementation visit Nodejs-Client

REDIS

Master-Replica replication

Use replicaof to make a Redis instance a copy of another Redis server. A few things to understand ASAP about Redis replication

  • Redis replication is asynchronous, but you can configure a master to stop accepting writes if it appears to be not connected with at least a given number of replicas

  • Redis replicas are able to perform a partial resynchronization with the master if the replication link is lost for a relatively small amount of time.

Challenges To achieve HA

  • Cannot really use Deployment to create Redis pods.Deployment are not meant for stateful components. There are many reasons for that.
  • Cannot use clusterIp service since its loadbalances the pod (in round-robin by default). Loadbalancing is not desired for Redis cluster because we only want to write on master and not on Replicas because writing on replicas will be discarded on the next sync with the master node
  • Cannot hard-coded the config file since replicaof field will keep changing in long-run.
  • How to deciede which will be the next master if the master dies.

Expected Solutions

  • We have to use Statefulset. Unlike a Deployment, a StatefulSet maintains a sticky identity for each of their Pods. These pods are created from the same spec, but are not interchangeable: each has a persistent identifier that it maintains across any rescheduling.
  • We have to use headless service. Kube-proxy does not handle these Services, and there is no load balancing or proxying done by the platform for them.For headless Services that define selectors, the endpoints controller creates Endpoints records in the API, and modifies the DNS configuration to return records (addresses) that point directly to the Pods backing the Servic.
  • One of the Solution is using initContainers to make the connection between replicas and master on the fly. Initcontainer spawns before the pod creation and makes the master-replica connection.
  • Redis solution to this is Redis Sentinels. Redis Sentinel provides high availability for Redis. In practical terms this means that using Sentinel you can create a Redis deployment that resists without human intervention certain kinds of failures.

To check all the Kubernetes command used kubernetes-command

Redis Persistence

Redis is a database though it's in-memory but we are enabling persistence in it by both append-only mode and rdb mode as well both the mode have its drawback.

redisha's People

Contributors

sourikghosh avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar

Forkers

iventong

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.