Comments (5)
Just tested it again a couple of times with the latest image (8.0.12-debian-9-r45) and it works fine. All slaves are consistent. Will close this for now but I'll test occasionally as I totally do not trust it :D
from charts.
Hi @pentago
Could you share the logs of each slave pod? I think there could have been an issue during the initialisation of one of them...
I could not reproduce it locally (check my results below) but I'd like to know the reason why data was not replicated on all your pods since it's a critical issue as you mentioned.
$ helm install --name mysql bitnami/mysql --set slave.replicas=2
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-mysql-master-0 1/1 Running 0 2m
mysql-mysql-slave-0 1/1 Running 0 1m
mysql-mysql-slave-1 1/1 Running 0 39s
$ kubectl exec mysql-mysql-slave-0 ls /bitnami/mysql/data
auto.cnf
ib_buffer_pool
ib_logfile0
ib_logfile1
ibdata1
ibtmp1
my_database
mysql
mysql-bin.000001
mysql-bin.000002
mysql-bin.index
mysql-relay-bin.000003
mysql-relay-bin.000004
mysql-relay-bin.index
mysql_upgrade_info
performance_schema
sys
$ kubectl exec mysql-mysql-slave-1 ls /bitnami/mysql/data
auto.cnf
ib_buffer_pool
ib_logfile0
ib_logfile1
ibdata1
ibtmp1
my_database
mysql
mysql-bin.000001
mysql-bin.000002
mysql-bin.index
mysql-relay-bin.000003
mysql-relay-bin.000004
mysql-relay-bin.index
mysql_upgrade_info
performance_schema
sys
from charts.
The issue occurred when I deleted existing chart installation and try to do the fresh install again by using existing, previously used PVC and volume with pre-existing data on it.
The first slave gets all the data but the second one didn't.
Steps to reproduce:
- Install chart with fresh, pre-provisioned clean volume and PVC
- import some database (larger if possible, my was 500 megs)
- wait for data to replicate to slaves
- delete/purge chart
- install chart again using previously used existing PVC with data
- check the status and compare data on both slaves.
I'd realy like to hear how these exact steps went on your end because on my end it results in different amount of data across slaves, not sure why yet.
from charts.
Hi @pentago
I couldn't reproduce it. I followed these steps:
- Install chart using old version with clean volumes and PVC
$ helm install bitnami/mysql --name mysql --set image.tag=5.7.23-r51,slave.persistence.enabled=false,slave.replicas=2
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-mysql-master-0 1/1 Running 0 6m
mysql-mysql-slave-0 1/1 Running 0 6m
mysql-mysql-slave-1 1/1 Running 0 6m
REPLICA_PASSWORD=$(kubectl get secret --namespace default mysql-mysql -o jsonpath="{.data.mysql-replication-password}" | base64 --decode)
ROOT_PASSWORD=$(kubectl get secret --namespace default mysql-mysql -o jsonpath="{.data.mysql-root-password}" | base64 --decode)
- Import database (I use this test db: https://github.com/datacharmer/test_db):
- Wait for data to replicate to slaves
$ kubectl exec mysql-mysql-master-0 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
374M /bitnami/mysql/data/
$ kubectl exec mysql-mysql-slave-0 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
$ kubectl exec mysql-mysql-slave-1 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
- delete/purge chart
$ helm delete --purge mysql
- Install chart again using previously used existing PVC with data
$ kubectl get pvc
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
data-mysql-mysql-master-0 Bound local-pv-9f804d79 29Gi RWO fast-disks 8m
$ helm install bitnami/mysql --name mysql --set master.persistence.existingClaim=data-mysql-mysql-master-0,slave.persistence.enabled=false,slave.replicas=2,root.password=$ROOT_PASSWORD,replication.password=$REPLICA_PASSWORD
- Check the status and compare data on both slaves.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
mysql-mysql-master-0 1/1 Running 0 3m
mysql-mysql-slave-0 1/1 Running 0 1m
mysql-mysql-slave-1 1/1 Running 0 1m
$ kubectl exec mysql-mysql-slave-0 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
$ kubectl exec mysql-mysql-slave-1 -- du -h /bitnami/mysql/data/
1.1M /bitnami/mysql/data/performance_schema
8.0K /bitnami/mysql/data/my_database
179M /bitnami/mysql/data/employees
9.7M /bitnami/mysql/data/mysql
676K /bitnami/mysql/data/sys
501M /bitnami/mysql/data/
from charts.
Great thanks @pentago !! Keep us in the loop
from charts.
Related Issues (20)
- [bitnami/kafka] Unable to deploy Kafka 3.7.0 cluster HOT 1
- [bitnami/mongodb] new empty-dir volume should have configurable sizelimt HOT 3
- [bitnami/redis] Can't initialize Background Jobs. Error message: Operation not permitted HOT 3
- [bitnami/redis] redis pod not ready HOT 1
- valkey-cluster Readiness probe failed: cluster_state:fail - nodes don't join the cluster
- Most bitnami charts missing from ArtifactHUB. HOT 3
- [bitnami/schema-registry] Support AWS_MSK_IAM HOT 1
- Mediawiki - Is it gone ? HOT 2
- [bitnami/grafana-loki] serviceMonitor and promtail.yaml config has any bug . HOT 5
- [bitnami/kafka] Execution error using helm install bitnami/kafka 30.0.2 HOT 5
- Why can't I enable zookeeper SSL? HOT 1
- [bitnami/minio] Minio Helm chart persistence not working on Windows 11 Docker Desktop w/ HyperV and Kubernetes HOT 6
- [Bitnami/elasticsearch]: Init container `copy-tls-certificates` image name hardcoded and does not respect global imageRegistry value.
- [bitnami/redis-cluster] Replace default Secret by Secret Provider Class HOT 1
- [bitnami/redis-cluster] Unable to deploy redis-cluster 7.2.5 HOT 2
- [bitnami/keycloak] How do I add my custom provider? HOT 3
- [bitnami/kong] Does any approaches to add config(not Key/Value pair) to Nginx config file? HOT 1
- How to know wordpress username/password
- [bitnami/apisix] Enable admission webhook at ingress controller
- [bitnami/flux] Use Helm Lookup for CRDs HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from charts.