Comments (7)
@ravishivt @v1k0d3n @intlabs ....well hot dang! :D I'll try to get playing with this next week. :)
from gluster-kubernetes.
That you for the feedback and suggestions! I'll try to address them here as best I can. :)
- Pardon my ignorance, I imagine you're talking about installing packages on the k8s nodes to actually be able to mount GlusterFS volumes, yes? If so, what does the glusterfs-client package contain?
- I would certainly love to have this, I'm just restricted by time. :) It would require some careful restructuring of the ansible playbooks, but I know it can be done. If you or someone you know would be willing to help out with this, I would ask to have a separate issue raised for it and we can have the discussion there.
- Currently we don't run the deploy script as part of the ansible provisioning because we want to have the user walk through the deployment themselves. The scope of the project currently isn't in bringing up a full cluster stack form scratch, it's specifically on the deployment of a hyper-converged GlusterFS setup on Kubernetes. The vagrant-ansible setup is a PoC/development demo, and not something we would recommend for production deployment. I would also think that, while it would provide many logical conveniences, moving to a pure ansible implementation would limit the portability of the deployment.
All THAT said, something we could do is automatically copy the deploy directory into the master node of the vagrant setup so all you'd have to do is log in, edit the topology file (I don't want to hide the importance of needing to understand the topology file), and run the script. Would that be a useful compromise?
from gluster-kubernetes.
- Yup, the glusterfs-client package allows Ubuntu to mount "glusterfs" filesystem types.
- I think the ansible playbook that targets the specific Vagrantfile setup isn't necessary if we go with a general playbook (see part 3). Actually, it might be better to refer users to a project that is dedicated to getting a kubernetes or openshift deployment up and have this project focus on getting the GlusterFS+Heketi hyper-converged deployment up on an existing k8s/OCP deployment. For a k8s setup referral, I'd suggest https://github.com/att-comdev/halcyon-kubernetes. Alternatively, we could git submodule the other project(s).
- I agree that this project shouldn't be focused on bringing up a full cluster from scratch, see my comment in part 2 but I don't see a reason why we have to make the user manually go through all the steps. The user would still need to specify a topology.json file and an ansible inventory file but then the ansible role would then do all the manual stuff for them. I've gotten this set up in my fork, https://github.com/ravishivt/gluster-kubernetes/tree/myenv/deploy. You specify a k8s cluster that's already up and running in inventory.ini and topology.json and then run:
ansible-playbook -i inventory.ini -e 'ansible_user=deploy' deploy.yml
The ansible roles do everything for you including:
- upgrade-os: Upgrade OS packages.
- build-heketi: Builds heketi from source. This may not be necessary when heketi posts latest release binaries, see heketi/heketi#627, although it is still useful to build for targets with uncommon architectures.
- gk-prep: Installs glusterfs-client 3.8 and loads required kernel modules.
- gk-deploy: Copies the gk-deploy script, k8s/ocp templates, topology, and then runs the gk-deploy script.
- gk-post: Automatically finds the HEKETI_SERVER_CLI and creates the StorageClass under a user-configured name from group_vars/all.yml.
The playbook has a few empty placeholders for CentOS and OpenShift specific logic but it's working well for my Ubuntu k8s cluster.
There's also an abort.yml playbook to delete the StorageClass and run gk-deploy with --abort. Lastly, there's a destroy_vg.yml playbook to find Volume Groups and delete them after a confirmation prompt.
from gluster-kubernetes.
Hi! Apologies for the delay, your last comment hit around when I was heading on end-of-year vacation.
Hmm... not bad! Thus far there seems there are two issues at play here:
- Establish a convenient, more generic procedure for bringing up a development environment with Kubernetes already running and ready to go.
- Use Ansible to help automate the pre-requisites, running, and removal of the deployment.
As you just submitted #149 for item 2, I will address item 1. I also suggest editing the subject of this Issue to reflect this. :)
To be frank, the entire vagrant/
directory started as just a development convenience for me. If it can be replaced with a more general solution using git submodules I'm all for it. I had a look at the halcyon-kubernetes project and I find it lacking since it doesn't support Fedora or libvirt (Red Hat, sorry ;) ). I wouldn't be able to get a good look at it and modify it for my use for some time, I think, but if we could I agree this would make the project a lot more accessible.
from gluster-kubernetes.
No need to apologize, thanks for getting back to me. I updated this issue's title as you suggested.
Take a look at https://github.com/att-comdev/halcyon-vagrant-kubernetes. It is a separate project that submodules halcyon-kubernetes. This project uses vagrant to stand up X nodes using VirtualBox, libvrt, or OpenStack. Once set up, it then runs the ansible playbook from halcyon-kubernetes to initialize the cluster on those nodes. That being said, Fedora is not officially supported as you pointed out.
The difficulty is adding the vagrant logic to add virtual disk(s) to all those environments: VirtualBox, libvrt, and OpenStack. I used similar vagrant code as what is in this project but it never worked consistently. Eventually, I gave up and manually set up 3x VMware instances.
from gluster-kubernetes.
@ravishivt, i just happened to notice the issues/PR you created on our halcyon-vagrant-kubernetes repo, and we'd be happy to support fedora if it helps your needs. @intlabs and a larger kolla-kubernetes are using the project, so we're dedicated to maintaining and improving it. i will be able to read through, review and address your items tomorrow. thanks for the interest and PR!
Also, @jarrpa, project does support KVM, but if you find there are issues with it (or require libvirt) we'd be happy to improve on this functionality. https://github.com/att-comdev/halcyon-vagrant-kubernetes/blob/master/Vagrantfile#L82-L87
from gluster-kubernetes.
@jarrpa Ironically I only use halcyon-kubernetes with CentOS/Fedora hosts and Libvirt. I need to get the documentation in the GitHub repo updated, but in the mean-time, you may find the documents I wrote for Kolla-Kubernetes useful, http://docs.openstack.org/developer/kolla-kubernetes/development-environment.html. Hope this helps.
I've been using Halcyon for Ceph, and so it would be great to get Gluster running as well. If there's any way I can help with that just give me a shout :)
@ravishivt Thanks for your PRs, they were great additions. I must confess I have only been using the playbooks in the Vagrant environment, and so had not tested with multiple NICs. Cheers!
from gluster-kubernetes.
Related Issues (20)
- Error: Failed to allocate new volume: No space HOT 2
- Unable to access db HOT 1
- missing required field "selector" in io.k8s.api.apps.v1.DeploymentSpec HOT 1
- Unable to deploy on Ubuntu 18.04 -> pods not found. HOT 6
- Request to structure the README to include more projects
- 401 with latest heketi:dev image HOT 1
- probe failed
- glusterFS pod deploy failing
- PVC in pending status-no other error HOT 2
- Mount failed:E [glusterfsd.c:795:gf_remember_backup_volfile_server] 0-glusterfs: failed to set volfile server: File exists HOT 5
- Pod devices for topology get stuck, if pods are restarted.
- vagrant - failed to install glusterfs-client
- is this project still ALIVE? HOT 9
- Which gluster node are my pods/pvc talking to?
- Error waiting for job 'heketi-storage-copy-job' to complete HOT 4
- Kubernetes DaemonSet extensions/v1beta1 deprecated
- speed up deploying gluster
- heketi deployment has CrashLoopBackOff state, I am using ./gk-deploy script
- glusterfs on kubernetes stock on Error waiting for job 'heketi-storage-copy-job' to complete. stage
- About GlusterFs containers support geographic replication
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from gluster-kubernetes.