ehazlett / stellar Goto Github PK
View Code? Open in Web Editor NEWSimplified Container System
License: MIT License
Simplified Container System
License: MIT License
I am setting up stellar and configuring the first vm. I got this error when trying:
DEBU[0000] starting server agent
DEBU[0000] starting grpc server
DEBU[0000] initializing server
DEBU[0000] network init
DEBU[0000] allocating network subnet for node
DEBU[0000] service.network allocating subnet
DEBU[0000] setting up subnet 172.16.0.0/22
DEBU[0000] setting up local gateway 172.16.0.1
DEBU[0000] setting up gateway
DEBU[0000] bind interface: eth0
FATA[0000] operation not permitted
I tried to specify an endpoint for a UDP application like this:
"endpoints": [
{
"service": "dns",
"protocol": "udp",
"port": 53
}
]
But it doesn't seem to work. Is the feature implemented or do I've to install Radiant?
Hi,
I have a "containerize apps" project and use a makefile to run cobra commands for building app binaries, then an image, then run a container.
Your makefile capability is impressive and I'm keen to adopt some of your design, especially the package target.
What is your intended use case for the package target?
Currently if I want to make a cluster of nodes connected using the internet (not trough a private network like a VPN) anyone can join my cluster because the AdvertiseAddress
is exposed on the internet.
It would be great to add an authentication layer with a token or a certificate so that only trusted node can participate in the cluster.
The Makefile is hardcoded to use amd64
. What are the chances this could be built for arm
?
I wanted to built stellar myself with the built via Docker option (because there are no armhf binaries) and got this
-> building daemon e70a943-dev
../../vendor/github.com/mholt/caddy/caddytls/client.go:30:2: cannot find package "github.com/xenolf/lego/acme" in any of:
/go/src/github.com/ehazlett/stellar/vendor/github.com/xenolf/lego/acme (vendor tree)
/usr/local/go/src/github.com/xenolf/lego/acme (from $GOROOT)
/go/src/github.com/xenolf/lego/acme (from $GOPATH)
make: *** [Makefile:50: daemon] Error 1
-> Built dev version e70a943 (linux/amd64)
I now manually added a go get ...
but now I get
# github.com/ehazlett/stellar/vendor/github.com/mholt/caddy/caddytls
../../vendor/github.com/mholt/caddy/caddytls/config.go:122:10: undefined: acme.KeyType
make: *** [Makefile:50: daemon] Error 2
On Nomad and on docker with watchtower it's possible to automatically update the app if a new image from the Docker hub comes up.
I would love to see that feature implemented and maybe an automatic revert to the previous working container if the update fail?
Add the ability to issue an image pull for the cluster.
I found here #8 that it's possible to interact with the API using HTTP but I don't really know the endpoints of that API, is it possible to help me how to find them or maybe add swagger for an easy use of the API?
At the part where Stellar tried to build the local route and get the error below:
DEBU[0005] services started duration=5.0005632s
ERRO[0005] timeout starting service stellar.services.proxy.v1
DEBU[0010] skipping local route 172.16.0.0/22
ERRO[0010] rpc error: code = Unknown desc = failed to dial "/run/containerd/containerd.sock": context deadline exceeded
DEBU[0020] skipping local route 172.16.0.0/22
DEBU[0030] skipping local route 172.16.0.0/22
DEBU[0040] skipping local route 172.16.0.0/22
DEBU[0050] skipping local route 172.16.0.0/22
DEBU[0060] skipping local route 172.16.0.0/22
DEBU[0060] pruning datastore
The last parts of the message just continue until I stop it.
Hi,
i love the idea of a lightweight orchestrator instead of an overblown kubernetes.
I just created a 2 node cluster within virtual machines to play around if i am able to switch a kubernetes cluster over to stellar....
I also ran over #21 :-D but got the example running.
The mount options is on the same level as docker is using right? Do you have a sample with nfs mounts? How do you handle this stuff?
Thanks Martin!
PS: I gonna prepare a ansible role to setup a node/cluster - i hate doing repeatable tasks manually..
Hi! I'm diving into the code base of stellar in efforts to learn more about containers and go, but in some places that I get stuck, I wonder if there is a chat room I can discuss in? If there is not one currently established, are there any plans to make one?
If wanted I can help with this....
When deleting an application the corresponding nameserver records are not removed.
In building the network service we need to decide how to handle IP assignment.
Hi Evan,
First of all, thank you so much for this project, this is just unbelievably great. I am an ops guys trying to learn go, been researching around containerd and ended-up on your awesome project.
I have installed the latest CNI version following a cillium project doc (https://cilium.readthedocs.io/en/v1.1/kubernetes/install/) as I did not have loopback before.
I thought of raising this issue to ask you what would be the recommended approach, like updating the plugin to use 0.3.1 or for me to downgrade the CNI (couldn't find the right key to download the 0.3.0 version yet) ?
jralmaraz@jralmaraz-vn:~/git/stellar$ sudo sctl --addr 127.0.0.1:9000 apps create -f ./example.conf
FATA[0000] rpc error: code = Unknown desc = incompatible CNI versions; config is "0.3.1", plugin supports ["0.1.0" "0.2.0" "0.3.0"]
Thanks in advance and happy new year !
Cheers,
Jose
Currently when using multiple nodes behind a frontend proxy (i.e. TCP) certificates can take a little while to retrieve as the acme challenge can bounce around to different nodes until it lands on the correct node. There are a couple things we could possibly do:
There is an issue with the update detection that causes the proxy service to reload often. There is no apparent negative effects as the reloads are cheap but it should be improved. Perhaps switch to a cluster event system for notifications.
Hi,
i tested stellar on 2 VMs before going on baremetal. Everything fine - now on baremetal i get the following error log
$ stellar config --nic enp3s0 > stellar.config
$ stellar -D server --config ./stellar.config
DEBU[0000] seed peers seedPeers="[]"
DEBU[0000] getPeersFromCache: []
DEBU[0000] cluster peers peers="[]" seed_peers="[]"
INFO[0000] registered service id=stellar.services.version.v1
INFO[0000] registered service id=stellar.services.node.v1
INFO[0000] registered service id=stellar.services.health.v1
INFO[0000] registered service id=stellar.services.cluster.v1
INFO[0000] registered service id=stellar.services.datastore.v1
INFO[0000] registered service id=stellar.services.gateway.v1
INFO[0000] registered service id=stellar.services.network.v1
INFO[0000] registered service id=stellar.services.application.v1
INFO[0000] registered service id=stellar.services.nameserver.v1
INFO[0000] registered service id=stellar.services.proxy.v1
INFO[0000] registered service id=stellar.services.events.v1
DEBU[0000] starting server agent
DEBU[0000] starting grpc server addr="172.16.0.6:9000"
DEBU[0000] initializing server
DEBU[0000] network init
DEBU[0000] allocating network subnet for node y
DEBU[0000] service.network allocating subnet
FATA[0000] invalid CIDR address:
The configfile sounds similar:
{
"NodeID": "y",
"GRPCAddress": "172.16.0.6:9000",
"TLSServerCertificate": "",
"TLSServerKey": "",
"TLSClientCertificate": "",
"TLSClientKey": "",
"TLSInsecureSkipVerify": false,
"ContainerdAddr": "/run/containerd/containerd.sock",
"Namespace": "default",
"DataDir": "/var/lib/stellar",
"StateDir": "/run/stellar",
"Bridge": "stellar0",
"UpstreamDNSAddr": "8.8.8.8:53",
"ProxyHTTPPort": 80,
"ProxyHTTPSPort": 443,
"ProxyTLSEmail": "",
"GatewayAddress": "172.16.0.6:9001",
"EventsAddress": "172.16.0.6:4222",
"EventsClusterAddress": "172.16.0.6:5222",
"EventsHTTPAddress": "172.16.0.6:4322",
"CNIBinPaths": [
"/opt/containerd/bin",
"/opt/cni/bin"
],
"ConnectionType": "local",
"ClusterAddress": "172.16.0.6:7946",
"AdvertiseAddress": "172.16.0.6:7946",
"Debug": false,
"Peers": [],
"Subnet": "172.16.0.0/12",
"ProxyHealthcheckInterval": "5s"
}
I tried to modify the subnet, reviewed the code where the output comes from but i cannot get the cause of this failure :-(
Thanks, Martin
Currently it's not possible to check the logs of an application (unless we use ctr
) launched by stellar, it would be great to add a command like this one to be able to fetch the logs about the application from any node: sctl apps logs <appname>
.
There is a race condition when an app is created, deleted and re-created quickly. There are tombstone records created for the delete which do not get resolved until after the re-create which then remove the valid records in the nameserver service.
Note: due to the eventual consistency this will get resolved on the next datastore replication but can cause issues in the meantime. A possible solution is to check for tombstone records when creating and remove the tombstone to prevent the delete.
Does this support MacOS ? I would like to use it for Mac desktops in order to run workloads on our Macs :)
It seems contanerd suppors windows via the Windows hyper shim.
Right now most of the tests are unit based. We need to add integration tests.
Running into the following error on ARM when attempting to create an app:
sctl --addr 10.0.1.123:9000 apps create -f example.conf
FATA[0000] rpc error: code = Unknown desc = failed to find plugin "loopback" in path [/usr/local/bin /usr/local/bin/cni]
Here's my stellar.conf
{
"ConnectionType": "local",
"ClusterAddress": "10.0.1.123:7946",
"AdvertiseAddress": "10.0.1.123:7946",
"Debug": false,
"NodeID": "rpi-cluster-03",
"GRPCAddress": "10.0.1.123:9000",
"TLSServerCertificate": "",
"TLSServerKey": "",
"TLSClientCertificate": "",
"TLSClientKey": "",
"TLSInsecureSkipVerify": false,
"ContainerdAddr": "/run/containerd/containerd.sock",
"Namespace": "default",
"DataDir": "/var/lib/stellar",
"StateDir": "/run/stellar",
"Bridge": "stellar0",
"UpstreamDNSAddr": "10.0.1.1:53",
"ProxyHTTPPort": 80,
"ProxyHTTPSPort": 443,
"ProxyTLSEmail": "",
"GatewayAddress": "127.0.0.1:9001",
"EventsAddress": "10.0.1.123:4222",
"EventsClusterAddress": "10.0.1.123:5222",
"EventsHTTPAddress": "10.0.1.123:4322",
"CNIBinPaths": [
"/usr/local/bin",
"/usr/local/bin/cni"
],
"Peers": [],
"Subnet": "172.16.0.0/12",
"ProxyHealthcheckInterval": "5s"
}
The example config's CNIBinPaths
appeared to specify containerd and cni binaries. I grabbed the latest release of cni for ARM and your ARM build of containerd. Here's my /usr/local/bin
:
|-- cni
| |-- cnitool
| `-- noop
|-- containerd
|-- containerd-shim
|-- containerd-shim-runc-v1
|-- containerd-shim-runc-v2
|-- ctr
|-- sctl
|-- stellar
`-- stellar-cni-ipam
Am I missing a cni plugin for this to work? Thanks for all of your help so far!
Does stellar support fail-over? I'm able to deploy containers to specific nodes, but it's unclear how or if these get replicated or scaled.
Btw - Your video was very informative. Where can I find part II?
https://www.youtube.com/watch?v=diRmI45Yhg4
Currently there is no way to clean up containerd automatically just like we would with Docker.
Is it possible to introduce a manual garbage collector for the images, containers, snapshots, content (ctr content
) and more (?)?
The command could be like this: sctl gc
Occasionally the proxy service can report duplicate upstreams. The nameserver service appears to have the correct data so this looks like a bug in the local query to get applications:
Proxy
foo.com http://172.16.8.4:80 (up: 479.478µs), http://172.16.0.4:80 (up: 947.151µs), http://172.16.4.5:80 (up: 826.847µs), http://172.16.8.4:80 (up: 2.05114ms)
Nameserver Records
blog.app00.stellar A 172.16.0.4
blog.app00.stellar TXT node=ctr-00; updated=2018-10-24T23:42:16-04:00
blog.app00.stellar SRV blog.app00.stellar service=web proto=http priority=0 weight=0 port=80
blog.app01.stellar A 172.16.8.4
blog.app01.stellar TXT node=ctr-02; updated=2018-10-24T23:42:21-04:00
blog.app01.stellar SRV blog.app01.stellar service=web proto=http priority=0 weight=0 port=80
blog.app02.stellar A 172.16.4.5
blog.app02.stellar TXT node=ctr-01; updated=2018-10-24T23:42:28-04:00
blog.app02.stellar SRV blog.app02.stellar service=web proto=http priority=0 weight=0 port=80
Seems like building is not working via Docker make docker-generate
. I'm not familar enough with protoc to know what it means by the plugin failure.
I ran this on two machines to ensure it can be replicated: AWS Instance and my MBP.
[snip]
go: finding github.com/mholt/caddy v0.0.0-20180907212407-13a54dbdda25
-> building cli 42605bd-dev
-> building cni-ipam 42605bd-dev
-> Built dev version 42605bd (linux/amd64)
Removing intermediate container d25e59d175b5
---> 55835e7cb1df
Successfully built 55835e7cb1df
Successfully tagged stellar-dev:latest
protoc-gen-grpc-gateway: program not found or is not executable
--grpc-gateway_out: protoc-gen-grpc-gateway: Plugin failed with status code 1.
2018/11/06 17:29:13 protoc -I.:/go/src/github.com/ehazlett/stellar/vendor/github.com/gogo/protobuf:/go/src/github.com/ehazlett/stellar/vendor/github.com/gogo/googleapis:/go/src/github.com/ehazlett/stellar/vendor:/go/src:/usr/local/include:/usr/include --grpc-gateway_out=import_path=github.com/ehazlett/stellar/api/services/application/v1,Mgogoproto/gogo.proto=github.com/gogo/protobuf/gogoproto,Mgoogle/api/annotations.proto=github.com/gogo/googleapis,Mgoogle/api/http.proto=github.com/gogo/googleapis,Mgoogle/protobuf/any.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/descriptor.proto=github.com/gogo/protobuf/protoc-gen-gogo/descriptor,Mgoogle/protobuf/duration.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/empty.proto=github.com/gogo/protobuf/types,Mgoogle/protobuf/timestamp.proto=github.com/gogo/protobuf/types:/go/src /go/src/github.com/ehazlett/stellar/api/services/application/v1/application.proto
make: *** [Makefile:38: generate] Error 123
tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
make: *** [docker-generate] Error 1
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.