progrium / envy Goto Github PK
View Code? Open in Web Editor NEWLightweight dev environments with a twist
License: MIT License
Lightweight dev environments with a twist
License: MIT License
Should be able to auth using a personal access token as your username.
one of my environments keeps restarting itself while I'm logged in. Is there any way for me to debug this and see what caused the crash?
Envy releases will be tested against a version of Docker that will be included in the envy image, however, the readme should suggest mounting your Linux host docker binary to ensure parity with host docker.
$ docker run -d ... -v $(which docker):/bin/docker ... progrium/envy
Unfortunately the above will break on OS X hosts running boot2docker. You cannot use the darwin binary inside envy, but more importantly host mounts are relative to the VM and not OS X. So perhaps it would be:
$ docker run -d ... -v /usr/local/bin/docker:/bin/docker ... progrium/envy
Perhaps its more trouble than it's worth?
Since https://hub.docker.com/r/progrium/envy/ uses HOST_DATA rather than HOST_ROOT, the instructions in the README no longer work; when I do -e HOST_DATA=/mnt/envy
to start a container, I get this error when shelling in:
$ ssh envy
Timestamp: 2016-01-30 21:40:18.794089272 -0500 EST
Code: System error
Message: not a directory
Frames:
---
0: setupRootfs
Package: github.com/opencontainers/runc/libcontainer
File: rootfs_linux.go@40
---
1: Init
Package: github.com/opencontainers/runc/libcontainer.(*linuxStandardInit)
File: standard_init_linux.go@57
---
2: StartInitialization
Package: github.com/opencontainers/runc/libcontainer.(*LinuxFactory)
File: factory_linux.go@242
---
3: initializer
Package: github.com/docker/docker/daemon/execdriver/native
File: init.go@35
---
4: Init
Package: github.com/docker/docker/pkg/reexec
File: reexec.go@26
---
5: main
Package: main
File: docker.go@18
---
6: main
Package: runtime
File: proc.go@63
---
7: goexit
Package: runtime
File: asm_amd64.s@2232
FATA[0000] Error response from daemon: Cannot start container 2c4bae1da20a5c69beb01f85c0e9915338f9027122e595228e746f8e2d1e7452: [8] System error: not a directory
Connection to xx.yy.zz.ww closed.
If I start the container with -e HOST_ROOT
as specified in the README, I get this when I shell in:
$ ssh envy
/bin/enterenv: line 5: HOST_DATA: parameter null or not set
Connection to xx.yy.zz.ww closed.
Right now all environments are private to each user. You should be able to share an environment with others users. I should be able to do it without being an admin, and it should be simple filesystem based.
List of users with access to environment are defined in /root/.users
. No matter contents, owning user always has access. If file is empty, no other users have access. If file has *
then all users have access.
User environments will be referenced in the form <username>/<env>
. However, <env>
is optional, which implies the user's default self-named environment.
$ ssh progrium+bob/[email protected] # goes to shared "cool-env" under user "bob"
$ ssh [email protected] # goes to shared default user env for "bob"
Collisions with usernames and a user environment is simply based on these rules:
Since authentication and environment resources are separate, you can just reference the other user's env URL and authenticate using your own credentials.
https://envy.host/u/bob
https://envy.host/u/bob/cool-env
Fork of #10 (comment)
Steps to reproduce:
exit
)envy.host
docker ps -a | grep a-user
and notice that the container is still runningI can't login using the chrome method over port 80, as I use 2 step authentication. It would be nice to see support for this some time in the future. Thanks.
I'd like to use envy for some of my web development projects, but I'm not sure how to get access to the port the server is running on. For example, if I'm in an environment and my node server is running on port 8080, how do I map that to the outside world so I can hit it from a browser? Maybe have a command addport [public] [private]
or something like that.
Basically, integrate termshare: https://github.com/progrium/termshare
Jeff we need moar docs ;D
Just noticed this: https://gist.github.com/leafmulch/77e9bbf8b11643265853
I had this working a couple hours ago, but couldn't install redis-cli as your video demoed, so decided to scrub everything and update boot2docker to 1.7 (it was 1.6.x). Now it's not working at all, but I notice there's been a recent update to jpetazzo/dind and errors in logs of that container:-
docker@boot2docker:/.ssh$ docker images jpetazzo/dind/.ssh$ docker images progrium/envy
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
jpetazzo/dind latest f9729f762697 About an hour ago 331.7 MB
docker@boot2docker:
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
progrium/envy latest 39378f2a0bbc 3 days ago 231.3 MB
docker@boot2docker:/.ssh$ ssh -i github_rsa -p 2222 [email protected]/.ssh$ docker logs leafmulch.leafmulch
Building environment ...
Connection to 0.0.0.0 closed.
docker@boot2docker:
time="2015-06-24T15:30:24.382129945Z" level=warning msg="/!\ DON'T BIND ON ANY IP ADDRESS WITHOUT setting -tlsverify IF YOU DON'T KNOW WHAT YOU'RE DOING /!"
time="2015-06-24T15:30:24.382430533Z" level=info msg="Listening for HTTP on tcp (0.0.0.0:2375)"
time="2015-06-24T15:30:24.382587402Z" level=info msg="Listening for HTTP on unix (/var/run/docker.sock)"
time="2015-06-24T15:30:24.386267813Z" level=warning msg="Running modprobe bridge nf_nat failed with message: , error: exit status 1"
time="2015-06-24T15:30:24.505171613Z" level=info msg="Loading containers: start."
time="2015-06-24T15:30:24.505776187Z" level=info msg="Loading containers: done."
time="2015-06-24T15:30:24.505853733Z" level=info msg="Daemon has completed initialization"
time="2015-06-24T15:30:24.505879465Z" level=info msg="Docker daemon" commit=0baf609 execdriver=native-0.2 graphdriver=aufs version=1.7.0
Right now, /home/<username>
and /root
are both persisted as volumes across all environments for a user. Root because that lets us customize the profile scripts for the root/default user, and the home directory in case you want to use your own user and have profile scripts there.
However, root script loading will change depending on the distribution of the environment. It might make sense to create a root volume for each environment instead of each user. If there are scripts a user wants sourced across all environments, they can put them in their home directory and source from root.
When thinking about where to store this per env root in the admin directory tree, it starts to make sense that maybe each environment is its root directory with it's env definition inside it. /env
becomes /root/env
inside an environment. This sort of makes sense.
Tried envy under video instruction on a AWS ec2 instance with debian jessie installed.
Start envy container all good (host ssh moved to 2222)
docker run -d --privileged --name envy -v /mnt/data -v /var/run/docker.sock:/var/run/docker.sock -p 80:80 -p 22:22 -e HOST_DATA=/mnt/data progrium/envy
try to ssh with my github key got below error
ssh -i ~/.ssh/github.pem [email protected]
FATA[0000] Error response from daemon: Cannot start container a99166b44939e8508228a828a57e319e298ecb6cb6865f6821f94ea0d9bbe24c: [8] System error: not a directory
server end looks all right :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a097e0f61d36 jpetazzo/dind:latest "wrapdocker" 30 seconds ago Up 29 seconds yikaus.yikaus
30784eb3c15b progrium/envy:latest "codep '/bin/execd - 38 seconds ago Up 37 seconds 0.0.0.0:22->22/tcp, 0.0.0.0:80->80/tcp envy
Here is my docker setup :
docker info
Containers: 2
Images: 24
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 28
Dirperm1 Supported: true
Execution Driver: native-0.2
Kernel Version: 3.16.0-4-amd64
Operating System: Debian GNU/Linux 8 (jessie)
CPUs: 1
Total Memory: 999 MiB
Name: ip-172-31-11-63
ID: L43G:BZ6F:3BN2:ARVR:ESHU:NIKJ:FY3S:FWQ4:W5HY:QFXX:W3WU:O7OV
WARNING: No memory limit support
WARNING: No swap limit support
docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.3.3
Git commit (client): 7c8fca2
OS/Arch (client): linux/amd64
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.3.3
Git commit (server): 7c8fca2
OS/Arch (server): linux/amd64
Any idea on this issue ?
Thanks
Kevin
Right now we initiate /root
with a .bashrc
script. We use this to register aliases for envy commands and soon to source in any startup scripts/profiles in the owner's home directory.
Problem is .bashrc
is not sourced with distros that just have a posix shell like Alpine. Posix doesn't define startup files that it sources by default, but it does source whatever file is defined in an ENV
variable. I think we should standardize this to load /root/.envrc
. We should put anything Envy related here, and then have .bashrc
source it in cases where bash is used instead of sh.
In case anybody is curious about the "rc" naming convention: http://unix.stackexchange.com/questions/3467/what-does-rc-in-bashrc-stand-for
Can you please populate the README and explain what how and this tool works. Thanks.
I'm trying to add support to list environments for the current user and I was thinking of adding an envy-ls
command here: https://github.com/progrium/envy/blob/master/scripts/enterenv#L54. I've got it listing out the environments, but it restarts the container on every run. Any way for me to bypass that? Also, where is the code to execute that function when something exits with a 128 status?
after building latest envy, I get this error whenever I'm inside a container and run docker ps
It looks like the progrium/dind is running docker 1.6.2.
bash-4.3# docker version
Client version: 1.6.2
Client API version: 1.18
Go version (client): go1.4.2
Git commit (client): bc9c64f
OS/Arch (client): linux/amd64
Server version: 1.6.2
Server API version: 1.18
Go version (server): go1.4.2
Git commit (server): bc9c64f
OS/Arch (server): linux/amd64
Unless I'm misunderstanding how things work, using envy allows anybody with a github account to run containers on my host. It would be nice to easily restrict to my github user, or even better, to a github organization.
I edited my Dockerfile to FROM mysql and obviously it won't work but I exited the session, before I realised the mistake, now I can't log in to that environment any more.
If this kind of situation happens, would it be nice to provide an option to reverse the Dockerfile change to last working state?
And I also wonder how to manually fix this, I attached the main docker instance and went to /data/users/[username]/envs and removed the whole folder named after that environment, didn't help.
Based on current specs if you want to keep the source for projects around, you'd persist in /root
to keep with the environment, or /home/<user>
to make available in all environments.
Since Docker containers are how you keep services running, you can imagine using Syncthing to sync some files (such as source files) in an environment or in your home directory with files on your laptop/workstation. I think this should be attempted first to see what the experience is like.
Eventually, we can build in support for NFS for fewer moving parts and user set up. I'm thinking 3 ways to use NFS:
These are ordered in by bang for buck and work involved, but also provide different data configurations.
The first means you keep data with the envy server. No offline unless you're running locally. Some set up required, but not much.
The second stores it somewhere else, which is best in the cloud so you don't have to worry about data loss on your envy host. Requires an accessible NFS somewhere, but otherwise config is easy.
Last option lets you keep your data local, exposing it to envy only when you're using it. It's the most transparent but involves a client, which is a big deal. It also doesn't work for web access unless you run the client and then use the browser. Presumably web access is remote and you wouldn't be at your laptop to run the client anyway.
if I resize my terminal window after starting a new dev session and I open vim, it still thinks its at the old dimensions. I've verified using stty -a
that the dimensions don't get updated in an envy session when the terminal is resized.
I've also tried the same thing when just inside an ubuntu docker container (not using envy) and the dimensions get updated there
I took a stab at implementing the support for exposing ports for web servers on my fork and the way I made it work was to recreate the nested docker instance passing in the appropriate --expose
flags.
In order for that to work, I had to be able to recreate the docker instance without losing the data I already had there, so I ended up passing in --volume "$HOST_ROOT/users/$USER/docker/$ENVIRON:/var/lib/docker"
to the environment container and was wondering if it would make sense to have this over here too.
WDYT?
I'm seeing a lot of left over containers if envy crashes, or a connection fails. Have you seen this behavior? IE user.TIMESTAMP
I've spent some time hacking on the project and as a result I now have lots and lots of "trash" on my Docker instance. It would be nice if we had an easy way to ๐ฃ them all with one or two commands like:
docker rm -f $(docker ps -q -f 'label=envy')
docker rmi $(docker images -f 'label=envy')
Any thoughts on that?
Based on assumptions set up in #15 we can create a special case for jumping into environments built based on projects/repos that exist on any Github repo. This is particularly interesting for:
It's not meant as an ideal way to try a project, since that would most likely be running the project's standard Dockerfile or Docker Hub image. Instead, this is about having the development environment already set up.
Github is a particular domain. Obviously we're pretty dependent on them as is, but it would be good to support non-Github repos. That said, let's focus on a good experience for Github that should be extendable in the future.
Environment names that start with a known domain name are treated as externally imported. Example: github.com/shazow/go-sshkit
. This will produce an environment named gh-shazow-go-sshkit
. The gh
is the name of the known domain.
$ ssh progrium+github.com/shazow/[email protected]
https://envy.host/gh/shazow/go-sshkit
Notice the name of the known domain is used as the path prefix. Based on authentication, it will create the resolved environment name gh-shazow-go-sshkit
for the user. This is used internally. The URL in the browser will not change.
Idea is you can have your existing project repos include a Dockerfile.envy
that's a Dockerfile made specifically for building a development environment for that project.
When you create a new environment in Envy, it will look for a repo on your github by that name and see if it has a Dockerfile.envy
. If it does, it clones it as the env directory and builds with Dockerfile.envy
. This means an env directory can have a Dockerfile or envy Dockerfile, and Envy will prefer ones with envy suffix. However, it will not clone repos with just a plain Dockerfile as to discourage bad practice.
I just found out about https://hub.docker.com/_/docker/ yesterday and was wondering if envy should move on to using that since jpetazzo's dind seems to be deprecated now ๐ป
When a user is created (logs in for first time), check if they have a repo called envy.home
and clone that into their user home directory mounted into all environments.
The envy data/config filesystem /data
is exposed as /admin
to all users right now. This is intended to only be mounted for admin users. Right now, all users are "admin" users.
With a fresh filesystem, any Github user can log in and the first one will be made admin. This user can then add users that are allowed to log in, and add which users are admins.
Imagining something stupid simple for both: /data/config/users
and /data/config/admins
. Both are just files with line separated usernames. When users file is not present, any user is allowed to login. When admin file is not present, first user to login is made admin and is written to admin AND user file.
On another note, perhaps the envy mount should be /admin
to be consistent with environment mounted /admin
, and we call it the "admin root".
We're using the canonical jpetazzo/dind
image, but it's based on ubuntu and so it's much bigger than it needs to be. This make the first time experience suck. We should use an image based on the contributed alpine based Dockerfile:
https://github.com/jpetazzo/dind/tree/master/alpine
envy should also pull it on boot, as well as ubuntu, just to get a jump on loading those default base images later.
I normally use tmux when I'm in my dev environment so I can run the web server in one window, run grunt in another and vim in another. I've noticed that if I don't kill the tmux session and exit ssh, the processes are still running, but I can't reattach when I log in again. I've tried storing the tmux socket in a shared directory, but that doesn't work either. How are others handling multiple terminals?
Need to change HTTP to only serve over TLS. By default it will generate a self-signed cert and copy into place. It can be overwritten with a custom cert.
SSH needs to do the same for the host key instead of baking in a default everybody uses. See:
https://github.com/shazow/go-sshkit/blob/master/client.go#L12
I've noticed my clock regularly gets out of sync in envy environments. Any good way to fix this? Is there a way for docker to use the system's clock?
I'm trying to run a container with a docker volume from inside of an envy instance, but the folders always end up empty. Is this because of the docker inside docker? Is there any way around it?
Is there a nice way I can use this with a docker engine configured to use a tcp:// port instead of /var/run/docker.sock?
I get this currently:-
2015/06/24 12:48:36 Failed to handshake: EOF
2015/06/24 12:48:48 Failed to handshake: EOF
2015/06/24 12:48:57 Failed to handshake: EOF
Tried running without sharing /var/run/docker.sock as a volume and instead setting DOCKER_HOST to the same as docker engine hosting envy, but get a connection that just sits and does nothing.
Sessions with an explicitly blank environment name will produce a throwaway environment. It will be named env.$(date +%s)
and the image will be deleted when the session ends.
$ ssh [email protected]
https://envy.host/u/progrium/-
I have a base docker image for my dev setup and if I type rebuild
it builds from that. The problem is I regularly update that image, but rebuild isn't pulling latest, so I have to log into host machine and manually pull and then rebuild. Any thoughts on making this easier? I'd be happy to submit a pull request, but I wanted to discuss first in case something was planned already
when env-init
is called a second time it seems to copy the base env template directory to the env location, but since it exists, it's put inside. for some reason it seems to not be honoring the if condition:
https://github.com/progrium/envy/blob/master/scripts/enterenv#L23
but it's probably something stupid i'm not seeing.
Project is intentionally obfuscated to limit use. It's too early. Here is a checklist of things that need to be done. Suggestions welcome.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.