adroll / hologram Goto Github PK

I'm envisioning a CLI (#37) or a chrome extension that uses hologram provided credentials to open the AWS console. Do you think something like this would be useful?

From here: https://github.com/AdRoll/hologram/blob/master/transport/remote/client.go#L37-L42

    tlsConf := &tls.Config{
        RootCAs: pool,
        // Hologram only uses TLS to ensure the credentials that go across the wire are kept secret, and since go uses
        // ECDHE by default, we actually don't care about leaking keys or authenticating either end of the connection.
        InsecureSkipVerify: true,
    }

Is that a reasonable assumption? Does it allow MITM-ing the hologram server connection?

It looks like it's repeatedly pulling the same stuff every time I call ./hologram.sh build_all. This works fine most of the time, but on the spotty Amtrak WiFi connection I'm on right now, it's a bit miserable.

You have

    searchRequest := ldap.NewSearchRequest(
        "dc=keeponprovoking,dc=com",

Which should probably be somewhat configurable 😄

I came across 99designs/aws-vault#198, not sure if it applies and have nothing to test with, but someone should probably check if this issue applies to hologram or not as it could mean leaking credentials to anyone on the network

I noticed that if I restart a computer running the hologram agent that the user must type hologram use again to reacquire role credentials. Could the "current role" state be persisted to disk somehow?

Can this be done?

Instead of having to remember a bunch of ugly ARNs, it would be nice if the hologram CLI tool looked at some sort of config file (could default to ~/.hologram perhaps) that listed human-readable alias mappings for full ARNs. For example:

[Aliases]
prod-zomg = arn:aws:iam::123456789012:role/ZOMG
whoa-wtf = arn:aws:iam::09876543210:role/muahahhaa

Then hologram use prod-zomg would do the obvious thing.

Hologram server and STS seem equivalent at first glance: http://docs.aws.amazon.com/STS/latest/UsingSTS/Welcome.html

What does Hologram server provide that STS doesn't? Why not use STS as the backend for Hologram agent?

A while back AWS added support to their SDKs to support ECS Task roles. This mechanism basically allows for multiple roles to be active, and which role is used is controlled by the AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable. (see docs here: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/task-iam-roles.html)

This would be nice for Hologram because it can be used to run multiple processes under different roles by setting an environment variable. It might be a better alternative to #61 and #51

I've implemented support for this in a local branch, and would be happy to submit a PR, but wanted to check first if there's any interest from the maintainers in merging this. My local branch has a number of unrelated changes on it (upgrade to Go 1.12/modules, upgrade to the latest version of the protobuf library, upgrade all the dependencies), so it would be a good bit of work to pluck out just this feature. I'm not going to go through the effort if it will never get merged. If there's interest from the maintainers I can backport my changes against the latest master (or, if there's interest in taking my whole fork with all the latest Go support happy to help with that too)

Go reimplements its own syscall interface on OSX instead of using libSystem as Apple recommends/requires. Because of that, if Apple changes their syscall ABI, Go programs can break, and that happened with Sierra.

More detail here: golang/go@2da5633

I have it working again by just switching the Hologram Dockerfile to use 1.7 instead of 1.4, but haven't had a chance to put together a proper PR yet. If someone wants to make the one-character change (plus a comment character) before I get to the PR, please be my guest 😄

Getting this with a new-hire.

The hologram-ping command is not using tls to connect and therefore the connection is refused by hologram-server.

This line needs to look more like this one.

Our current makefile assumes you're developing on a mac, that all your clients will also be macs and that your server is going to be ubuntu (or at least debian) based.

I propose we simplify it and allow it to be more flexible, by ensuring the following:

• We should be able to build and use hologram just by doing "go get github.com/AdRoll/hologram/..."
• The build process should be as standard as possible (even getting rid of the dependency for gox, if we can just cross-compile by setting GOOS or using a docker container) so all binaries appear in the expected places.
• Use our makefile (or rake or whatever) to build rpm/deb/pkg packages for both client and server on demand (but there's no point on building all packages when doing "make")

I'm not sure what would cause this, but I've seen this a few times on more than one computer:

$ hologram use somerole
[ERROR  ] 2016-02-22T09:04:09-05:00 dial tcp: lookup hologram.server.dns: no such host

If I try pinging hologram.server.dns (not the real domain) myself, everything else on my system seems to be able to resolve it just fine. Hologram will continue complaining about the "no such host" for a while and then after a few minutes, will start resolving it fine again.

It might worth noting that hologram.server.dns started as a cname for an Amazon ELB. After seeing this issue for a bit, I pointed my hologram agent directly at the ELB DNS, in case hologram didn't like cnames, but the issue persists.

Any suggestions? I don't see anything particularly interesting in the agent logs, but if someone has suggestions on how to debug this I'm happy to try things next time it happens.

Instead of rebuilding the protobuf definitions or the bindata assets on every build the intermediate .go files they generate should be checked in the repository already, and only regenerated when the source changes (using go generate)

This will simplify our build process and the dependencies needed to build and test hologram (as opposed to actually hacking on its internals) and will potentially get us to the point where hologram is go gettable

My understanding is that before the agent can be used to assume an AWS role, each user must authenticate and receive a token from the hologram server. To do this one must use the hologram-authorize utility.

When I try to authorize with the hologram, the server does a search for my user and tries to compare the the md5 password hash along with my ssh public key with the one in LDAP. I use FreeIPA as my LDAP server. When hologram searches for my user the results do not contain the userPassword field.

So I have a few questions?
Should my LDAP server return the userPassword md5 hash value? (doesn't seem like a secure thing to do)
Or am I doing something wrong?

I am open to the possibility that I am doing something wrong in some manner but I am not able to decern that from the hologram documentation.

https://github.com/AdRoll/hologram/blob/master/agent/support/darwin/hologram-boot.sh mentions the AdRoll server specifically. Is the script intended for public consumption?

Hologram doesn't expose the iam/info endpoint.

In an EC2 instance, the iam/info endpoint exposes (among other things) the ARN of the instance profile associated with the instance. However, with Hologram, there is no instance profile, only an arn. It could generate a fake instance profile based on the role ARN, e.g., if the current role ARN is arn:aws:iam::123456789012:role/MyRole then expose arn:aws:iam::123456789012:instance-profile/MyRole

This will solve one particular class of use case -- clients that expect the iam/info endpoint to exist but don't need it to resolve to the ARN of a real instance profile. See hashicorp/terraform#12704 and hashicorp/terraform#12951 for one such use case. But, it wouldn't solve for the use case where a client expects the returned ARN to correspond to an actual instance profile.

Thoughts?

We've got developer workstations covered but what about servers not running in EC2?

mTLS support would be nice to avoid local proxies to feed in the certs

I'm using gpg-agent 2.2.1 with the --enable-ssh-support flag set, on macOS Sierra. When I attempt to assume a role using Hologram, I receive the error message:

agent: failed to sign challenge

When I switch my SSH_AUTH_SOCK back over to the standard ssh-agent everything works fine again. Hologram is the only software that I've encountered issues with where gpg-agent will not operate as a drop-in replacement for ssh-agent, so I'm assuming that the problem is not with my configuration.

Any ideas?

I see some code (and even tests) in the server for refreshing credentials when an authentication failure occurs, but it doesn't seem to be happening in practice for my running server. It's kind of hard to reduce other than saying that I have the whole thing running and working, but after adding keys to my LDAP server, Hologram won't let people authenticate with the new keys until I restart the daemon.

I'm not sure how feasible this is, but judging by the pf.conf documentation, it should be possible to restrict access to 169.254.169.254 based on user. My concern is just that any multi-user Mac (do people actually do that?? this is mostly a theoretical concern for me) would allow users to impersonate the IAM role of the hologram user.

Possible approaches, from easier to harder:

1. Just configure pfctl to prevent anyone but the hologram user from touching 169.254.169.254
2. Perhaps pfctl is fancy enough to allow fancy rules that redirect to different ports based on user: in that case, we could actually support a proper multi-user hologram where each user gets redirected to a port that serves up different credentials.

On linux agents, the --uid-owner iptables flag allows us to do similar fanciness, and is probably more of a concern since multi-user linux boxes are more common.

In addition to verifying the user's SSH key, the hologram server should optionally require two-factor authentication using one-time passwords.

hologram use calls user.Current() (why? SSH authentication on the server side seems to just iterate over all known SSH keys) which seems to fail on darwin when cross-compiled from linux:

golang/go#6376

I'm using hologram against an active directory server, and in usercache.go:

filter, []string{"sshPublicKey", "cn"},
username := entry.GetAttributeValue("cn")

it would be nice if that cn field were configurable to be sAMAccountName, which is the Microsoft-flavored way to get a username.

hologram url could write a federated access URL for your current hologram user to stdout. On a Mac, you could then run hologram url | xargs open and it would pop open a browser pointing at the AWS console.

The logic for this could live in pure client code and wouldn't need any protocol modifications. I'm pretty sure I could implement it in a handful of lines if you were interested.

My main concern about this is that the logic doesn't really feel specific to hologram. We'd just be running a standard STS operation against the current instance credentials. Putting it under hologram would thus be just for the sake of a consistent Amazon "user experience", rather than because it needs to be here.

Thoughts?

truffles:hologram copumpkin$ ./hologram.sh build_all
>> Getting package github.com/golang/protobuf/...
>> Getting package golang.org/x/crypto/ssh
>> Getting package github.com/aybabtme/rgbterm
>> Getting package github.com/mitchellh/go-homedir
>> Getting package github.com/nmcclain/ldap
>> Getting package github.com/goamz/goamz/...
>> Getting package github.com/smartystreets/goconvey/...
>> Getting package github.com/howeyc/gopass
>> Getting package github.com/peterbourgon/g2s
# cd /go/src/golang.org/x/crypto; git pull --ff-only
From https://go.googlesource.com/crypto
 * [new branch]      master     -> origin/master
Your configuration specifies to merge with the ref 'master'
from the remote, but no such ref was fetched.
package github.com/howeyc/gopass
    imports golang.org/x/crypto/ssh/terminal: exit status 1

or

truffles:hologram copumpkin$ ./hologram.sh build_all
>> Getting package github.com/golang/protobuf/...
>> Getting package golang.org/x/crypto/ssh
>> Getting package github.com/aybabtme/rgbterm
>> Getting package github.com/peterbourgon/g2s
>> Getting package github.com/nmcclain/ldap
>> Getting package github.com/mitchellh/go-homedir
>> Getting package github.com/goamz/goamz/...
>> Getting package github.com/smartystreets/goconvey/...
>> Getting package github.com/howeyc/gopass
# cd /go/src/golang.org/x/crypto; git pull --ff-only
From https://go.googlesource.com/crypto
 * [new branch]      master     -> origin/master
There is no tracking information for the current branch.
Please specify which branch you want to merge with.
See git-pull(1) for details

    git pull <remote> <branch>

If you wish to set tracking information for this branch you can do so with:

    git branch --set-upstream-to=origin/<branch> master

package github.com/howeyc/gopass
    imports golang.org/x/crypto/ssh/terminal: exit status 1

It seems to say some variation of those every time I run it, and then proceeds to build as usual.

Main issues I see:

• There's a failure somewhere (or is it a warning?) that isn't causing the rest of the build to complain
• I didn't do anything special to make this happen

https://github.com/AdRoll/hologram/blob/master/server/bin/main.go#L133

I spent some time trying to figure out why the server would just die trying to talk to my LDAP server over port 389 before realizing that it was just trying to speak TLS to it. The readme doesn't mention it, and it doesn't seem fundamental. I realize that it's a sensible default to require TLS, but some LDAP servers (e.g., Amazon's Simple AD) don't support it.

See here:

tlsConfig := &tls.Config{
    InsecureSkipVerify: true,
}

Note that this is independent of the feature added in response to #8, which disables TLS altogether.

I'm trying to install hologram-agent on linux, but the mere method for building hologram seems broke. The ./hologram.sh build_all script tries to pull a container on dockerhub under the adroll/hologram_env repo except it doesn't exist, and when I try to build the image locally it fails to build due to gem install fpm --no-rdoc --no-ri failing to since it requires ffi requires Ruby version >= 2.3.

How are people installing hologram on linux? Or even macOS for that matter?

It seems like the hologram server is playing the part of a simplified SAML identity provider, and the client could be taking SAML assertions from such a provider and calling AssumeRoleWithSAML directly.

I'm wondering if the AdRoll team considered using that approach and what the downsides might be.

Currently, the temporary credentials are unlikely to leak and they will only last a short time if for whatever reason they do, but it would make me even more comfortable if the credentials were IP-locked.

In practice, this means intersecting the user's requested role with the following policy during the AssumeRole call.

{
    "Version": "2012-10-17",
    "Statement": [ {
        "Effect":    "Allow",
        "Action":    "*",
        "Resource":  "*",
        "Condition": { "IpAddress": { "aws:SourceIp": "my_ip" } }
    } ]
}

A complication is that in some setups (including mine), the IP I use to talk to the hologram server is not the same IP that I use to talk to AWS APIs, so the protocol would need to be amended to allow me to tell the hologram server what my public IP is.

Not sure it's necessarily worth the effort for most users, but it would be a nice cherry on the security cake.

There's also another slight complication, which is that locking by IP isn't currently supported for a handful of API calls. In most use cases, they aren't very common APIs to use. AWS is aware and will likely fix the issue soon.

I know that @epall might be interested in this, but others might be as well: I've pushed my NixOS service definition for the Hologram server to the nixpkgs master branch. This means that anyone running NixOS (the upcoming 15.06 release or people who like to live on unstable branches) can spin up a Hologram server super easily:

{
  ec2.metadata = true;

  services.hologram-server = {
    enable           = true;
    awsAccount       = "1234567890";
    awsDefaultRole   = "somerole";
    ldapHost         = "blah";
    ldapBaseDN       = "CN=something,DC=somethingelse";
    ldapBindDN       = "zomg";
    ldapBindPassword = "ihatepasswords";
  };

  networking.firewall.allowedTCPPorts = [ 3100 ];
}

Not sure if it's widespread enough to warrant a mention as a quick way to try it out, but I figured I'd point it out in case someone wanted to try it. It's probably an abuse of the ticket system so feel free to close this ticket either way 😄

When adding a new SSH Key through hologram-authorize, the server actually requests to the ldap server the user's password so it can compare its hash with the one sent by the user. Hologram should not request ldap for the user's password (among other reasons because implementations like AD won't actually return it) and should instead create a new connection to the ldap server and try to bind to it with the user-provided username/password combination.

( See https://github.com/AdRoll/hologram/blob/master/server/server.go#L152 )

On Mac, to make building possible user has to run boot2docker or vagrant. We should mention about it in the README file.

    bindErr := ldapServer.Bind(config.LDAP.Bind.DN, config.LDAP.Bind.Password)
    if bindErr != nil {
        log.Error("Could not bind to LDAP! %s", err.Error())

The Bind call is saved to bindErr, but the error message attempts to print err, which at that point will be nil. This leads to a nil pointer dereference error instead of the actual desired error message 😄

If I visit http://169.254.169.254/latest/meta-data/iam/security-credentials/, I see hologram-access in there with the usual credentials in it.

Note that although Amazon does not currently support multiple simultaneous roles, the URL scheme was clearly set up to allow for it.

I'm wondering (assuming that it doesn't break aws-cli, boto, and other major tools) if we could possibly allow a particular hologram agent to serve up multiple credential sets simultaneously, for APIs that are aware of it. In practice, this would just lead to there being multiple entries under security-credentials. If it does end up breaking existing APIs, we could also add a custom "revision" to the top level.

My use case is writing handy developer tools that know about Hologram and wants simultaneous access to multiple AWS accounts/roles. For example, I would like a simple cost-monitoring widget in my menubar that can talk to a particular Amazon account/role regardless of my current Hologram role.

I'm just a user of hologram me, not a developer. My colleagues have been distributing hologram and it works pretty well. However as a Mac user I've had some issues that don't get any attention. This seems to be the only place to ask some questions...

• Occasionally I will find my laptop fan running and when I check Activity Monitor I find hologram-boot is using a ton of CPU. Is that a known bug?
• Why is hologram-anything running when I've not used hologram me or anything else since a reboot? Can I make that stop?
• How do I uninstall everything hologram? My colleagues have only distributed an installer package.

Thanks. Sorry for polluting the "issues" log like this.

Is there possibility to run hologram-agent on the client-side machine using WSL (e.g. Ubuntu) in Windows 10?

I was just dealing with the client/server portion of the code and noticed that the handling for hologram me is almost identical to hologram use, except that it assumes the default role. Could the client/server portion be simplified by unifying the messages and logic processing them?

I had an existing setup on my hologram server that got wiped out when I reinstalled a new deb.

This is probably incompatible with how you've structured things internally, but it seems a tad more user-friendly (and also more compatible with what github enterprise excepts from an LDAP server). The difference boils down to calling ssh.ParseAuthorizedKey in the key loop in usercache.go instead of base64.StdEncoding.DecodeString and then ssh.ParsePublicKey. Perhaps it could be configurable?

I'm also happy to make PRs for these changes, if that's helpful.

I've experimented a bit with network namespaces on Linux and have been able to arrange for different processes to get different views of http://169.254.169.254.

What I would love is the ability to hologram different programs and shells independently. So I can keep one hologram pointing at one account in one terminal tab, and another hologram at a different account in another terminal tab. Given the network namespace work is done (I can post a sample script if it helps), I think this would be fairly easy.

Ultimately, I think it would make sense to break down the agent as follows:

1. A component that can serve up an EC2 metadata simulation with credentials supplied somehow (not necessarily hologram server; I have another use case mind which I might elaborate on in another ticket)
2. A hook for (1) that lets it get its credentials from the Hologram server
3. A user-facing program (perhaps called inrole or similar) that can set up the network namespaces as needed and spin up a one-off instance of (1) for the namespace.

After that, we can do things like:

inrole account1 role1 /bin/bash
inrole account3 role5 aws iam list-account-aliases

And then all the Mac OS users (me included) are sad.

Thoughts?

In my original PR for the feature, I had an LDAP query that checked for the role attribute. @BillMedernach changed it in 03519df to look for (objectClass=groupOfNames) and that class doesn't exist on several directory servers, which I assume breaks Hologram on servers other than the one AdRoll uses?

I only noticed this just now because I was running on my branch using the original code. Can we make that query configurable or make it more general somehow?

#79 upgraded to Go 1.7.1, but that doesn't seem sufficient to fix our issues. It seemed to work at first, but after a while, I get stack traces in the agent along the lines of:

failed MSpanList_Insert 0x8a48c8 0x2f9cbb4ded2cb 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.MSpanList_Insert(0x7e7bf8, 0x8a48c8)
    /usr/src/go/src/runtime/mheap.c:692 +0x8f
runtime.MHeap_Alloc(0x7e7640, 0x1, 0x10000000025, 0xe8e9)
    /usr/src/go/src/runtime/mheap.c:240 +0x66
runtime.MCentral_CacheSpan(0x7f0918, 0x0)
    /usr/src/go/src/runtime/mcentral.c:85 +0x167
runtime.MCache_Refill(0x89f000, 0x25, 0x800000000)
    /usr/src/go/src/runtime/mcache.c:90 +0xa0

goroutine 37 [running]:
runtime.switchtoM()
    /usr/src/go/src/runtime/asm_amd64.s:198 fp=0xc2080de1f0 sp=0xc2080de1e8
runtime.mallocgc(0x800, 0x3abec0, 0x1, 0x0)
    /usr/src/go/src/runtime/malloc.go:178 +0x849 fp=0xc2080de2a0 sp=0xc2080de1f0
runtime.newarray(0x3abec0, 0x800, 0x0)
    /usr/src/go/src/runtime/malloc.go:365 +0xc1 fp=0xc2080de2d8 sp=0xc2080de2a0
runtime.makeslice(0x39d320, 0x3d1, 0x800, 0x0, 0x0, 0x0)
    /usr/src/go/src/runtime/slice.go:32 +0x15c fp=0xc2080de320 sp=0xc2080de2d8
crypto/tls.(*block).reserve(0xc208082cc0, 0x452)
    /usr/src/go/src/crypto/tls/conn.go:438 +0x74 fp=0xc2080de3a0 sp=0xc2080de320
crypto/tls.(*block).readFromUntil(0xc208082cc0, 0x8b2988, 0xc20803c0a0, 0x452, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/conn.go:452 +0x73 fp=0xc2080de408 sp=0xc2080de3a0
crypto/tls.(*Conn).readRecord(0xc20807a2c0, 0x16, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/conn.go:586 +0xc60 fp=0xc2080de950 sp=0xc2080de408
crypto/tls.(*Conn).readHandshake(0xc20807a2c0, 0x0, 0x0, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/conn.go:779 +0xe7 fp=0xc2080deb10 sp=0xc2080de950
crypto/tls.(*clientHandshakeState).doFullHandshake(0xc208070480, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/handshake_client.go:227 +0x68 fp=0xc2080df0a8 sp=0xc2080deb10
crypto/tls.(*Conn).clientHandshake(0xc20807a2c0, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/handshake_client.go:197 +0x1981 fp=0xc2080df528 sp=0xc2080df0a8
crypto/tls.(*Conn).Handshake(0xc20807a2c0, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/conn.go:977 +0xf1 fp=0xc2080df568 sp=0xc2080df528
crypto/tls.DialWithDialer(0xc208044800, 0x53cdf0, 0x3, 0xc2080587d0, 0x50, 0xc2080d07e0, 0x50dce0, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/tls.go:141 +0x495 fp=0xc2080df678 sp=0xc2080df568
crypto/tls.Dial(0x53cdf0, 0x3, 0xc2080587d0, 0x50, 0xc2080d06c0, 0x0, 0x0, 0x0)
    /usr/src/go/src/crypto/tls/tls.go:165 +0x82 fp=0xc2080df6c8 sp=0xc2080df678
github.com/AdRoll/hologram/transport/remote.NewClient(0xc2080587d0, 0x50, 0x0, 0x0, 0x0, 0x0)
    /go/src/github.com/AdRoll/hologram/transport/remote/client.go:44 +0x260 fp=0xc2080df780 sp=0xc2080df6c8
github.com/AdRoll/hologram/agent.(*client).requestCredentials(0xc20801ec80, 0xc208044580, 0xc208087760, 0x17, 0x0, 0x0)
    /go/src/github.com/AdRoll/hologram/agent/client.go:169 +0x62 fp=0xc2080df980 sp=0xc2080df780
github.com/AdRoll/hologram/agent.(*client).AssumeRole(0xc20801ec80, 0xc208087760, 0x17, 0x0, 0x0)
    /go/src/github.com/AdRoll/hologram/agent/client.go:157 +0x112 fp=0xc2080df9d0 sp=0xc2080df980
github.com/AdRoll/hologram/agent.(*credentialsExpirationManager).maybeRefreshCredentials(0xc208044480, 0x0, 0x0)
    /go/src/github.com/AdRoll/hologram/agent/credentials_expiration_manager.go:63 +0x1e6 fp=0xc2080dfa68 sp=0xc2080df9d0
github.com/AdRoll/hologram/agent.(*credentialsExpirationManager).GetCredentials(0xc208044480, 0xc200000000, 0x0, 0x0)
    /go/src/github.com/AdRoll/hologram/agent/credentials_expiration_manager.go:48 +0x120 fp=0xc2080dfab8 sp=0xc2080dfa68
github.com/AdRoll/hologram/agent.(*metadataService).getCredentials(0xc20801ec60, 0x8b2a80, 0xc208064500, 0xc20803ad00)
    /go/src/github.com/AdRoll/hologram/agent/metadata_service.go:128 +0x53 fp=0xc2080dfc10 sp=0xc2080dfab8
github.com/AdRoll/hologram/agent.*metadataService.(github.com/AdRoll/hologram/agent.getCredentials)·fm(0x8b2a80, 0xc208064500, 0xc20803ad00)
    /go/src/github.com/AdRoll/hologram/agent/metadata_service.go:62 +0x45 fp=0xc2080dfc38 sp=0xc2080dfc10
net/http.HandlerFunc.ServeHTTP(0xc20800aca0, 0x8b2a80, 0xc208064500, 0xc20803ad00)
    /usr/src/go/src/net/http/server.go:1265 +0x41 fp=0xc2080dfc58 sp=0xc2080dfc38
net/http.(*ServeMux).ServeHTTP(0xc20800d920, 0x8b2a80, 0xc208064500, 0xc20803ad00)
    /usr/src/go/src/net/http/server.go:1541 +0x17d fp=0xc2080dfcb0 sp=0xc2080dfc58
net/http.serverHandler.ServeHTTP(0xc208056300, 0x8b2a80, 0xc208064500, 0xc20803ad00)
    /usr/src/go/src/net/http/server.go:1703 +0x19a fp=0xc2080dfd08 sp=0xc2080dfcb0
net/http.(*conn).serve(0xc208064320)
    /usr/src/go/src/net/http/server.go:1204 +0xb57 fp=0xc2080dffd8 sp=0xc2080dfd08
runtime.goexit()
    /usr/src/go/src/runtime/asm_amd64.s:2232 +0x1 fp=0xc2080dffe0 sp=0xc2080dffd8
created by net/http.(*Server).Serve
    /usr/src/go/src/net/http/server.go:1751 +0x35e

goroutine 1 [select]:
main.main()
    /go/src/github.com/AdRoll/hologram/cmd/hologram-agent/main.go:122 +0x1376

goroutine 5 [syscall]:
os/signal.loop()
    /usr/src/go/src/os/signal/signal_unix.go:21 +0x1f
created by os/signal.init·1
    /usr/src/go/src/os/signal/signal_unix.go:27 +0x35

goroutine 6 [IO wait]:
net.(*pollDesc).Wait(0xc208011250, 0x72, 0x0, 0x0)
    /usr/src/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc208011250, 0x0, 0x0)
    /usr/src/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc2080111f0, 0x0, 0x8b0be0, 0xc208088018)
    /usr/src/go/src/net/fd_unix.go:419 +0x40b
net.(*TCPListener).AcceptTCP(0xc20803c078, 0xc20805edf8, 0x0, 0x0)
    /usr/src/go/src/net/tcpsock_posix.go:234 +0x4e
net.(*TCPListener).Accept(0xc20803c078, 0x0, 0x0, 0x0, 0x0)
    /usr/src/go/src/net/tcpsock_posix.go:244 +0x4c
net/http.(*Server).Serve(0xc208056300, 0x8b2708, 0xc20803c078, 0x0, 0x0)
    /usr/src/go/src/net/http/server.go:1728 +0x92
net/http.Serve(0x8b2708, 0xc20803c078, 0x8b2888, 0xc20800d920, 0x0, 0x0)
    /usr/src/go/src/net/http/server.go:1606 +0xa1
github.com/AdRoll/hologram/agent.(*metadataService).listen(0xc20801ec60)
    /go/src/github.com/AdRoll/hologram/agent/metadata_service.go:67 +0x3be
created by github.com/AdRoll/hologram/agent.(*metadataService).Start
    /go/src/github.com/AdRoll/hologram/agent/metadata_service.go:50 +0x44

goroutine 7 [IO wait]:
net.(*pollDesc).Wait(0xc2080112c0, 0x72, 0x0, 0x0)
    /usr/src/go/src/net/fd_poll_runtime.go:84 +0x47
net.(*pollDesc).WaitRead(0xc2080112c0, 0x0, 0x0)
    /usr/src/go/src/net/fd_poll_runtime.go:89 +0x43
net.(*netFD).accept(0xc208011260, 0x0, 0x8b0be0, 0xc208088748)
    /usr/src/go/src/net/fd_unix.go:419 +0x40b
net.(*UnixListener).AcceptUnix(0xc20801ed40, 0x36aa0, 0x0, 0x0)
    /usr/src/go/src/net/unixsock_posix.go:282 +0x56
net.(*UnixListener).Accept(0xc20801ed40, 0x0, 0x0, 0x0, 0x0)
    /usr/src/go/src/net/unixsock_posix.go:293 +0x4c
github.com/AdRoll/hologram/transport/local.(*server).listen(0xc20801ed60)
    /go/src/github.com/AdRoll/hologram/transport/local/server.go:33 +0x4f
created by github.com/AdRoll/hologram/transport/local.NewServer
    /go/src/github.com/AdRoll/hologram/transport/local/server.go:65 +0x121

goroutine 20 [sleep]:
net.func·019()
    /usr/src/go/src/net/dnsclient_unix.go:240 +0x5a
created by net.loadConfig
    /usr/src/go/src/net/dnsclient_unix.go:269 +0x20c
failed MSpanList_Insert 0x8a3000 0x2f9cbb56eb615 0x0
fatal error: MSpanList_Insert

runtime stack:
runtime.throw(0x7be56b)
    /usr/src/go/src/runtime/panic.go:491 +0xad fp=0x7fff5fbffa50 sp=0x7fff5fbffa20
runtime.MSpanList_Insert(0x7eaa48, 0x8a3000)
    /usr/src/go/src/runtime/mheap.c:692 +0x8f fp=0x7fff5fbffa78 sp=0x7fff5fbffa50
MHeap_FreeSpanLocked(0x7e7640, 0x8a3000, 0x100)
    /usr/src/go/src/runtime/mheap.c:583 +0x163 fp=0x7fff5fbffab8 sp=0x7fff5fbffa78
MHeap_Grow(0x7e7640, 0x8, 0x0)
    /usr/src/go/src/runtime/mheap.c:420 +0x1a8 fp=0x7fff5fbffaf8 sp=0x7fff5fbffab8
MHeap_AllocSpanLocked(0x7e7640, 0x1, 0x0)
    /usr/src/go/src/runtime/mheap.c:298 +0x365 fp=0x7fff5fbffb38 sp=0x7fff5fbffaf8
mheap_alloc(0x7e7640, 0x1, 0x12, 0x0)
    /usr/src/go/src/runtime/mheap.c:190 +0x121 fp=0x7fff5fbffb60 sp=0x7fff5fbffb38
runtime.MHeap_Alloc(0x7e7640, 0x1, 0x10000000012, 0xe8e9)
    /usr/src/go/src/runtime/mheap.c:240 +0x66 fp=0x7fff5fbffb98 sp=0x7fff5fbffb60
MCentral_Grow(0x7ef3b8, 0x0)
    /usr/src/go/src/runtime/mcentral.c:197 +0x8b fp=0x7fff5fbffc00 sp=0x7fff5fbffb98
runtime.MCentral_CacheSpan(0x7ef3b8, 0x0)
    /usr/src/go/src/runtime/mcentral.c:85 +0x167 fp=0x7fff5fbffc38 sp=0x7fff5fbffc00
runtime.MCache_Refill(0x89f000, 0x12, 0x0)
    /usr/src/go/src/runtime/mcache.c:90 +0xa0 fp=0x7fff5fbffc60 sp=0x7fff5fbffc38
runtime.mcacheRefill_m()
    /usr/src/go/src/runtime/malloc.c:368 +0x57 fp=0x7fff5fbffc80 sp=0x7fff5fbffc60
runtime.onM(0x632bf8)
    /usr/src/go/src/runtime/asm_amd64.s:273 +0x9a fp=0x7fff5fbffc88 sp=0x7fff5fbffc80
runtime.mallocgc(0x120, 0x518020, 0x0, 0x0)
    /usr/src/go/src/runtime/malloc.go:178 +0x849 fp=0x7fff5fbffd38 sp=0x7fff5fbffc88
runtime.newobject(0x518020, 0x89f000)
    /usr/src/go/src/runtime/malloc.go:353 +0x49 fp=0x7fff5fbffd60 sp=0x7fff5fbffd38
runtime.newG(0x283ba)
    /usr/src/go/src/runtime/proc.go:233 +0x2a fp=0x7fff5fbffd78 sp=0x7fff5fbffd60
allocg(0x7d7520)
    /usr/src/go/src/runtime/proc.c:925 +0x1f fp=0x7fff5fbffd88 sp=0x7fff5fbffd78
runtime.malg(0x8000, 0x7d76e0)
    /usr/src/go/src/runtime/proc.c:2106 +0x1f fp=0x7fff5fbffdb8 sp=0x7fff5fbffd88
runtime.mpreinit(0x7d7f20)
    /usr/src/go/src/runtime/os_darwin.c:137 +0x27 fp=0x7fff5fbffdd0 sp=0x7fff5fbffdb8
mcommoninit(0x7d7f20)
    /usr/src/go/src/runtime/proc.c:201 +0xc9 fp=0x7fff5fbffdf8 sp=0x7fff5fbffdd0
runtime.schedinit()
    /usr/src/go/src/runtime/proc.c:138 +0x55 fp=0x7fff5fbffe20 sp=0x7fff5fbffdf8
runtime.rt0_go(0x7fff5fbffe50, 0x1, 0x7fff5fbffe50, 0x0, 0x1, 0x7fff5fbffee0, 0x0, 0x7fff5fbffefe, 0x7fff5fbfff21, 0x7fff5fbfff46, ...)
    /usr/src/go/src/runtime/asm_amd64.s:95 +0x116 fp=0x7fff5fbffe28 sp=0x7fff5fbffe20

Currently when the hologram-agent is not running a user gets the following cryptic error message:
dial tcp: missing address

It would be much more helpful if the user is notified that the hologram-agent isn't running.

At the highest level, my goal is for different hologram users to have different access to different roles.

I don't know of the best way to do this, but off the top of my head, we could have a custom LDAP property that lists ARNs of roles the particular user is allowed to assume. Ideally the system would also be group-aware, and a user's access would be the union of the ARNs associated with the user's groups and the user's own special access.

Then, I think I would just type hologram use arn:aws:iam::123456789012:role/mytest (being able to enter full ARNs is nice for cross-account access) and have it just work.

Is anything blatantly wrong with this? Can it be done more cleanly? Does it seem like the sort of thing that belongs in hologram?

I have deployed the Hologram server to an EC2 instance running Debian and when I run hologram use foo I see the connection opened on the server but then something is closing it. BTW, I'm building the binaries from the credential_keys_in_text_file branch.

[INFO   ] 2017-05-02T19:33:23Z Hologram server is online, waiting for termination.
[INFO   ] 2017-05-02T19:33:34Z Enabling debug mode.
[DEBUG  ] 2017-05-02T19:33:41Z (/go/src/github.com/AdRoll/hologram/server/server.go:59) Opening new connection handler.
[DEBUG  ] 2017-05-02T19:33:41Z (/go/src/github.com/AdRoll/hologram/server/usercache.go:328) Could not find derp in the keys file cache; updating from the file.
[ERROR  ] 2017-05-02T19:33:41Z (/go/src/github.com/AdRoll/hologram/server/server.go:65) Error reading data from stream: read tcp y.y.y.y:3100->x.x.x.x:33059: use of closed network connection