Make the following (property -> value) pairs acceptable during API requests in addition to only their ids (current behavior).

Blueprints

• zone -> zone.title
• profile -> profile.slug
• size -> instance_size.instance_id

Other possible options

• security_group -> security_group.group_id

Specifically lines 31 and 32 of setup.py. This causes the install to fail b/c at the point where we import stackdio we don't yet have django installed.

Narrative

Sometimes we run into an issue where salt still keeps track of keys for hosts that have died or been deleted (usually on spot clusters). This takes a big hit on performance when salt is provisioning or orchestrating, so it would be good if we can automatically delete these keys from salt. After talking with @WLPhoenix, we think it would be good if we can have a cron job that will periodically check the status of all salt minions and delete keys for minions that are non-responsive.

Development Tasks

Write a cron job that:

1. Purges the keys for all hosts living in terminated, or deleted stacks
2. For any stack that has all hosts dead, change the status to either terminated or stopped depending on the status of the hosts
3. Attempt to re-launch & re-provision stacks which have some hosts that have died

• Part 3 could be kind of hairy, because sometimes the hosts are still alive and just the minion daemon has died
• This will never delete a stack from the stackdio database, only alter its status
• We DO NOT want to delete keys for stopped stacks. When you try to start them again, they reuse the same keys. Salt cloud does not regenerate keys in this case.

Note: This would all be done using some combination salt-cloud -Q to check the status of the hosts and salt manage.down to see a list of unresponsive hosts.

I have noticed that we are having some issues where owner tags are not being put on clusters that are stood up. I have noticed some issues with the name tag not showing up sometimes as well.

Stack trace:

Traceback:
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/django/core/handlers/base.py" in get_response
  113.                         response = callback(request, *callback_args, **callback_kwargs)
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/django/views/generic/base.py" in view
  68.             return self.dispatch(request, *args, **kwargs)
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/django/views/decorators/csrf.py" in wrapped_view
  77.         return view_func(*args, **kwargs)
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/rest_framework/views.py" in dispatch
  400.             response = self.handle_exception(exc)
File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/rest_framework/views.py" in dispatch
  397.             response = handler(request, *args, **kwargs)
File "/home/stackdio/stackdio/stackdio/server/core/api.py" in get
  113.         from stackdio import get_version

Exception Type: ImportError at /api/version/
Exception Value: cannot import name get_version

If we can set the block device mappings, then our volumes can be attached when the instance starts rather than later.

There's currently no way to upgrade a backing profile without re-creating all blueprints that depend on it (either by creating a new profile or deleting it and re-creating). Making image_id editable would fix that.

I would like to have the ability to schedule a stack for termination. This would give me the option of scheduling stacks to be deleted when they are no longer necessary. Say I only need a stack for 2 weeks, then I can schedule a stack to be automatically deleted in two weeks.

The quickstart script (scripts/stackdio_quickstart.sh) is pointing to the old (private) repo link for stackdio (line 101), and needs to be updated to point to the new (public) repo.

Currently, the API allows you to change the name of the blueprint, which is desired. However, the location of the props file on the file system is dependent on the blueprint name, and that location isn't changed when the name is changed. Therefore, if you were to rename a blueprint, then create a new blueprint with the same name as the old one, your properties for the two blueprints would reference the same file and be forever linked.

Steps to recreate:

• Add a new profile
• Add VPC id
• Note that region is disabled
• Submit new profile
• See 400 error, region required

Creating this issue for long term discussions regarding code quality and CI.

The UI really needs some work to be ship-shape for a public release. Known issues include:

• No error handling
• Unable to edit blueprints
• Enable/disable issues (#152 for example)

Feel free to add more as found.

We need to decide if it's worth continuing on with the current UI framework, or if it's better to cut our losses and start from scratch with something like Angular.

The api will allow you to create a blueprint that has different orderings for the same role on different hosts. We should catch this at blueprint creation time.

The biggest issue here is that it could cause circular requisites in the overstate file, and neither stackdio nor salt will catch this, so you end up with a "maximum recursion depth exceeded" error.

We want the ability to enable termination protection on stacks in order to avoid accidentally deleting the wrong stack.

Development Tasks

• Add 'protected' as an attribute to the Stack model
• If 'protected' is set, turn on termination protection when launching stacks
• Also if 'protected' is set, remove 'Delete' and 'Terminate' from the available actions.

We need an endpoint to list all zones at once, without hitting every region endpoint.

Currenly, when you create a volume associated with a host on a stack, it is always a standard (magnetic) volume, and we would like to be able to create sad volumes.

Pip changed the signature on parse_requirements in version 6, and it now requires a session variable.

We're running into issues with administrators trying to create and manage blueprints that other users can then launch. It's worth looking at whether a [create,edit,launch,delete,etc] permission per object would make more sense than our current model of owners and admins.

We have an ldap_guide.md file as a placeholder, but we need to fill it with useful information.

[2014-11-03 10:06:00,059: ERROR/MainProcess] Task stacks.destroy_hosts[4063f2b2-6b80-423b-8825-299bd2b38faa] raised exception: APIException()
Traceback (most recent call last):
  File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/celery/task/trace.py", line 233, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/stackdio/.virtualenvs/stackdio/lib/python2.6/site-packages/celery/task/trace.py", line 420, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/stackdio/stackdio/stackdio/server/stacks/tasks.py", line 1916, in destroy_hosts
    driver.delete_security_group(security_group.name)
  File "/home/stackdio/stackdio/stackdio/server/cloud/providers/aws.py", line 540, in delete_security_group
    raise BadRequest(e.error_message)
BadRequest

It looks like what's happening is a host takes an exceptionally long time to shut down, which is fine. StackdIO gives a reasonable time-out error for that. After that, it doesn't detect that there are hosts to shut down, and proceeds to immediately try to delete the security group. Because that host still exists, it fails at the AWS level.

This is so that in the API web interface we don't see fields that are irrelevant during creation of entities (like statuses, etc...).