askmanta's Introduction

Ask Manta

Ask Manta is a job runner for Joyent's Manta cloud storage and map-reduce service. Like Joyent's own mjob, but higher-level. Uploads dependencies, tracks running jobs, returns any errors and so on.

CLI

Ask Manta is available on the commandline: the askmanta command (just ask manta). Run askmanta -h to get an idea of what subcommands are available.

Job files

Ask Manta job definitions are YAML files will look reasonably familiar if you've used Manta step definitions before.

askmanta's People

Contributors

Watchers

askmanta's Issues

Install fails bc of configuation requirements

If I install askmanta without first configuring environmental variables, it gives me this error:

  Running setup.py (path:/var/folders/4m/zzbwyss11r77z85h92r63lcr0000gn/T/pip-VuyTdS-build/setup.py) egg_info for package from git+https://github.com/stdbrouw/askmanta
    Traceback (most recent call last):
      File "<string>", line 17, in <module>
      File "/var/folders/4m/zzbwyss11r77z85h92r63lcr0000gn/T/pip-VuyTdS-build/setup.py", line 3, in <module>
        from askmanta import VERSION
      File "askmanta/__init__.py", line 5, in <module>
        import job, manifest
      File "askmanta/job.py", line 5, in <module>
        from askmanta.environment import client
      File "askmanta/environment.py", line 9, in <module>
        if 'localhost' in os.environ['MANTA_URL']:
      File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/UserDict.py", line 23, in __getitem__
        raise KeyError(key)
    KeyError: 'MANTA_URL'

This should be somehow covered on initialization of jobs etc or more explicitly stated.

i wrote a connect module but wasn't sure how to integrate it, hence no PR ;).

import os
import logging
import manta 

logging.basicConfig()

MANTA_URL = os.getenv('MANTA_URL')
MANTA_USER = os.getenv('MANTA_URL')
MANTA_KEY_ID = os.getenv('MANTA_KEY_ID')

class MantaConnectionError(Exception):
  pass

def _check_env():
  """
  Make sure environmental variables are present.
  """
  if not MANTA_URL:
    raise MantaConnectionError(
      'You must set MANTA_URL as an environmental variable.'
    )

  if not MANTA_USER:
    raise MantaConnectionError(
      'You must set MANTA_USER as an environmental variable.'
    )

  if 'localhost' not in MANTA_URL:
    if not MANTA_KEY_ID:
      raise MantaConnectionError(
        'For a non-local client, you must set'
        'MANTA_KEY_ID as an environmental variable.'
      )  

def connect():
  """
  connect to manta 
  """
  _check_env()

  if 'localhost' in MANTA_URL:
    client = manta.MantaClient(MANTA_URL, MANTA_USER, internal=True)
  else:
    signer = manta.SSHAgentSigner(MANTA_KEY_ID)
    client = manta.MantaClient(MANTA_URL, MANTA_USER, MANTA_KEY_ID)

  return client

# HACK for now, should figure out how 
# to announce errors on init of various classes.
client = connect()