Giter VIP home page Giter VIP logo

gluettalax's Introduction

GLUEttalax

Glue ETL without constipation

Build Status PyPI version PyPI Downloads Code style: black

usage: gluettalax <command> [parameters]

Commands:
 add_partition <db> <table> [--partition1=value...] [--location=path]
    Create a new Glue partition.
    Example: add_partition datalake usage --year=2019 --month=09

 add_partitions <db> <table> [s3_path]
    Create new Glue partitions in a given location.
    Example: add_partition datalake usage s3://example/usage/year=2020/month=10

 del_partition <db> <table> [--partition1=value...]
    Delete a Glue partition.
    Example: del_partition datalake usage --year=2019 --month=09

 help [command]
    Display information about commands.

 list_crawlers [pattern] [--noheaders]
    List Glue crawlers.
    Example: list_crawlers 'test*' --noheaders

 list_jobs [pattern] [--noheaders]
    List Glue jobs.
    Example: list_jobs 'test*'

 list_partitions <db> <table> [pattern] [--noheaders]
    List the partitions in a table.
    Example: list_partitions datalake usage

 list_runs [job_name] [--lines=num] [--noheaders]
    Print Glue jobs history.
    Example: list_runs my_batch_job --lines 10

 list_tables [pattern] [--noheaders]
    List Glue tables.
    Example: list_tables 'test*' --noheaders

 run_crawler <crawler_name> [--async] [--timeout=seconds]
    Run a crawler. If not async, wait until execution is finished.
    Example: run_crawler my_usage_crawler --async

 run_job <job_name> [--async] [--param1=value...]
    Run a Glue job. if not async, wait until execution is finished.
    Example: cmd_run_job --DATALAKE_BUCKET=test --THE_DATE=20191112 --HOUR=15

Command aliases:
 lsc -> list_crawlers
 lsj -> list_jobs
 runc -> run_crawler
 lsr -> list_runs
 runj -> run_job
 lsp -> list_partitions
 addp -> add_partition
 rmp -> del_partition
 lst -> list_tables

gluettalax's People

Contributors

andreax79 avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.