Giter VIP home page Giter VIP logo

s3-tree's Introduction

s3-tree Build Status

Generates a tree of an S3 bucket contents

WARNING! The number of HTTP calls are not optimized. Does a request per 'folder'.

Install

npm i -S s3-tree

Usage

const s3tree = require("s3-tree");

const generator = s3tree({ bucket: "my-bucket-name" });

const tree = await generator.generate("/my-folder/subfolder/");
console.log(JSON.stringify(tree, null, 2));

Will log something like:

{
  "file1": "my-folder/subfolder/file1",
  "file2": "my-folder/subfolder/file2",
  "folder": {
    "file3": "my-folder/subfolder/folder/file3",
    "file4": "my-folder/subfolder/folder/file4",
    "sub-folder": {
      "file5": "my-folder/subfolder/folder/sub-folder/file5",
      "file6": "my-folder/subfolder/folder/sub-folder/file6",
      "sub-sub-folder": {
        "file7": "my-folder/subfolder/folder/sub-folder/sub-sub-folder/file7",
        "file8": "my-folder/subfolder/folder/sub-folder/sub-sub-folder/file8"
      }
    }
  }
}

API

The s3tree accepts two options:

  • bucket - Obligatory. The S3 bucket name
  • s3 - Optional. The aws-sdk S3 class instance. For example: new AWS.S3({apiVersion: '2006-03-01')

The s3tree.generate(path, depth) function takes:

  • path - any string. E.g.
    • "/", "", or
    • "/folder", "folder/", "folder", or
    • "/1/2/3/4", "1/2/3/4/", "1/2/3/4", etc.
  • depth - Optional. The number of folders deep to traverse and generate. 0 will return only the direct contents of the folder.

Returns a Promise.

CLI

Install

$ npm i -g s3-tree

Usage:

s3-tree BUCKET [PATH] [DEPTH]

Example:

s3-tree my-cat-gifs / 10

Prints JSON object (aka the tree). Example:

$ s3-tree my-bucket-name my-folder/subfolder/
{
  "file1": "my-folder/subfolder/file1",
  "file2": "my-folder/subfolder/file2",
  "folder": {
    "file3": "my-folder/subfolder/folder/file3",
    "file4": "my-folder/subfolder/folder/file4",
    "sub-folder": {
      "file5": "my-folder/subfolder/folder/sub-folder/file5",
      "file6": "my-folder/subfolder/folder/sub-folder/file6",
      "sub-sub-folder": {
        "file7": "my-folder/subfolder/folder/sub-folder/sub-sub-folder/file7",
        "file8": "my-folder/subfolder/folder/sub-folder/sub-sub-folder/file8"
      }
    }
  }
}
$

CHANGELOG

v1.0

  • Node callbacks interface. No Promises. See README here.
  • The deep dependency aws-sdk version was >=2.0.0 <3.0.0.

v2.0

  • Only promises are supported now. For node callbacks use generator.generate(path).then(r => cb(null, r), cb)
  • The deep dependency aws-sdk version is >=2.3.0 <3.0.0 now.
  • node.js < v4 are not supported, unless you polyfill Promises global.Promise = smthng;

v3.0

  • Dropped lodash dependency.
  • Modernised code from ES5->ES6. node.js < 8 are not supported
  • No API changes.

v3.1

  • New depth property to limit how much deep the tree should be traversed.

s3-tree's People

Contributors

drewvartanian avatar koresar avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar

s3-tree's Issues

question..

Hi there,
this might be like something I am looking for..

Basically I need to walk an entire s3 bucket hierarchy find files in each bucket/folder and process the contents..

Would this code help me ?

create html output

is it possible to add a cli argument to create html tree view from result?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.