Giter VIP home page Giter VIP logo

gost's Introduction

Gost

Build Status Coverage Status Go Report Card

Filesystem abstraction layer for Golang, that works with Local filesystem and Amazon S3 with a unified API. You can even copy-paste files from different sources. FTP, Dropbox etc. will follow soon.

Quick Example

import "github.com/usmanhalalit/gost/s3"

// Initialize a filesystem
fs, err := s3.New(s3.Config{ your-aws-credentials })

// Read
note, err := fs.File("my-note.txt").ReadString()
//Write
err := fs.File("another-note.txt").WriteString("another note")

// Traverse naturally
movies := fs.Directory("movies")
files := movies.Files()
movies.File("Pirated-movie.mp4").Delete()

// Copy file from one source to another
localFile := lfs.File("photo.jpg")
s3Dir := fs.Directory("photos")
err := localFile.CopyTo(s3dir)

Initialize

Get the library:

go get github.com/usmanhalalit/gost

You just initialize the S3 and Local adapters differently, everything else in the API is same.

Amazon S3

import "github.com/usmanhalalit/gost/s3"

fs, err := s3.New(s3.Config{
	ID: "aws-id",
	Key: "aws-key",
	Region: "es-west-1",
	Bucket: "your-bucket",
})

Local

import "github.com/usmanhalalit/gost/local"

fs, err := local.New(local.Config{
	BasePath: "/home/user",
})

Read and Write

Read

Simple read, suitable for small files.

fileContent, err := fs.File("test.txt").ReadString()

Bytes read, compatible with io.Reader, so you can do buffered read.

b := make([]byte, 3)
n, err := fs.File("test.txt").Read(b)

Write

Simple write

fs.File("test.txt").WriteString("sample content")

Bytes write

n, err := file.Write(bytes)
// n == number of bytes written

Traversing

You can explore the filesystem like you in your desktop file explorer. File and directories are chained in a natural way.

dirs, err := fs.Directory("Parent").Directory("Child").Directories()
files, err := fs.Directory("Parent").Directory("Child").Files()
dirs, err := fs.Directory("Parent").Directory("Child").Files()

Listing

Get all files and loop through them

files, err := fs.Files()
for _, file := range files {
    fmt.Println(file.ReadString())
}

Get all directories and loop through them

dirs, err := fs.Directories()
for _, dir := range dirs {
    files := dir.Files()
    fmt.Println(files)
}

Get the directory which contains a file

dir := fs.File("test.txt").Directory()

Stat

Get file size and last modified timestamp:

stat, _ := fs.File("test.txt").Stat()
fmt.Println(stat.Size)
fmt.Println(stat.LastModified)

You can get stat of directories too, but it's not available on S3.

fs.Directory("Downloads").File("test.txt").GetPath()

Create and Delete

Delete a file and directory:

fs.File("test.txt").Delete()
// Delete an entire directory, beware please!
fs.Directory("Images").Delete()

Create a new directory:

fs.Directory("Images").Create()

To create a new file simply write something to it:

fs.File("non_existent_file").WriteString("")

Copy and Paste Between Different Sources

You can copy a file to any Directory, be it in in the same filesystem or not(local or S3)

localFile := lfs.File("photo.jpg")
s3Dir := s3fs.Directory("photos")
err := localFile.CopyTo(s3dir)

Fun, eh?

You can optionally provide a new filename too:

err := localFile.CopyTo(anotherDir, "copied_file.jpg")

Also there is a helper to copy file in the same Directory:

file.Copy("copied_file.jpg")

Custom Adapter

Yes, you can write one and it'll be appreciated if you contribute back. . gost.go file has all the interfaces defined. Basically you've to implement gost.File and gost.Directory interfaces. Check the local adapter to get an idea.

API Documentation

Please follow the Go Doc: https://godoc.org/github.com/usmanhalalit/gost

Also check the _test files here to get more idea about the usage.


You can follow me on Twitter ๐Ÿ™‚

ยฉ Muhammad Usman. Licensed under MIT license.

gost's People

Contributors

nicored avatar sharonjl avatar usmanhalalit avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

gost's Issues

Very nice work!

how can i connect to this S3 api? if it's not available, can u do the api for it?

e.g. would like to use it as a s3 alternative using my server's local filesystem instead of s3 so need to connection from external app to use this one.

how can i get this done? pls help. will buy u a coffee if it's already done

Recursively directory creation not work

Code:

var Directory gost.Directory = initDirectory()

func initDirectory() gost.Directory {
        if dir, err := local.New(local.Config{
		BasePath: "/path/to/public",
	}); err != nil {
		log.Fatal(err.Error())
	} else {
		return dir
	}

	return nil
}

func MakeTestFile() {
	newFile := Directory.File("test/test/test.txt")

	if !newFile.Exists() {
		log.Println("File does not exist")
		if err := newFile.Directory().Create(); err != nil {
			log.Fatal(err.Error())
		}
	}

	if n, err := newFile.Write([]byte("Test txt")); err != nil {
		log.Fatal(err.Error())
	} else if n == 0 {
		log.Println("Null bytes writed")
	}
}

Problem:

When I try to create a folder in a subdirectory higher than one level, I get the error on newFile.Directory().Create():

File does not exist
Fatal: mkdir /path/to/public/uploads/test/test: no such file or directory

Because there is no test folder in the uploads folder. What can I do in such a situation? If I create folders using os, I cannot use S3 in production

Compatibility with webdav.Filesystem interface

Hello,

Thanks for your work.

What would have been even greater is to keep the compatibility with the existing File/Filesystem interfaces to provide an abstraction layer that can be used with existing libraries (webdav.FileSystem and webdav.File for example).

An other point : using minio-go instead of aws sdk would enable use of s3 compatible services.

Best regards.

Permissions on local directory

I can't access local directories created by the library, it seems as though the directories are given the permission 644 instead of 777.

Remove global service

Your S3 interface currently implements a global service that's used for the file system, which is not ideal when working with multiple buckets in different regions as the creation of subsequent filesystems will error as it will reuse the service created in the original file system.

I propose that the SetService function be removed as well as the global service in s3/s3.go

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.