Giter VIP home page Giter VIP logo

emkay.s3's Introduction

Emkay.S3

This package contains a small wrapper for Amazon S3. It provides possibility to upload files, upload the content of a folder (including subfolders!), enumerate buckets, enumerate the content of a specific 'subfolder', delete buckets and delete files from specific subfolders. All those are also available via MSBuild tasks which are also part of this package. Which is the main motivation of this project.

##Download

The Emkay.S3 library is available on nuget.org via package name Emkay.S3.

To install it, run the following command in the Package Manager Console

PM> Install-Package Emkay.S3

More information about NuGet package avaliable at https://nuget.org/packages/Emkay.S3

##Getting Started

In order to use the tasks in your project, you need to import the Emkay.S3.Tasks.targets file. Maybe you need to adjust the paths to your needs.

<PropertyGroup>
	<EmkayS3ClassLibrary>$(MSBuildStartupDirectory)\Lib\Emkay.S3.dll</EmkayS3ClassLibrary>
</PropertyGroup>
<UsingTask AssemblyFile="$(EmkayS3ClassLibrary)" TaskName="PublishFolder" />
<UsingTask AssemblyFile="$(EmkayS3ClassLibrary)" TaskName="PublishFiles" />

Emkay S3 folder publisher is an MSBuild task which can be used for publishing recursively the content of a folder to your S3 bucket. After that you can publish the content of a source folder to S3 by using this statement inside an MSBuild target. By default the files will be public available. Region is "us-east-1" by default, but any of the AWS S3 region names can be used.

<Target Name="S3_upload" DependsOnTargets="Publish">
	<Message Text="Publishing to S3 ..." />

	<Message Text="Source folder: $(source)"/>
	<Message Text="Bucket: $(S3_bucket)"/>
	<Message Text="Destination folder: $(S3_subfolder)"/>

	<PublishFolder
  		Key="$(S3_key)"
  		Secret="$(S3_secret)"
  		Region="$(S3_region)"
  		SourceFolder="$(source)"
  		Bucket="$(S3_bucket)"
  		DestinationFolder="$(S3_subfolder)" />
</Target>

PublishFolderWithHeaders example:

<Target Name="S3_upload" DependsOnTargets="Publish">
  <ItemGroup>
	<Directories Include="css;scripts;">
		<Content-Type>text/css</Content-Type>
		<Content-Encoding>gzip</Content-Encoding>
	</Directories>
  </ItemGroup> 

	<PublishFolderWithHeaders
		Key="(Key)"
		Secret="(Secret)"
		Region="us-east-1"
		SourceFolders="$(Directories)"
		Bucket="$(Bucket)"
		DestinationFolder="$(DestinationFolder)" />
</Target>

License

The source code is available under the MIT license.

emkay.s3's People

Contributors

afinzel avatar cankoluman avatar makomweb avatar uluhonolulu avatar

Stargazers

 avatar  avatar  avatar

Watchers

 avatar  avatar

emkay.s3's Issues

Update Nuget

Hey guys,

Any chance you can update the nuget package if it now works with the new amazon sdk?

Use single PublishFiles, PublishFolders (remove *WithHeaders tasks) ?

I was looking to add a couple things, but the code seems like it could use some cleanup.

One of these things is separate method for "WithHeaders". Why bother? Why not just call PublishFiles? If you pass header metadata (which I guess is any metadata?) it sets headers. If you don't want to set headers, don't pass metadata.

As is, there's effectively two implementations of Publish methods and 3 layers of duplicated code leading into S3Client.PutFile() or S3Client.PutFileWithHeaders() which are 98% identical.

Perhaps could deprecate the WithHeaders methods, leaving them in (inheriting and identical), but I get the sense this isn't heavily used enough that this would be a huge deal..?

Display the stack trace on an error/exception

When an error occurs during publishing, only the message of the exception is logged:
https://github.com/makomweb/Emkay.S3/blob/master/Emkay.S3/PublishFiles.cs#L50

I would like to be able to see the entire details of the exception, including the stack trace to help diagnose and troubleshoot the issue.

I am currently experiencing an error and this is the only error message in the build log:

Publishing folder has failed because of A WebException with status RequestCanceled was thrown.

I would probably make a pull request to the forked repository here:
https://github.com/Dealermade/DM.Emkay.S3

Thank you.

AWSSDK 2.6.3 Breaks

Hey,

It may be me but I think your repo is now broken:

  1. Your dependency is the incorrect version of AWSSDK (I did a PR to fix that up)
  2. I got "Publishing folder has failed because of No RegionEndpoint or ServiceURL configured", this is fixed by specifying the region in S3Client when creating the AmazonS3Client.
  3. After fixing those you get "Method not found: 'Void Amazon.S3.Model.PutObjectRequest.set_Timeout(System.Nullable`1<System.TimeSpan>)'", http://stackoverflow.com/questions/8608274/having-problems-accessing-bucket-that-is-not-using-us-region-with-the-net-sdk suggests that this method is no longer available.

I am going to roll back to the previous version. I stopped after getting error 3. I hope this helps you.

Adam

Cleanup: Remove *Factory and Interfaces

There's a ton of extra junk in here that seems unnecessary to me. First of all, Interfaces that all only have one implementation is a bit useless. Is there a concrete case where other instances are used?

Moq is used in unit tests, but Moq can be used to mock concrete classes.. so having all the extra interfaces just to support Moq (which doesn't need them) seems overkill to me. I realize this can be a contentious issue as some people don't like to mock concrete classes, but it adds lots of extra code and layers to deal with with absolutely zero benefit.

The other one that I find extraneous along the same lines is the S3ClientFactory. There's only one S3Client instance.. So Factory is unnecessary here. Driving it home, the implementation is one line!!

public class S3ClientFactory : IS3ClientFactory
{
    public IS3Client Create(string key, string secret)
    {
        return new S3Client(key, secret);
    }
}

}


Before I do a PR, is the project maintainer open to these changes?

Ability to specify the buffer size when uploading files to S3

Thank you for Emkay.S3.

I would like to request the ability to specify the buffer size when uploading files to S3.

For example,

    <PublishFiles
      Key="$(Key)"
      Secret="$(Secret)"
      Bucket="$(Bucket)"
      SourceFiles="@(SourceFiles)"
      DestinationFolder="$(DestinationFolder)"
      BufferSizeKiloBytes="65536"
      TimeoutMilliseconds="$(TimeoutMilliseconds)" />

It seems the default buffer size is 8k but if it can be set to a higher value from MSBuild, uploading large files will be faster.

Thank you!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.