The go-cloud-storage project came to make the use of Cloud Storage easier and more flexible, regardless of the provider, just use a simple and intuitive library interface. Below we list some implemented features:
- Simple bucket creation and deletion regardless of provider.
- Simple object insertion/update without worrying about conversions or pointers.
- Ease of obtaining the object with automatic conversion to the type you want.
- Object listing.
- Removal of object, multiple objects and prefixes.
Implemented providers:
- Google Cloud Storage
- AWS S3
Use go get.
go get github.com/GabrielHCataldo/go-cloud-storage
Then import the go-cloud-storage package into your own code.
import "github.com/GabrielHCataldo/go-cloud-storage/cstorage"Let's start by instantiating the go-cloud-storage Interface indicating which provider we will use, see:
NOTE: Each instance follows the standard of the provider's documentation, just follow it.
IMPORTANT: Always check the documentation in the structures and functions fields. For more details on the examples, visit All examples link.
- Google Storage
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"google.golang.org/api/option"
"time"
)
func main() {
cs, err := newInstanceGoogleStorage()
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
} else {
logger.Info("cloud storage instance created successfully!")
cs.SimpleDisconnect()
}
}
func newInstanceGoogleStorage() (cstorage.CStorage, error) {
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
return cstorage.NewGoogleStorage(ctx, option.WithCredentialsFile("firebase-admin-sdk.json"))
}Output:
[INFO 2024/01/12 08:17:17] main.go:18: cloud storage instance created successfully!
- AWS S3
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"github.com/aws/aws-sdk-go-v2/config"
"time"
)
func main() {
cs, err := newInstanceAwsS3Storage()
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
} else {
logger.Info("cloud storage instance created successfully!")
cs.SimpleDisconnect()
}
}
func newInstanceAwsS3Storage() (cstorage.CStorage, error) {
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
cfg, err := config.LoadDefaultConfig(ctx)
if helper.IsNotNil(err) {
return nil, err
}
return cstorage.NewAwsS3Storage(cfg), nil
}Output:
[INFO 2024/01/12 08:21:44] main.go:18: cloud storage instance created successfully!
With the instance created, we will continue with basic examples:
Creating a bucket is very simple, see the example below:
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"google.golang.org/api/option"
"os"
"time"
)
const bucketNameDefault = "go-cloud-storage-example"
func main() {
cs, err := newInstanceGoogleStorage() // or newInstanceAwsS3Storage
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
return
}
defer cs.SimpleDisconnect()
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
err = cs.CreateBucket(ctx, cstorage.CreateBucketInput{
// name of the bucket to be created (required)
Bucket: bucketNameDefault,
// project id where the bucket will be created (required only google storage)
ProjectId: os.Getenv("GOOGLE_STORAGE_PROJECT_ID"),
// bucket, if empty using default region
Location: "",
})
if helper.IsNotNil(err) {
logger.Error("error create bucket:", err)
} else {
logger.Info("bucket", bucketNameDefault, "created successfully!")
}
}Output:
[INFO 2024/01/12 09:18:05] main.go:51: bucket go-cloud-storage-example created successfully!
For more bucket examples, such as multiple creation, access the link.
Removing a bucket is very simple, just enter the name of the bucket you want to delete, see the example below:
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"google.golang.org/api/option"
"os"
"time"
)
const bucketNameDefault = "go-cloud-storage-example"
func main() {
cs, err := newInstanceGoogleStorage() // or newInstanceAwsS3Storage
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
return
}
defer cs.SimpleDisconnect()
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
err = cs.DeleteBucket(ctx, bucketNameDefault)
if helper.IsNotNil(err) {
logger.Error("error delete bucket:", err)
} else {
logger.Info("bucket", bucketNameDefault, "deleted successfully!")
}
}Output:
[INFO 2024/01/12 09:22:47] main.go:29: bucket go-cloud-storage-example deleted successfully!
For more bucket examples, such as multiple deletion, access the link.
You can put any type of content in the bucket, see the example:
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"google.golang.org/api/option"
"os"
"time"
)
type testStruct struct {
Name string `json:"name,omitempty"`
BirthDate time.Time `json:"birthDate,omitempty"`
Balance float64 `json:"balance"`
Emails []string `json:"emails,omitempty"`
}
func main() {
cs, err := newInstanceGoogleStorage() // or newInstanceAwsS3Storage
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
return
}
defer cs.SimpleDisconnect()
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
keyObject := "examples/json-example"
err = cs.PutObject(ctx, cstorage.PutObjectInput{
// name of the bucket where the object will be created (required)
Bucket: "go-cloud-storage",
// key of the object that will be created (required)
Key: keyObject,
// type of content of the object that will be created (required)
MimeType: cstorage.MimeTypeJson,
// content of the object that will be created (required)
Content: initTestStruct(),
})
if helper.IsNotNil(err) {
logger.Error("error put object on bucket:", err)
} else {
logger.Info("object", keyObject, "putted successfully!")
}
}
func initTestStruct() testStruct {
return testStruct{
Name: "Foo Bar",
BirthDate: time.Now(),
Balance: 203.12,
Emails: []string{"[email protected]", "[email protected]"},
}
}Output:
[INFO 2024/01/12 09:38:36] main.go:44: object examples/json-example putted successfully!
For more object examples, such as multiple creation, access link.
To obtain a single object, simply pass the name and key of the bucket, to parse the content to the desired type, simply use the ParseContent function of the returned object, see:
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"google.golang.org/api/option"
"os"
"time"
)
type testStruct struct {
Name string `json:"name,omitempty"`
BirthDate time.Time `json:"birthDate,omitempty"`
Balance float64 `json:"balance"`
Emails []string `json:"emails,omitempty"`
}
func main() {
cs, err := newInstanceGoogleStorage() // or newInstanceAwsS3Storage
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
return
}
defer cs.SimpleDisconnect()
keyObject := "examples/json-example"
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
obj, err := cs.GetObjectByKey(ctx, "go-cloud-storage", keyObject)
if helper.IsNotNil(err) {
logger.Error("error get object by key on bucket:", err)
} else {
var dest testStruct
err = obj.ParseContent(&dest)
if helper.IsNotNil(err) {
logger.Error("error parse object content:", err)
}
logger.Info("object", keyObject, "obtained successfully! obj:", obj, "content parsed:", dest)
}
}
func initTestStruct() testStruct {
return testStruct{
Name: "Foo Bar",
BirthDate: time.Now(),
Balance: 203.12,
Emails: []string{"[email protected]", "[email protected]"},
}
}Output:
[INFO 2024/01/12 09:49:29] main.go:40: object examples/json-example obtained successfully! obj: {"Key":"examples/json-example","Url":"https://storage.googleapis.com/go-cloud-storage/examples/json-example","MimeType":"application/json","Content":[123,34,110,97,109,101,34,58,34,70,111,111,32,66,97,114,34,44,34,98,105,114,116,104,68,97,116,101,34,58,34,50,48,50,52,45,48,49,45,49,50,84,48,57,58,51,56,58,51,54,46,50,51,50,51,52,50,45,48,51,58,48,48,34,44,34,98,97,108,97,110,99,101,34,58,50,48,51,46,49,50,44,34,101,109,97,105,108,115,34,58,91,34,102,111,111,98,97,114,64,103,109,97,105,108,46,99,111,109,34,44,34,102,111,111,98,97,114,50,64,103,109,97,105,108,46,99,111,109,34,93,125],"Size":132,"VersionId":"","LastModifiedAt":"2024-01-12T12:38:37Z"} content parsed: {"name":"Foo Bar","birthDate":"2024-01-12T09:38:36-03:00","balance":203.12,"emails":["[email protected]","[email protected]"]}
To list, simply enter which bucket you want, you can customize your searches using the opts parameter, see:
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"google.golang.org/api/option"
"os"
"time"
)
func main() {
cs, err := newInstanceGoogleStorage() // or newInstanceAwsS3Storage
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
return
}
defer cs.SimpleDisconnect()
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
opt := cstorage.NewOptsListObjects().SetPrefix("").SetDelimiter("")
objs, err := cs.ListObjects(ctx, "go-cloud-storage", opt)
if helper.IsNotNil(err) {
logger.Error("error list objects on bucket:", err)
} else {
logger.Info("list objects obtained successfully! objs:", objs)
}
}Output
[INFO 2024/01/12 09:59:19] main.go:28: list objects obtained successfully! objs: [{"Key":"examples/json-example","LastModifiedAt":"2024-01-12T12:38:37Z","Size":132,"Url":"https://storage.googleapis.com/go-cloud-storage/examples/json-example"}]
To remove a specific object, simply enter the bucket and key, see:
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"google.golang.org/api/option"
"os"
"time"
)
func main() {
cs, err := newInstanceGoogleStorage() // or newInstanceAwsS3Storage
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
return
}
defer cs.SimpleDisconnect()
keyObject := "examples/json-example"
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
err = cs.DeleteObject(ctx, cstorage.DeleteObjectInput{
Bucket: "go-cloud-storage",
Key: keyObject,
})
if helper.IsNotNil(err) {
logger.Error("error delete object by key on bucket:", err)
} else {
logger.Info("object", keyObject, "deleted successfully!")
}
}Output:
[INFO 2024/01/12 10:47:09] main.go:31: object examples/json-example deleted successfully!
For more examples of object deletion, visit link.
To remove all objects from a prefix, see:
package main
import (
"context"
"github.com/GabrielHCataldo/go-cloud-storage/cstorage"
"github.com/GabrielHCataldo/go-helper/helper"
"github.com/GabrielHCataldo/go-logger/logger"
"github.com/aws/aws-sdk-go-v2/config"
"google.golang.org/api/option"
"os"
"time"
)
func main() {
cs, err := newInstanceGoogleStorage() // or newInstanceAwsS3Storage
if helper.IsNotNil(err) {
logger.Error("error create new instance cloud storage:", err)
return
}
defer cs.SimpleDisconnect()
prefix := "examples/"
ctx, cancel := context.WithTimeout(context.TODO(), 5*time.Second)
defer cancel()
err = cs.DeleteObjectsByPrefix(ctx, cstorage.DeletePrefixInput{
Bucket: "go-cloud-storage",
Prefix: prefix,
})
if helper.IsNotNil(err) {
logger.Error("error delete objects by prefix on bucket:", err)
} else {
logger.Info("prefix", prefix, "deleted successfully!")
}
}Output:
[INFO 2024/01/12 10:52:58] main.go:32: prefix examples/ deleted successfully!
For more examples of prefix deletion, visit link.
Make a pull request, or if you find a bug, open it an Issues.
Distributed under MIT license, see the license file within the code for more details.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent