Giter VIP home page Giter VIP logo

go-priam's Introduction

go-priam - Backup and restore Cassandra DB to AWS S3

GoDoc

go-priam provides a simple method for backing up cassandra keyspaces to Amazon AWS S3 bucket. It is written in go/golang and uses nodetool for performing backups and sstableloader for restoring from backups.

Install

If you have Go installed, then do:

go get github.com/alext29/go-priam

This should download, compile and install the app on your machine.

Running via command line

Before you begin there is a little bit of housekeeping to do.

  • Make sure password less ssh is setup between the machine you are running and all cassandra nodes. You can run from machine that is part of the cassandra cluster as well.
  • For restoring from backup, sstableloader requires access to all cassandra nodes via local subnet on port 9042. You may need to set up port-forwarding using ssh -fNT -L if needed.
  • Make sure incremental_backups is set to true in cassandra.yaml file (restart cassandra after changing config file). If this is not set incremental backup would not do anything.

Backup

Backup operation generates timestamps based on current time. The generated timestamp should be newer than last backup, else it would complain.

Full Backup:

go-priam [OPTIONS] -keyspace <KEYSPACE> backup

Incremental Backup:

go-priam [OPTIONS] -keyspace <KEYSPACE> -incremental backup

Incremental backup only uploads the incremental data with respect to the last backup. It it fails to find a previous backup it will do a full backup.

List backups:

go-priam [OPTIONS] -keyspace <KEYSPACE> history

Prints out timestamps of all existing backups, including incremental backups, in a tree form.

Restore

Restore operation will delete all existing data in given keyspace and restore to given timestamp. Any data added to the DB post backup would be lost.

Restoring to last backup:

go-priam [OPTIONS] -keyspace <KEYSPACE> restore

Restoring to specific timestamp:

go-priam [OPTIONS] -keyspace <KEYSPACE> -timestamp <TIMESTAMP> restore

When restoring to an incremental backup, all necessary files till the last full backup are downloaded and restored from. Timestamp is assumed to be monotonically increasing else the code would barf while take backup.

Configuration parameters

go-priam help gives a complete list of all command line parameters.

	-incremental            Switch to indicate incremental backup.
	-aws-access-key         AWS Access Key ID to access S3.
	-aws-base-path          Base path to copy/restore files from S3.
	-aws-bucket             S3 bucket name to store backups.
	-aws-region             Region of s3 account.
	-aws-secret-key         AWS Secret Access key to access S3.
	-cassandra-classpath    Directory where cassandra jarfiles are placed.
	-cassandra-conf         Directory where cassandra conf files are placed.
	-cqlsh-path             Path to cqlsh.
	-host                   IP address of any one of the cassandra nodes.
	-keyspace               Cassandra keyspace to backup.
	-nodetool-path          Path to nodetool on the cassandra host.
	-private-key            Path to private key used for password less ssh.
	-snapshot               Restore to this timestamp.
	-sstableloader          Path to sstableloader on cassandra hosts.
	-temp-dir               Temporary directory to download files to.
	-user                   Usename for password less ssh to cassandra host.

Configuration file

Configuration parameters may be specified in a yaml file as well. The default location for the configuration file is ${HOME_DIR}/.priam.conf or you may point it to any arbritary file by setting $PRIAM_CONF environment variable.

A sample config file priam.conf is provided for reference.

go-priam's People

Contributors

alext29 avatar

Stargazers

Hussain Sabir avatar  avatar HyeongGun Lee, Henry avatar Mario Deluna avatar kazzinak avatar Joel Donahue avatar Anthony Scalisi avatar Jack Angers avatar Francesco Pedrini avatar  avatar Bruno Andrade avatar Oleksandr Kukhar avatar

Watchers

 avatar Oleksandr Kukhar avatar kazzinak avatar  avatar

go-priam's Issues

Installation via go is not working

BLDM2558-MAC:~ ksachdeva$ go get github.com/alext29/go-priam
BLDM2558-MAC:~ ksachdeva$ go-priam
-bash: go-priam: command not found
BLDM2558-MAC:~ ksachdeva$

The go-priam is awesome!

@alext29 thank you for your job!

It's the most complete jawa-less C*-backup solution I've seen over long-time github crawling.
Is my pleasure to say "Thank you"!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.