ablox / ablox Goto Github PK

AnyBlox - Data Containers in Go - version, snapshot, share, fork, backup, and restore your application data with a Docker like interface. Every container provides the complete view of the entire data set while only containing a subset of the data. Efficiently transfer only changed portions of data containers.

License: Apache License 2.0

Go 100.00%

ablox's Introduction

ablox

This system is currenly a work in progress. The server side is working and is easy to setup. It currenly only supports the NBD protocol.

To use:

Clone the repository
Run from prompt> go get github.com/urfave/cli
Run from prompt> go run server/server.go hostname port

hostname - the hostname or IP address you want to listen on. Hostname can be localhost if you only want it available locally
port - The port you want to listen on.

Put any files you want to attach in a subfolder "sample_disks". They are immediately avaiable for all users.

Right now, any users are allowed to accesss any files. File locking is not occuring on the server so you must make sure you only modify the disk from one system. Multiple read-only copies are fine.

Please file issues if you find any problems.

Background info

Installation

Go
QEMU

brew install qemu

Setup Docker and Ubuntu under VMWare

sudo su - apt-get update apt-get install open-ssl openssh-server open-vm-tools apt-transport-https ca-certificates nbd-client nbd-serverß

Follow instructions: https://docs.docker.com/engine/installation/linux/ubuntulinux/

// to be able to connect to an NBD server, you have to make sure the module is loaded. On starting the OS, run: sudo modprobe nbd

Tools

Wireshark
QEMU
Virtualbox
Intellij/PyCharm with golang plugin (Delve debugger currently has a bug itself!, so breakpoint will only stop 1 time! and it won't stop anymore even after you restart the debugging session)

ablox's People

Contributors

Stargazers

Watchers

Forkers

gitter-badger prtksxna

ablox's Issues

add support for port argument

add support for listen argument

Server crashed - no such file or directory

Client console:

Raymonds-MacBook-Air:share ray$ qemu-system-x86_64 -smp cpus=2,cores=2 -hda nbd://localhost:8000/UbuntuServer.vdi -m 2048
qemu-system-x86_64: -hda nbd://localhost:8000/UbuntuServer.vdi: Failed to read export length

Server console:

/private/var/folders/96/s9kywjm13sg242q45n5ylwt00000gn/T/rungo localhost 8000
GOROOT=/usr/local/go
GOPATH=/Users/ray/goProjects:/Users/ray/goProjects
/usr/local/go/bin/go build -o /private/var/folders/96/s9kywjm13sg242q45n5ylwt00000gn/T/rungo /Users/ray/git/jacob-nbd/server.go
Hello World, we have &{0xc820054070}
We have a new connection from: 127.0.0.1:52334
2 packets processed to get 20 bytes
Received from client (count  20) Data: '����IHAVEOPT��������' ([0 0 0 0 73 72 65 86 69 79 80 84 0 0 0 1 0 0 0 16])
Options are: 0
Reading instruction (count  36) Data: '����IHAVEOPT��������UbuntuServer.vdi' ([0 0 0 0 73 72 65 86 69 79 80 84 0 0 0 1 0 0 0 16 85 98 117 110 116 117 83 101 114 118 101 114 46 118 100 105])
Payload is: (count  16) Data: 'UbuntuServer.vdi' ([85 98 117 110 116 117 83 101 114 118 101 114 46 118 100 105])
command is: 1
have request to bind to: UbuntuServer.vdi
Opening file: /Users/ray/git/jacob-nbd/sample_disks/UbuntuServer.vdi
Error encountered: open /Users/ray/git/jacob-nbd/sample_disks/UbuntuServer.vdi: no such file or directory
panic: Nooooooooo


goroutine 5 [running]:
panic(0x130220, 0xc820010850)
    /usr/local/go/src/runtime/panic.go:481 +0x3e6
_/Users/ray/git/jacob-nbd/utils.ErrorCheck(0x645630, 0xc82000a7b0)
    /Users/ray/git/jacob-nbd/utils/utils.go:33 +0x18f
main.export_name(0xc820014380, 0x6454e0, 0xc820028038, 0x10, 0xc8200107b0, 0x10, 0x10, 0xc81fffce01)
    /Users/ray/git/jacob-nbd/server.go:62 +0x4a1
created by main.main
    /Users/ray/git/jacob-nbd/server.go:335 +0x12bf

Process finished with exit code 2

Change the buffer size for reading to be dynamic

server.go:91
buffer = make([]byte, 2048*1024) // set the buffer to 2mb

Should be able to support paged operation

S3Git integration (on Ubuntu img file) on NBD server.

Simple load test instructions

Can you please find a simple load test, add them to the readme, and commit the fix?

Even a load test for testing would be good. Something really simple to help us make sure the performance does not suffer.

Bug in error code for too many write connections to one file requested.

Add error checks to all functions in send_export_list

Find/Create QEMU img file that boots into Ubuntu

Remove the duplicated code that waits for network traffic

It is slowing things down and could be a source for bugs....

Add support for turning off autoflush

Add support for specifying a file

Add setup instructions for nbd server/client/qemu etc.

Demo set for James

@jsprinklex9 do you have a personal Mac, if so we can work on setting up the code base and demo on your system either sometime next week or weekend.

Limit writable shares to one concurrent per file

Port is not converted from int correctly

Clean up wait loop on initial client options

support listening on multiple ip addresses and ports

Right now, you have to specify one IP address when you start the server. Support listening on multiple IP addresses.

nbd-client listing not working from ubuntu

Ubuntu VM:
root@ubuntu:~# nbd-client -l 10.1.10.161 8000

OSX server.go:
/usr/local/go/bin/go run /Users/ray/goProjects/src/github.com/jacob-nbd/server/server.go 10.1.10.161 8000
Hello World, we have &{0xc82009e000}
We have a new connection from: 10.1.10.159:54886
Error encountered: EOF
Error encountered: EOF
Error encountered: EOF
Error encountered: EOF