Giter VIP home page Giter VIP logo

integration's Introduction

Notice: Development on this repo is deprecated as we continue our v3 rearchitecture. Please see https://github.com/storj/storj for ongoing v3 development.

Storj integration/staging tree

This repository is for development and integration testing of all of the distributed Storj network services.

Development

To update a module in the container, change the package.json before building the image. It's best to use specific git hashes in the form storj/bridge#bd62050f1585278e6cbf18e620b6501f814359e3 as the version to bust the build cache for testing development branches.

Build

To build docker image, from within this repository:

docker build -t storj-integration .

Create a persistent container, ports for the bridge and farmers are exposed for testing uploading and downloading:

docker create -p 6382:6382 -p 3000:3000 -p 9000:9000 -p 9001:9001 -p 9002:9002 -p 9003:9003 -p 9004:9004 -p 9005:9005 -p 9006:9006 -p 9007:9007 -p 9008:9008 -p 9009:9009 -p 9010:9010 -p 9011:9011 -p 9012:9012 -p 9013:9013 -p 9014:9014 -p 9015:9015 -p 9016:9016 -t -i storj-integration bash

And to start and attach to the container:

docker start -a -i <hash_of_container>

Note: If you're running macos, you may need to increase the CPUs and memory available to docker.

Running the Sandbox

To run a sandbox test network:

First edit the file at ./config/storj-bridge/config.json to point to a email server within the container, so that you can test the mailer and registration.

And then run this script to start everything (16 farmers, 6 renters and 1 bridge):

./scripts/start_everything.sh

You can then use pm2 to look at logs and status for each service by name:

pm2 status
pm2 log
pm2 log bridge --lines 100
pm2 log renter-6
pm2 log farmer-4
pm2 stop farmer-4

And check that mongo has the data that is expected:

mongo
use storj-sandbox

Modify files for testing purposes:

vim ./node_modules/storj-bridge/lib/constants.js

To stop all of the services, and exit the container:

./scripts/stop_everything.sh
exit

Testing File Transfers

Using the CLI included with https://github.com/Storj/libstorj you can register and transfer files for development and testing.

STORJ_BRIDGE=http://localhost:6382 ./src/storj register

And then check your email and activate the account, and then add buckets and files:

STORJ_BRIDGE=http://localhost:6382 ./src/storj add-bucket
STORJ_BRIDGE=http://localhost:6382 ./src/storj upload-file <bucket_hash> <filename>
STORJ_BRIDGE=http://localhost:6382 ./src/storj download-file <bucket_hash> <file_hash>

Testing Bridge GUI

Update stripe key for billing config at ./config/storj-billing/config within the container to have a test key:

export STRIPE_KEY="sk_test_*******************"

After cloning https://github.com/storj/bridge-gui-vue externally:

cd bridge-gui-vue
npm install

And add a .env file to the base directory of bridge-gui-vue with:

STRIPE_PUBLISHABLE_KEY="pk_test_*******************"

And start the gui development server:

npm run dev

Note: The default settings should use port 6382 for bridge and 3000 for billing and connect by default.

Troubleshooting

  • One benefit of having an entire network in a container is that code is shared for all services, change it once and restart the services to get insight into problems when developing.
  • Make sure that all of the services are running, take a look at ./script/start_everything.sh for what should be running. In particular make sure that rabbitmq-server is running.
  • As an entire network is being run, there are considerable resources that are needed, make sure that docker has enough CPUs and memory available to it. Otherwise you can reduce the number of farmers and renters that are started by editing ./script/start_everything.sh, this will have an impact on the ability to upload files during testing.

integration's People

Contributors

braydonf avatar bryanchriswhite avatar bookchin avatar jtolio avatar kaloyan-raev avatar richardlitt avatar

Stargazers

Alexander Milovanov avatar Andrei avatar Nagisa avatar Tome Boshevski avatar

Watchers

Sean Harvey avatar Ethan Adams avatar Egon Elbre avatar  avatar  avatar James Cloos avatar Philip Hutchins avatar Dan Willoughby avatar Maximillian von Briesen avatar Ivan Fraixedes avatar Erik van Velzen avatar Tome Boshevski avatar dylan avatar Adam S. avatar Michał Niewrzał avatar Malcolm Bouzi avatar Jennifer Li Johnson avatar John Gleeson avatar  avatar karl avatar Stefan Benten avatar Yehor Butko avatar Kevin J Baird avatar Artur M. Wolff avatar  avatar

integration's Issues

Stucked at 'preparing file'

When running docker straight from a git clone, uploading files is impossible.
After a long time of 'Preparing file', the process exists with 'Unable to receive storage offer'.

storj-billing burns the CPU

I noticed that if I "start everything" and leave the container idle, it consumes ~30% of my laptop CPU power.

As far as I can see this is caused by short-living storj-billing processes. When the storj-billing process finishes, another one is immediately launched. It would be nice to have some delays between the runs.

"hdIndex" is expected with "hdKey"

I am in the process of setting up a testnet with 15 farmers, and I came across this integration repo. In trying to setup the complex, I get the following error:

AssertionError: "hdIndex" is expected with "hdKey"

I looked at the code and it appears I need to generate HD keys for the renters, but I cannot find any online references on how to generate the complex private key. Can you shed some light on this for me?

Thanks,
Steve

storjshare-start

I see this version uses storjshare-start in the package.json;
Where does this comes from? I haven't managed to find this package somewhere on the Storj git.

storj mailer isn't sending an email

In ./config/storj-bridge/config.json, you have the option to edit this json to something like this

"mailer": {
"host": "smtp.gmail.com",
"port": 465,
"secure": true,
"auth": {
"user": "[email protected]",
"pass": "the password"
},

when I check storjtestuser email, nothing was sent. I can fix this later- when I get some downtime. Just wanted to create an issue before I forget.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.