Giter VIP home page Giter VIP logo

falcon-file-transfer-optimizer's Introduction

Falcon: Online File Transfers Optimization

The application can only correctly function on Linux-based operating systems due to several Linux-based functional dependencies. There are two configuration files for specifying the source, destinations, maximum allowed thread, and many other options.

Please use python3 and install necessary packages using the requirements.txt file, preferably in a virtual environment, to avoid package version conflicts. For GridFTP client optimization, please follow the link below.

GridFTP version: Please checkout Falcon-GridFTP directory

Usage

  1. Please create virtual environments on both source and destination server. For exmaple: run python3 -m venv <venv_dir>/falcon
  2. Activate virtual environment: run source <venv_dir>/falcon/bin/activate
  3. Install required python packages: pip3 install -r requirements.txt
  4. On the destination server, please edit config_receiver.py and run python3 receiver.py
  5. On the source server, please edit config_sender.py and run python3 sender.py

falcon-file-transfer-optimizer's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

falcon-file-transfer-optimizer's Issues

Specialized Case Assistance for Falcon Software

Hello, I am a researcher working for The University of Alabama in Huntsville Computer Science Department. I am attempting to use Falcon as part of a file transfer study.

The issue currently is that we don't have access to the server we are pulling from, so we cannot use the sender half of the code. The server we are pulling from is an AWS S3 bucket.

Would it be possible to modify or customize the code to pull from an S3 bucket without the sender code being installed by the server? Additionally, are there any offshoots or updated versions of Falcon that are more suited for this use case, and is there further documentation of Falcon that could assist us in modifications to the code for our use case?

nullpointer exception while using AdaptiveGridFTPClient

Hi,

I am currently trying to evaluate Faclon's ability on large-scale data transfer. While trying to use AdaptiveGridFTPClient as mentioned in the readme, I am unable to execute the runner. My configuration file is basically pointing to the local-hosted ftp server (as a test). My hostname keeps on coming as null while executing.

Can you please help me with this?

  • First, can you provide an example configuration on how I can use ftp there?
  • Secondly, hostname keeps on coming as null no matter what.
  • What do INPUT_DIR and OUTPUT_DIR represent in src/main/java/client/ConfigurationParams.java? Do I need to update the paths?
  • How do we do authenticated ftp with username and password in this?

I have attached a screenshot
image

(My background: I am a researcher at UAH)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.