Giter VIP home page Giter VIP logo

basic-ftp's Introduction

Basic FTP

npm version npm downloads Node.js CI

This is an FTP client library for Node.js. It supports FTPS over TLS, Passive Mode over IPv6, has a Promise-based API, and offers methods to operate on whole directories. Active Mode is not supported.

Advisory

Prefer alternative transfer protocols like HTTPS or SFTP (SSH). FTP is a an old protocol with some reliability issues. Use this library when you have no choice and need to use FTP. Try to use FTPS (FTP over TLS) whenever possible, FTP alone does not provide any security.

Dependencies

Node 10.0 or later is the only dependency.

Installation

npm install basic-ftp

Usage

The first example will connect to an FTP server using TLS (FTPS), get a directory listing, upload a file and download it as a copy. Note that the FTP protocol doesn't allow multiple requests running in parallel.

const { Client } = require("basic-ftp") 
// ESM: import { Client } from "basic-ftp"

example()

async function example() {
    const client = new Client()
    client.ftp.verbose = true
    try {
        await client.access({
            host: "myftpserver.com",
            user: "very",
            password: "password",
            secure: true
        })
        console.log(await client.list())
        await client.uploadFrom("README.md", "README_FTP.md")
        await client.downloadTo("README_COPY.md", "README_FTP.md")
    }
    catch(err) {
        console.log(err)
    }
    client.close()
}

The next example deals with directories and their content. First, we make sure a remote path exists, creating all directories as necessary. Then, we make sure it's empty and upload the contents of a local directory.

await client.ensureDir("my/remote/directory")
await client.clearWorkingDir()
await client.uploadFromDir("my/local/directory")

If you encounter a problem, it may help to log out all communication with the FTP server.

client.ftp.verbose = true

Client API

new Client(timeout = 30000)

Create a client instance. Configure it with a timeout in milliseconds that will be used for any connection made. Use 0 to disable timeouts, default is 30 seconds.

close()

Close the client and any open connection. The client can’t be used anymore after calling this method, you'll have to reconnect with access to continue any work. A client is also closed automatically if any timeout or connection error occurs. See the section on Error Handling below.

closed

True if the client is not connected to a server. You can reconnect with access.

access(options): Promise<FTPResponse>

Get access to an FTP server. This method will connect to a server, optionally secure the connection with TLS, login a user and apply some default settings (TYPE I, STRU F, PBSZ 0, PROT P). It returns the response of the initial connect command. This is an instance method and thus can be called multiple times during the lifecycle of a Client instance. Whenever you do, the client is reset with a new connection. This also implies that you can reopen a Client instance that has been closed due to an error when reconnecting with this method. The available options are:

  • host (string) Server host, default: localhost
  • port (number) Server port, default: 21
  • user (string) Username, default: anonymous
  • password (string) Password, default: guest
  • secure (boolean | "implicit") Explicit FTPS over TLS, default: false. Use "implicit" if you need support for legacy implicit FTPS.
  • secureOptions Options for TLS, same as for tls.connect() in Node.js.

features(): Promise<Map<string, string>>

Get a description of supported features. This will return a Map where keys correspond to FTP commands and values contain further details. If the FTP server doesn't support this request you'll still get an empty Map instead of an error response.

send(command): Promise<FTPResponse>

Send an FTP command and return the first response.

sendIgnoringError(command): Promise<FTPResponse>

Send an FTP command, return the first response, and ignore an FTP error response. Any other error or timeout will still reject the Promise.

cd(path): Promise<FTPResponse>

Change the current working directory.

pwd(): Promise<string>

Get the path of the current working directory.

list([path]): Promise<FileInfo[]>

List files and directories in the current working directory, or at path if specified. Currently, this library only supports MLSD, Unix and DOS directory listings. See FileInfo for more details.

lastMod(path): Promise<Date>

Get the last modification time of a file. This command might not be supported by your FTP server and throw an exception.

size(path): Promise<number>

Get the size of a file in bytes.

rename(path, newPath): Promise<FTPResponse>

Rename a file. Depending on the server you may also use this to move a file to another directory by providing full paths.

remove(path): Promise<FTPResponse>

Remove a file.

uploadFrom(readableStream | localPath, remotePath, [options]): Promise<FTPResponse>

Upload data from a readable stream or a local file to a remote file. If such a file already exists it will be overwritten. If a file is being uploaded, additional options offer localStart and localEndInclusive to only upload parts of it.

appendFrom(readableStream | localPath, remotePath, [options]): Promise<FTPResponse>

Upload data from a readable stream or a local file by appending it to an existing file. If the file doesn't exist the FTP server should create it. If a file is being uploaded, additional options offer localStart and localEndInclusive to only upload parts of it. For example: To resume a failed upload, request the size of the remote, partially uploaded file using size() and use it as localStart.

downloadTo(writableStream | localPath, remotePath, startAt = 0): Promise<FTPResponse>

Download a remote file and pipe its data to a writable stream or to a local file. You can optionally define at which position of the remote file you'd like to start downloading. If the destination you provide is a file, the offset will be applied to it as well. For example: To resume a failed download, request the size of the local, partially downloaded file and use that as startAt.


ensureDir(remoteDirPath): Promise<void>

Make sure that the given remoteDirPath exists on the server, creating all directories as necessary. The working directory is at remoteDirPath after calling this method.

clearWorkingDir(): Promise<void>

Remove all files and directories from the working directory.

removeDir(remoteDirPath): Promise<void>

Remove all files and directories from a given directory, including the directory itself. The working directory stays the same unless it is part of the deleted directories.

uploadFromDir(localDirPath, [remoteDirPath]): Promise<void>

Upload the contents of a local directory to the current remote working directory. This will overwrite existing files with the same names and reuse existing directories. Unrelated files and directories will remain untouched. You can optionally provide a remoteDirPath to put the contents inside any remote directory which will be created if necessary including all intermediate directories. The working directory stays the same after calling this method.

downloadToDir(localDirPath, [remoteDirPath]): Promise<void>

Download all files and directories of the current working directory to a given local directory. You can optionally set a specific remote directory. The working directory stays the same after calling this method.


trackProgress(handler)

Report any transfer progress using the given handler function. See the next section for more details.

Transfer Progress

Set a callback function with client.trackProgress to track the progress of any transfer. Transfers are uploads, downloads or directory listings. To disable progress reporting, call trackProgress without a handler.

// Log progress for any transfer from now on.
client.trackProgress(info => {
    console.log("File", info.name)
    console.log("Type", info.type)
    console.log("Transferred", info.bytes)
    console.log("Transferred Overall", info.bytesOverall)
})

// Transfer some data
await client.uploadFrom(someStream, "test.txt")
await client.uploadFrom("somefile.txt", "test2.txt")

// Set a new callback function which also resets the overall counter
client.trackProgress(info => console.log(info.bytesOverall))
await client.downloadToDir("local/path", "remote/path")

// Stop logging
client.trackProgress()

For each transfer, the callback function will receive the filename, transfer type (upload, download or list) and number of bytes transferred. The function will be called at a regular interval during a transfer.

There is also a counter for all bytes transferred since the last time trackProgress was called. This is useful when downloading a directory with multiple files where you want to show the total bytes downloaded so far.

Error Handling

Any error reported by the FTP server will be thrown as FTPError. The connection to the FTP server stays intact and you can continue to use your Client instance.

This is different with a timeout or connection error: In addition to an Error being thrown, any connection to the FTP server will be closed. You’ll have to reconnect with client.access(), if you want to continue any work.

Logging

Using client.ftp.verbose = true will log debug-level information to the console. You can use your own logging library by overriding client.ftp.log. This method is called regardless of what client.ftp.verbose is set to. For example:

myClient.ftp.log = myLogger.debug

Static Types

In addition to unit tests and linting, the source code is written in Typescript using rigorous compiler settings like strict and noImplicitAny. When building the project, the source is transpiled to Javascript and type declaration files. This makes the library useable for both Javascript and Typescript projects.

Extending the library

Client

get/set client.parseList

Provide a function to parse directory listing data. This library supports MLSD, Unix and DOS formats. Parsing these list responses is one of the more challenging parts of FTP because there is no standard that all servers adhere to. The signature of the function is (rawList: string) => FileInfo[].

FTPContext

The Client API described so far is implemented using an FTPContext. An FTPContext provides the foundation to write an FTP client. It holds the socket connections and provides an API to handle responses and events in a simplified way. Through client.ftp you get access to this context.

get/set verbose

Set the verbosity level to optionally log out all communication between the client and the server.

get/set encoding

Set the encoding applied to all incoming and outgoing messages of the control connection. This encoding is also used when parsing a list response from a data connection. See https://nodejs.org/api/buffer.html#buffer_buffers_and_character_encodings for what encodings are supported by Node.js. Default is utf8 because most modern servers support it, some of them without mentioning it when requesting features.

Acknowledgment

This library uses parts of the directory listing parsers written by The Apache Software Foundation. They've been made available under the Apache 2.0 license. See the included notice and headers in the respective files containing the original copyright texts and a description of changes.

basic-ftp's People

Contributors

alandoherty avatar ansonyeung avatar broofa avatar cjsewell avatar codingcarlos avatar conlanpatrek avatar dependabot[bot] avatar fabianmeul avatar fossprime avatar inithink avatar kerimg avatar lgpseu avatar martijnimhoff avatar mike-odom avatar patrickjuchli avatar sparebytes avatar trs avatar xnerhu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

basic-ftp's Issues

Ad hoc error objects are problematic

It appears you're constructing ad-hoc error objects - vanilla objects with an error property - and then throwing those in handler functions. (E.g. building one here, then rejecting here and here I believe?)

This is problematic for a variety of reasons (no stacktrace info, breaks 3rd party logging code like bugtrag, etc.)

I'd suggest creating bonafide Error instances, and doing instanceof Error checks, rather than looking for the error property.

(BTW, thanks for putting this module together! I've been struggling to get the node-ftp module to work properly, and it no longer appears to be maintained. I'm hoping to use this module as a replacement. I like what I see, but 'still working through a few rough spots. I hope you don't mind me throwing a few issues you at you.)

This socket is closed

Hello.

I'm having several users reporting an error at some point in my app when downloading the updates by FTP.

Updates consist of many video files, and thus can take a long time to download. I'm trying to pinpoint what's causing it exactly but thi sis very hard to diagnose and I realize I'm not being very helpful by saying that : it works very well for me (able to download via FTP for hours) and for others it fails with an error like that in the middle of a download.

The FTP server we connect to is a standard Pure-FTPd on an Ubuntu 16.04

If I can help further, I'd love to as this has been bugging me for a bit.

parseControlResponse() will error on multibyte UTF8 characters

In _onControlSocketData(), data chunks are converted to Strings before concatenation, while prepending _partialResponse on the next chunk. This will lead to errors and/or exceptions if a data boundary falls on a multi-byte unicode character, however.

Instead, data should be processed as a byte stream, only converting to String once a complete line has been detected. See newline-based stream parser example. Bonus: the nifty unshift() trick for pushing the partial data back onto the stream.)

Question: sending another command why the previous one hasn't resolved yet

I am working on an ftp client using a gui (see: https://github.com/warpdesign/react-ftp) and I need to be able to send ftp commands (for eg. to change/display a directory) while another one (for eg. a transfer) is already in progress.

Right now, sending a command while another is already in progress (for eg. calling client.list before a previous call to client.download resolves) throws an error. Which is logical since FTP doesn't allow multiple transfers at the same time.

What's the best way to use basic-ftp for my needs: do I need to instantiate a new client for each transfer? Or is there another way?

Question about download/upload API

I am porting my ftp client so that it's now using basic-ftp instead of node-ftp.

Everything is working ok so far but it seems the download API differs in that node-ftp has a get API that's basically:

client.get(remotePath, callback(err, readableStream))

So I'm getting a readableStream that I can pipe through a writetableStream I create myself.

The good thing about that is that I can even download something from an FTP server and directly pass it through to the put method of another FTP client with the put API:

client.get(remotePath, (err, readableStream) => {
  if (!err) {
    client2.put(readableStream, remotePath);
   }
});

With basic-ftp, the download API needs a writableStream instead of a readableStream so this isn't possible anymore.

I briefly looked at the source code but couldn't find an easy to get a readableStream when downloading something. Is it possible without much work?

Control Socket Timeout In Electron

Describe the bug
Control Socket Times out before FTP completes. When uploading a large file, I'm getting the error { error: { ftpSocket: 'control', info: 'socket timeout' } }. I'm running this in a electron.js app. It seems to run fine in a plain node.js app. Any thoughts on why the control might timeout in an electron app, but not a command line app?

Example code
If I need to can can provide an example electron app.

Console output

  Connected to ***
  < 220 ProFTPD 1.3.5 Server (FTP-Server) [::ffff:***]
  Login security: No encryption
  > USER ***
  < 331 Password required for ***
  > PASS ###
  < 230 User *** logged in
  > TYPE I
  < 200 Type set to I
  > STRU F
  < 200 Structure set to F
  Trying to find optimal transfer strategy...
  > EPSV
  < 229 Entering Extended Passive Mode (|||50575|)
  Optimal transfer strategy found.
  > STOR largefile.mp4
  < 150 Opening BINARY mode data connection for largefile.mp4
  Uploading to ***:50575 (No encryption)
  {info: "socket timeout", ftpSocket: "control"}
 { error: { ftpSocket: 'control', info: 'socket timeout' } }
 { error: { ftpSocket: 'control', info: 'socket timeout' } }

Which version of Node.js are you using?
Electron: 3.0.10
Node 10.2.0

Additional context

How does timeout works?

Hello :)

I'm running basic-ftp 2.12.2

I'm downloading some large files over ftp with it, and so far it's worked great for my needs. I have however a question about the timeout argument.

I've received one or two complaints from users with poor connections, who regularly got timeouts when downloading from FTP. I noticed the timeout is set to 0 by default, so I tried to set it like this

const ftp = new FTP.Client(20000);

However, I still got a transfer stuck when I tried to reproduce the issue (by killing my ftp server during the transfer). It just gets stuck and never recovers from this kind of trouble, nor throws any error.

Am I doing something wrong? Thanks in advance for your help.

`parseControlResponse` does not handle tokens without a message correctly

Describe the bug
The parseControlResponse expects a token as either nnn or nnn-. This falls apart when the server sends just the code without a message nnn since it is trimmed and would become nnn.

The connect message is never resolved since the handle callback is never called.

Example code

async function doAThing() {
  await ftpClient.connect();
}

Console output

    < 220-
    220-This is an example message

    < 220-
    < 220-You better not use this ftp server for evil
    220-
    220-OR ELSE!
    220-
    220 // <--- NO MESSAGE

// Hangs

Which version of Node.js are you using?

$ node -v
v11.9.0

Suggestion: Pre compile to javascript

Please distribute this module as javascript aswell as typescript. I simply HATE dealing with typescript in every way.

Adding a folder with code already in javascript within src would be really helpful

[poor question] What would you like to do in this case

In my project, I connect to 100 FTP servers in 1 minute and receive files.
Below is a sample of the code.

const ftp = require("basic-ftp");

async function exec(other, timeout) {
    const client = new ftp.Client(timeout);
    /*
        connect
        list
        download
        etc..
    */
    client.close();
}

exec() is called periodically and has a new ftp.Client. Is this recommended?
Or do you recommend creating an object pool or a singleton to use only one object per FTP server?

EISCONN error after failed authentication (Error 530)

After a failed login attempt when calling access (wrong credentials for example), basic-ftp will throw an FTPError 530.

Any subsequent attempt to login with the correct credentials using access will throw an EISCONN Error.

So it appears that a new Client has to be created in order to login after an error 530 has been thrown?
Is it right?

ftp.list returns wrong name if filename starts with a space

Describe the bug
ftp.list trims spaces at the beginning of files. If a file is named " test", basic-ftp will return "test" instead so not only the listing is broken, but any action on this file will fail.

FileZilla has no such problems.

Which version of Node.js are you using?
e.g. Node 11.9.0

Additional context
This is a Unix server with files that contain spaces at the beginning.

I think the bug is here:

const name = groups[21].trim();

Replacing the call to trim with this should fix it (trimEnd cannot be used either since filenames can also have spaces at the end):

const name = groups[21].replace(/[\t\n\r]*$/, '');

Can't resolve 'fs' 'net' 'path' 'tls'

Hello,

i get some errors when i try to use the Plugin..
ERROR in ./node_modules/basic-ftp/lib/ftp.js Module not found: Error: Can't resolve 'fs' in '..\node_modules\basic-ftp\lib' Module not found: Error: Can't resolve 'fs' in '..\src\app' ERROR in ./node_modules/basic-ftp/lib/ftp.js Module not found: Error: Can't resolve 'net' in '..\node_modules\basic-ftp\lib' ERROR in ./node_modules/basic-ftp/lib/FtpContext.js Module not found: Error: Can't resolve 'net' in '..\node_modules\basic-ftp\lib' ERROR in ./node_modules/basic-ftp/lib/ftp.js Module not found: Error: Can't resolve 'path' in '..\node_modules\basic-ftp\lib' ERROR in ./node_modules/basic-ftp/lib/ftp.js Module not found: Error: Can't resolve 'tls' in '..\node_modules\basic-ftp\lib'

This is what I'm doing at the moment:
const ftp = require("basic-ftp") const fs = require("fs")

What I'm doing wrong?

Thank you

cli.upload transfers file but does not resolve

Hello there,

I have a webapp which needs to upload some excel files to a remote plain FTP server.

Everything seems to work fine, except for the fact that using client.upload as a promise transfers the file correctly (which I can then see on the server) but does not resolve after that and just hangs there until the timeout time is eventually reached.

This is the code I'm using:

// cli, stream_wb_translate, remoteFilename are defined in a higher scope
cli = new FTP.Client(30000);
cli.ftp.verbose = true;
return cli.access({
  host: ...,
  user: ...,
  password: ...,
  secure: false
})
.then((response) => {
  return cli.cd(path)
  .then((response) => {
    return cli.upload(stream_wb_translate, remoteFilename); // this hangs until timeout but the file is transferred
  })
  .then((response) => {
    // never reached
    return yetSomeOtherPromiseStuff();
  })
  .catch((e) => {
    throw e;
  });
})
.then(() => {
  // all good, never reached
})
.catch((message) => {
  // always reached
  return reject({
    msg: 'Send failed', 
    detail: errorUtils.parseFromPromise(message)
  });
})
.finally(() => {
  if (cli) {
    cli.close();
  }
});

Most probably I'm doing something wrong, but despite step-debugging I can't see it.

Thanks in advance for your attention. Have a nice day! ☺

Feature request for last modified time of file

Request for support for the MDTM command in extension of FTP command

https://tools.ietf.org/html/rfc3659#section-3.1

I currently use this code.

    client.send("MDTM " + src).then(obj => {
        let raw = +obj.message.split(" ")[1];
        let final = new Date();
        console.log(Math.floor(raw / 10000000000))
        final.setUTCFullYear(Math.floor(raw / 10000000000));
        raw = raw % 10000000000;
        console.log(Math.floor(raw / 100000000))
        final.setUTCMonth(Math.floor(raw / 100000000) - 1);
        raw = raw % 100000000;
        console.log(Math.floor(raw / 1000000))
        final.setUTCDate(Math.floor(raw / 1000000));
        raw = raw % 1000000;
        console.log(Math.floor(raw / 10000))
        final.setUTCHours(Math.floor(raw / 10000));
        raw = raw % 10000;
        console.log(Math.floor(raw / 100))
        final.setUTCMinutes(Math.floor(raw / 100));
        raw = raw % 100;
        console.log(Math.floor(raw / 1))
        final.setUTCSeconds(Math.floor(raw / 1));
        raw = raw % 1;
        final.setUTCMilliseconds(raw * 1000);
        console.log(final);
        return Promise.resolve(final.valueOf());
    })

Can't connect to same server subsequently - connection stuck, no timeout

So for some reason I am somehow "stuck" with my first connection.
After starting up my nodejs server "fresh" and calling my function which connects to a remote FTP, downloads some files and ftpClient.close()s before returning, all is fine.
All the other code after downloading the files is executed and the parent function returns properly - all good!
After waiting a couple of seconds, I make the exactly same call again, the FTP should connection again, download the same files ... you get the idea.
BUT: my console.log() directly after the await ftpClient.access() is never fired. The FTPClient is just stuck there forever, no timeout no nothing, just stuck. I have to restart my whole server all over again to be able to call my function again - again only once, the second time the same thing happens all over again.

Version: 2.14.0, nodeJS Version: v8.11.4

Unit test throws unhandled exception

The unit test sends the right command for the topic Download directory listing throws an unhandled exception. The exception is not relevant at this point, suppress it.

Upload creates empty files - No Errors

All my file are created but empty. Do you have an idea why this happens?

Code:

        const client = new ftp.Client();
        client.ftp.verbose = true;
        try {
            await client.connect(config.host, config.port);
            if(config.tls) {
              await client.useTLS();
            }
            await client.login(config.username, config.password);
            await client.useDefaultSettings();
            await client.uploadDir(config.localRoot);
        }
        catch(err) {
            console.log(err);
        }
        client.close();

Log:

Data socket uses TLSv1.2
> STOR webapp-fastboot-8e75705c056d7ebc3694d2513c93c176.js
< 125 Data connection already open; Transfer starting.
< 226 Transfer complete.
> CDUP
< 250 CDUP command successful.
> PASV
< 227 Entering Passive Mode (51,4,149,110,39,23).

Features:

  'LANG' => 'EN*',
  'UTF8' => '',
  'AUTH' => 'TLS;TLS-C;SSL;TLS-P;',
  'PBSZ' => '',
  'PROT' => 'C;P;',
  'CCC' => '',
  'HOST' => '',
  'SIZE' => '',
  'MDTM' => '',
  'REST' => 'STREAM' 

Notes:

  • Remote Server is Azure AppService (Windows)
  • Disc has enough space (Works with FileZilla)
  • Tried ASCII and BINARY mode, no change
  • Tried UTF-8 and latin1 encoding

upload without PASV (passive mode)

When attempting to use this module to upload some files to a FTP server I don't control (if I did, we'd be using sFTP), I get the following error:

(node:17303) UnhandledPromiseRejectionWarning: Can't parse PASV response

Upon reading the code, it doesn't appear that this library supports active mode at all. Unfortunately for me, the remote server doesn't support PASV:

Connected to 10.N.N.N.
220 Microsoft FTP Service
Name (10.N.N.N:matt): *********
331 Password required
Password:
230 User logged in.
Remote system type is Windows_NT.
ftp> PASV
?Invalid command

If you don't intend to support active mode, I completely understand. I'm just pointing out that it will constrain the use of this module.

Missing comma

Just a missing comma in the readme intro example after:

user: "very"

Will close connections because of error: { ftpSocket: 'control' }

I tested this module and basically it works but now I get an error and I don't know why...

Describe the bug
I'm trying to upload a file from a FTP server and I did this test with 2 different FTP server and I get the same error.

Example code
��

async function example() {
    const client = new ftp.Client();

    client.ftp.verbose = true;

    try {
        // ftp connect
        await client.access(options);

        // print current remote dir
        const string = await client.pwd();
        log.info(string);

        const fileName = 'test.txt';

        fs.stat(fileName, async function (err, stat) {
            try {
                if (err == null) {
                    log.info('--> File exists: ' + fileName);

                    const result = await client.upload(fs.createReadStream(fileName), fileName);

                    if (result === 226) {
                        log.info('--> Stored file: ' + fileName);

                        // remove file with a callback:
                        /*
                        fs_extra.remove(filePath + fileName, err => {
                            if (err) return console.error(err);
                            log.info('--> Deleted file: ' + filePath + fileName);
                        });
                        */
                    }
                } else if (err.code === 'ENOENT') {
                    log.info('--> Not found file: ' + fileName);
                } else {
                    log.info('Some other error: ', err.code);
                }
            } catch (e) {
                log.info("ERROR 1");
                log.info(e);
            }
        });
    } catch (err) {
        log.info(err);
    }

    client.close();
}

Console output
Connected to 192.168.0.116:211
< 220 ProFTPD 1.3.5b Server (Debian) [::ffff:192.168.0.116]
Login security: No encryption

USER pi
< 331 Password required for pi
PASS ###
< 230 User pi logged in
TYPE I
< 200 Type set to I
STRU F
< 200 Structure set to F
PWD
< 257 "/home/pi" is the current directory
Closing connections.
2018-11-14 11:43:04 INFO /home/pi
Trying to find optimal transfer strategy...
EPSV
Will close connections because of error:
{ ftpSocket: 'control' }
Closing connections.

Which version of Node.js are you using?
v8.11.3

Additional context
I'm running my nodeJS application in a Raspberry Pi 3B+ but yesterday It worked.

Support filenames with leading whitespace

This library doesn't work with filenames that have leading or trailing whitespace. This is one of these typical FTP protocol shortcomings and many clients don't handle these correctly – at least not in all cases.

The way to address this is to always use the full, absolute path. For example, when getting the size of such a file, using SIZE[SPACE][SPACE]test.txt won't work. Quotes also won't work. You have to use SIZE /absolute/path/to/ test.txt or whatever the path is.

The problem with that is that people usually use relative paths. So we'd have to always get the path of the current working directory first.

list returning wrong data

Describe the bug
I'm trying to get a list of files from my ftp server. The ftp server is running on a EC2 instance Ubuntu 18.04.1 with vsftpd version 3.0.3 (ami-0ac019f4fcb7cb7e6). My code runs in a Ubuntu 18.04.2 machine. The list seems to be wrong, I used other ftp server to test (https://dlptest.com/ftp-test/) and worked fine but I really don't know why the ftp I just created is getting wrong info.

Example code
const list = await client.list();

Console output

[  
   {  
      "name":"1001         4096 Feb 25 19:03 .",
      "type":1,
      "size":1001,
      "permissions":{  
         "user":0,
         "group":0,
         "world":0
      },
      "hardLinkCount":0,
      "link":"",
      "group":"",
      "user":"",
      "date":"drwxr-xr-x 2"
   },
   {  
      "name":"65534        4096 Feb 25 18:55 ..",
      "type":1,
      "size":65534,
      "permissions":{  
         "user":0,
         "group":0,
         "world":0
      },
      "hardLinkCount":0,
      "link":"",
      "group":"",
      "user":"",
      "date":"dr-xr-xr-x 3"
   },
   {  
      "name":"1001          487 Feb 25 19:03 package.json",
      "type":1,
      "size":1001,
      "permissions":{  
         "user":0,
         "group":0,
         "world":0
      },
      "hardLinkCount":0,
      "link":"",
      "group":"",
      "user":"",
      "date":"-rw------- 1"
   }
]

Which version of Node.js are you using?
Node 8.10.0

Error: None of the available transfer strategies work.

I'm trying to download a file from my FTP server. sometimes it does work , sometimes it doesn't

           const filename = 'myfile.jpg'
           const createStream = fs.createWriteStream(path + `${filename}`, 'utf8')
            let file = await client.download(createStream, filename, startAt = 0)

Linux file permissions

Is there a way to set default linux file permissions, or take ones from the users umask?

Currently everything is created as 744, however a way to specify this on a per file or per folder basis would be very helpful.

Node.js 6.11 support

Doesn't work with Google Cloud Functions

Detailed stack trace: /user_code/node_modules/basic-ftp/lib/ftp.js:107
    async useTLS(options = {}, command = "AUTH TLS") {

Upload only the files inside a directory

So I have a folder "downloads" in the directory where my project is running and I need to upload all the files and sub folders inside of it to a FTP server.

But when I do await client.uploadDir('download/'); it upload the folder to the server and I want only the files and sub folders inside of it.

Is it possible to do that?

Thanks in advance.

Question: events?

When basic-ftp is connected and I'm not sending any commands, how do I detect events like:

  • timeout
  • connection with the server lost/closed
    etc..

I saw no events-related methods in the API.

Add a way to configure the FtpContext's encoding

FtpContext's constructor has an encoding parameter which allows to set the encoding used when decoding results such as the LIST command result but there's no way to change its default value since the Client constructor simply calls:

class Client {
    constructor(timeout = 30000) {
        this.ftp = new FTPContext(timeout)
    }
}

I suggest adding a new encoding to the Client's constructor that defaults to "utf8":

class Client {
    constructor(timeout = 30000, encoding="utf8") {
        this.ftp = new FTPContext(timeout, encoding)
    }
}

Tickle server during data transfers

To make timeout settings more useful, we should tickle the server using the control connection during data transfers. Other clients do this by sending NOOP, STAT or PWD at regular intervals.

"Closing connections" issue

Describe the bug
I suspect a bug related to closing client connection. After successful file upload and close the connection I see infinitely repeated message Closing connections. in the console. I tried on two different FTP servers and it looks the same.

Example code

const ftp = require("basic-ftp");
const client = new ftp.Client();
client.ftp.verbose = true;
client.access({
    host: "ftp.drivehq.com",
    user: "davvvid1",
    password: "***",
    secure: true
}).then(() => {
    const Readable = require('stream').Readable;
    const stream = new Readable;
    stream.push('File content');
    stream.push(null);
    return client.upload(stream, 'file-name.txt');
}).then(() => {
    console.log('UPLOADED SUCCESSFULLY');
    return client.close();
}).catch((err) => {
    console.log('ERROR');
    console.log(err);
});

setInterval(() => {
    console.log('keep node active...');
}, 5000);

Console output

Connected to xx.xx.xx.xx
< 220 FTP HOSTING WELCOME MESSAGE;
> AUTH TLS
< 234 AUTH Command OK. Initializing TLS connection.
Control socket is using: TLSv1.2
Login security: TLSv1.2
> USER davvvid1
< 331 User name ok, need password.
> PASS ###
< 230 User davvvid1 logged on. Free service has restrictions and is slower.
> TYPE I
< 200 Type set to I
> STRU F
< 200 File structure set to file.
> PBSZ 0
< 200 PBSZ Command OK. Protection buffer size set to 0.
> PROT P
< 200 PROT Command OK.
Trying to find optimal transfer strategy...
> EPSV
< 501 Syntax error: Command not understood.
> PASV
< 227 Entering Passive Mode (66,220,9,50,17,140).
Optimal transfer strategy found.
> STOR file-name.txt
< 150 Connection accepted
Uploading (TLSv1.2)
< 226 Transfer complete
UPLOADED SUCCESSFULLY
Closing connections.
keep node active...
keep node active...
keep node active...
keep node active...
keep node active...
keep node active...
Closing connections.
keep node active...
keep node active...
keep node active...
keep node active...
keep node active...
keep node active...
Closing connections.

Which version of Node.js are you using?
Node 8.11.1

Additional context
setInterval in the code is only for keeping node process alive. Without it, process exits correctly. Maybe I create the stream in wrong way? But files are successfully uploaded and I see them on FTP servers. I tried on ftp.drivehq.com. Second FTP server is private and I even can't give you access to test but the issue is exactly the same so maybe the first example server would be enough.

VS Code typescript language server failed to import types

An notice will be shown when including the package.

const ftp = require("basic-ftp");

The notice:

[ts]
Could not find a declaration file for module 'basic-ftp'. %MyFolder%/node_modules/basic-ftp/lib/ftp.js' implicitly has an 'any' type.
  Try `npm install @types/basic-ftp` if it exists or add a new declaration (.d.ts) file containing `declare module 'basic-ftp';` [7016]

Well, I'll say it doesn't really relate to the module.
However, I would like to know whether any plan to get this working since many other npm package seems to work.
After some investigation, I believe other npm modules works because someone contributes on DefinitelyTyped which publishes the npm package @types/npm-package-name.
I think it is important becuase VS code is currently a very popular editor.

Closing client doesn't reject in-progress download

Describe the bug
When a client is closed, the in-progress download's promise doesn't resolve or reject, so the promise just hangs.
Not sure if this is working as expected, but I think it would be better to have any in-progress download promise resolved or rejected when the client is cancelled.

Example code

const fs = require("fs");
const Client = require("basic-ftp").Client;

async function demo() {
    const host = "<host>";
    const user = "<user>";
    const password = "<password>";
    const localFilePath = "./test.txt";
    const remoteFilePath = "test-repo/@cba_a3/addons/cba_common.pbo";
    const client = new Client();
    client.ftp.verbose = true;
    console.log("Connecting to host...");
    await client.access({
        host: host,
        user: user,
        password: password,
    });
    console.log("Connected to host. Starting download...");
    client.download(fs.createWriteStream(localFilePath), remoteFilePath)
        .then(() => {
            console.log("Download finished.");
        })
        .catch(() => {
            console.log("Download failed.");
        });
    setTimeout(async () => {
        console.log("Closing FTP client...");
        await client.close();
        console.log("FTP client closed.");
    }, 200);
}

demo();

Console output

Connecting to host...
Connected to XXX.XXX.XXX.XXX:21
< 220 ProFTPD 1.3.5b Server (FTP server) [XXX.XXX.XXX.XXX]
Login security: No encryption
> USER XXX
< 331 Password required for XXX
> PASS ###
< 230 User XXX logged in
> TYPE I
< 200 Type set to I
> STRU F
< 200 Structure set to F
Connected to host. Starting download...
Trying to find optimal transfer strategy...
> EPSV
< 229 Entering Extended Passive Mode (|||56144|)
Optimal transfer strategy found.
> RETR test-repo/@cba_a3/addons/cba_common.pbo
< 150 Opening BINARY mode data connection for test-repo/@cba_a3/addons/cba_common.pbo (253334 bytes)
Downloading from XXX.XXX.XXX.XXX:56144 (No encryption)
Closing FTP client...
Closing connections.
FTP client closed.

Which version of Node.js are you using?
8.11.3

Additional context
My use case: I have a list of download promises that are combined into a single promise that sequentially download the files one after the other. Each download promise checks if the process has already been cancelled and will reject, otherwise starts the download normally. The problem is that an in-progress download won't resolve until it has finished or failed. This stops the whole promise chain from resolving. I thought that closing the client would reject any in-progress download promise, but they just hang.

How do I use active mode?

Hi, I am very fortunate to have found this open source. I learned a lot from the code.

I have one thing to ask. How do I use active mode?

Many companies' FTP servers have active mode and passive mode for different security reasons.

Better mechanism for handling control socket errors when no active task

Managing the lifecycle of a client is a bit cumbersome at the moment. The problem is that errors on the control socket are routed through the current task.handler. (This is how data socket connections are handled as well, but isn't as big an issue there since data sockets are always created in the context of an upload or download task, so there's always a handler available to report the error through.)

Control sockets, unlike data sockets, are long-lived and can/do span multiple tasks. As a result, a client may go for extended periods with no active task, during which time socket errors get dropped on the floor. Leaving no documented way to know if, for example, there's a network disconnect or the remote server kills the connection.

I've worked around this for the moment by adding listeners to client.ftp.socket directly, but that feels a bit hacky, like I'm reaching into the internals of objects I'm not sure I should be touching. It's not clear if this is expected or might change in the future. This also requires adding the listeners if/when the socket is rebuilt. (I've opted for just creating a new client each time, though, to avoid possible issues with unexpected client state hanging around.)

I'm not sure what the right solution is here, but something more intuitive/better documented would be helpful. One possibility would be to have FtpContext inherit from EventEmitter, and forward socket events to its listeners, but I'm not sure how much I like that idea. It's adding complexity whereI feel like complexity should be getting removed. (E.g. If the Ftp Client/context/control socket were all tied together as a single object/API, then it'd be easier to rationalize how error handling should work.)

Is the timeout correct?

  • version : 3.4.2 , node 8.15
  • code
const ftp = require("basic-ftp");

const client = new ftp.Client(300);
await client.access({
    host: ... ,
    port: ... ,
    user: ... ,
    password: ...
});
  • result
Error: connect ETIMEDOUT xxx.xxx.xxx.xxx:21 (control socket)
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1191:14)

It took about 21000 ms, not 300 ms.

ENOTFOUND on Ubuntu

I'm getting the following error when using version 2.12.0 (works ok with 2.11.0):

Error: getaddrinfo ENOTFOUND #REDACTED#:21
at errnoException (dns.js:50:10)
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:92:26)
code: 'ENOTFOUND',
errno: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: '#REDACTED#',
host: '#REDACTED#',
port: 21

const ftp = require('basic-ftp');

connectToFTP()
    .then(function () {
        console.log('OK');
        process.exit(0);
    })
    .catch(function (err) {
        console.error(err);
        process.exit(1);
    });

async function connectToFTP () {
    const client = new ftp.Client(10000);

    client.ftp.verbose = true;
    client.ftp.log = function (message) {
        console.log(message);
    };

    try {
        await client.connect('#REDACTED#', 21);
        await client.useTLS();
    } catch (err) {
        console.error(err);
        throw err;
    } finally {
        client.close();
        console.log('done');
    }
}

running on:
Linux ip-172-31-25-241 4.4.0-1057-aws #66-Ubuntu SMP Thu May 3 12:49:47 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Download/upload progress?

I suppose this is something that could be done externally by analyzing the stream we write/read to/from, but it might be nice to have a way to track progress of oa download or upload in basic-ftp.

BTW, very nice module, it's the easiest I found for downloading a bunch of files sequentially with the same connection, and it works fine with promises :)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.