thephpleague / flysystem-sftp Goto Github PK

PR #63 has been open for a while, would it be possible to merge it?

I am getting this error and I have just updated to the latest packages.

 "require": {
        "league/flysystem": "1.0.55",
        "league/flysystem-sftp": "1.0.20",
        "vlucas/phpdotenv": "^2.5",
        "illuminate/database": "^5.5",
        "monolog/monolog": "^1.24",
        "pda/pheanstalk": "^3.2",
        "nesbot/carbon": "^2.24"
    },

When I check the secure log in my SFTP box, it opens and closes session for the new user fine but my script on my client box errors

PHP Notice:  Undefined offset: 256 in  vendor/phpseclib/phpseclib/phpseclib/Net/SFTP.php on line 443
PHP Notice:  Undefined offset: 256 in  /vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 4093
PHP Notice:  Error reading channel data in  /vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 3937
PHP Notice:  Undefined offset: 256 in  vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 4105
PHP Notice:  Undefined offset: 256 in vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 4106
PHP Notice:  Undefined offset: 256 in  vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 4113
PHP Notice:  Undefined offset: 256 in  vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 4117
PHP Notice:  Connection closed prematurely in  vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 3958

php -v
PHP 7.3.9 (cli) (built: Aug 28 2019 12:08:07) ( NTS )

When testing on a Core FTP Server the following issues happen:

• undefined index: type [line 229]
  $type = ($object['type'] === 1) ? 'file' : 'dir';
  Change to:
  $type = (isset($object['type']) && $object['type'] === NET_SFTP_TYPE_DIRECTORY) ? 'dir' : 'file';
• undefined index: type [line 398]
  $result['type'] = $info['type'] === NET_SFTP_TYPE_DIRECTORY ? 'dir' : 'file';
  change to:
  $result['type'] = (isset($object['type']) && $info['type'] === NET_SFTP_TYPE_DIRECTORY) ? 'dir' : 'file';

Thanks for this class its very useful!

Hello,

I need to implement your package to connect a sftp using a privateKey and an associated passphrase. I cannot provide this passphrase using your package.

Do you have an idea please ? Thanks a lot in advance

Hi,

I just noticed this repo doesn't have a lock file commited and, as of writing this, specifies a loose version constraint for its most important dependency: phpseclib/phpseclib:

"phpseclib/phpseclib": "~2.0"

It means, when a user includes a particular tagged version of flysystem-sftp, phpseclib/phpseclib version is going to vary with the time, depending when versions are resolved.

It makes debugging development vs production stuff confusing.

Is that an oversight or am I missing the greater scheme?

I am working on an older Laravel 4.2 application and want to make use of the SFTP adapter.

composer require league/flysystem-sftp
composer require league/flysystem

After requiring the two packages, I had the following added to composer.json:

		"league/flysystem": "^1.1@dev",
		"league/flysystem-sftp": "^1.0@dev"

The login method in SftpAdapter.php does not use the property "getters", instead it:

 if (! $this->connection->login($this->username, $this->password)) {

I've noticed this is fixed in 1.0.12, my composer.lock shows I have 1.0.6 installed. I updated composer.json to attempt installation of 1.0.12: "league/flysystem-sftp": "^1.0.12@dev", but it failed, seemingly due to the requirement for phpseclib 2.

What's the best approach to getting this fix into my current version? Thanks a lot

Hi,

A colleague and I believe found a error in this method:

public function getMetadata($path)
    {
        $connection = $this->getConnection();
        $info = $connection->stat($path);
        if ($info === false) {
            return false;
        }
        $result = Util::map($info, $this->statMap);
        $result['type'] = $info['type'] === NET_SFTP_TYPE_DIRECTORY ? 'dir' : 'file';
        $result['visibility'] = $info['permissions'] & $this->permPublic ? 'public' : 'private';
        return $result;
    }

We think the line:

$result['visibility'] = $info['permissions'] & $this->permPublic ? 'public' : 'private';

should be:

$result['visibility'] = $info['permissions'] **&&** $this->permPublic ? 'public' : 'private';

Here's my config. :

   'sftp' => [
        'driver' => 'sftp',
        'host'     => env('SFTP_HOST'),
        'port'     => 22,
        'username' => env('SFTP_USERNAME'),
        'root' => '/uploads/', 
        'privateKey' => env('SFTP_KEY_PATH'),
        'visibility' => 'public',
        'permPublic' => 0755,
        'directoryPerm' => 0755
    ]

   $remote_path = $clientName . '/' . $fileNameToStore;                    
   $ftp = Storage::disk('sftp')->put($remote_path, fopen($uploadedFile, 'r+'), 'public');

But the directoryPermission is always 0744.

File permission is changed to 0755 though.

If I use "League\Flysystem\Sftp\SftpAdapter" as Adapter then Method listContents() return always a blank array(). Other methods works so far? With "League\Flysystem\Adapter\Ftp" as adapter everything is ok. Any ideas (I use temporary an Snakeoil SSL Zertifikat, I think the phpseclib is the problem)?
Same with v 1.0.0 - 1.0.2

When using this SFTP adapter I cannot delete a directory on a server, I've debugged it and the following SFTP error is generated in the SFTP library you use: NET_SFTP_STATUS_FILE_IS_A_DIRECTORY: Specified file is a directory.

After further investigation, it seems you are calling delete in your deleteDir method. I believe this should be rmdir to remove a directory.

phpseclib 0.3.8 is a year old now. Versions 1.0.0 and 2.0.0 are now out, and are being used by framewords such as Laravel 5.1. This makes installing flysystem-sftp difficult on modern frameworks.

I don't know what v1 and v2 of phpseclib bring, and whether they would work for flysystem-sftp, but would they be compatible? Could the version requirements be extended?

Let's say your root has

-- dir1
   |-- some_dir
   |-- file1.txt
   `-- file2.txt
-- dir2

and you connect to that root and ask:

$contents  = $flysystem->listContents('/dir1');

you will get a Notice like this:

PHP Notice:  Undefined index: size in /your_project/vendor/league/flysystem-sftp/src/SftpAdapter.php on line 205

https://github.com/thephpleague/flysystem-sftp/blob/master/src/SftpAdapter.php#L205

Because $object['type'] == 2 doesn't have size index

I have a problem to upload in sftp...
I use sylex with wyrihaximus/sli-fly service provider, league/flysystem-ziparchive and obviously league/flysystem-sftp.

I have created a zip file and now I need to upload it on sftp server.
This is my config:

$app->register(new WyriHaximus\SliFly\FlysystemServiceProvider(), [

    'flysystem.filesystems' => [
        'zip' => [

            'adapter'   => 'League\Flysystem\ZipArchive\ZipArchiveAdapter',
            'args'      => [

                $app['root_dir'].'/web/file.zip',
            ],
        ],
        'sftp' => [

            'adapter'   => 'League\Flysystem\Sftp\SftpAdapter',

            'args'      => [

                [
                    'host'          => '192.241.178.210',
                    'port'          => 22,
                    'username'      => 'my_user,
                    'password'      => 'my_password',
                    'root'          => '/home/user/Documents/html/stream',
                    'timeout'       => 30,
                ]
            ]
        ]
    ],
]);

I create the zip file:

$this->filesystems['zip']->put('file.xml', $xml);

but when I try to upload it

$this->filesystems['sftp']->put('file.zip', '/var/www/Projects/Acme/web/file.zip');

On the server, file.zip was created, but it's a invalid file that I don't can open

Can i change the directoryperm => 755 to directoryPerm => 777? Because i can't create 0777 folder, only 0755, there is any possible way to change the folder permission?

Hey

I've been unable to figure out how to connect via a proxy?! I've searched on google, stackoverflow and your issues. Perhaps I'm blind? 👍

composer require league/flysystem-sftp

I got

You are running composer with xdebug enabled. This has a major impact on runtime performance. See https://getcomposer.org/xdebug
Using version ^1.0 for league/flysystem-sftp
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)

Installation failed, reverting ./composer.json to its original content.

                                                                                                                        
  [RuntimeException]                                                                                                    
  /Applications/MAMP/htdocs/code/web-app/vendor/league/flysystem-sftp does not exist and could not be created.

How do I avoid that ?

getting this error when running backup:clean from spatie/laravel-backup#705

Cleanup failed because: ftp_close(): SSL read failed.

Exception trace: #0 [internal function]: Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(2, 'ftp_close(): SS...', '/var/www/html/w...', 229, Array) #1 /var/www/html/www.mydomain.com/vendor/league/flysystem/src/Adapter/Ftp.php(229): ftp_close(Resource id #580) #2 /var/www/html/www.mydomain.com/vendor/league/flysystem/src/Adapter/AbstractFtpAdapter.php(577): League\Flysystem\Adapter\Ftp->disconnect() #3 /var/www/html/www.mydomain.com/vendor/league/flysystem/src/Adapter/Ftp.php(561): League\Flysystem\Adapter\AbstractFtpAdapter->getConnection() #4 /var/www/html/www.mydomain.com/vendor/league/flysystem/src/Adapter/Ftp.php(395): League\Flysystem\Adapter\Ftp->ftpRawlist('-A', 'http---www.find...') #5 /var/www/html/www.mydomain.com/vendor/league/flysystem/src/Adapter/AbstractFtpAdapter.php(535): League\Flysystem\Adapter\Ftp->getMetadata('http---www.find...') #6 /var/www/html/www.mydomain.com/vendor/league/flysystem/src/Filesystem.php(57): League\Flysystem\Adapter\AbstractFtpAdapter->has('http---www.find...') #7 /var/www/html/www.mydomain.com/vendor/laravel/framework/src/Illuminate/Filesystem/FilesystemAdapter.php(48): League\Flysystem\Filesystem->has('http---www.find...') #8 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/BackupDestination/Backup.php(29): Illuminate\Filesystem\FilesystemAdapter->exists('http---www.find...') #9 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/BackupDestination/BackupCollection.php(48): Spatie\Backup\BackupDestination\Backup->exists() #10 [internal function]: Spatie\Backup\BackupDestination\BackupCollection->Spatie\Backup\BackupDestination{closure}(Object(Spatie\Backup\BackupDestination\Backup), 1) #11 /var/www/html/www.mydomain.com/vendor/laravel/framework/src/Illuminate/Support/Arr.php(517): array_filter(Array, Object(Closure), 1) #12 /var/www/html/www.mydomain.com/vendor/laravel/framework/src/Illuminate/Support/Collection.php(282): Illuminate\Support\Arr::where(Array, Object(Closure)) #13 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/BackupDestination/BackupCollection.php(49): Illuminate\Support\Collection->filter(Object(Closure)) #14 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/Tasks/Cleanup/Strategies/DefaultStrategy.php(105): Spatie\Backup\BackupDestination\BackupCollection->oldest() #15 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/Tasks/Cleanup/Strategies/DefaultStrategy.php(118): Spatie\Backup\Tasks\Cleanup\Strategies\DefaultStrategy->removeOldBackupsUntilUsingLessThanMaximumStorage(Object(Spatie\Backup\BackupDestination\BackupCollection)) #16 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/Tasks/Cleanup/Strategies/DefaultStrategy.php(39): Spatie\Backup\Tasks\Cleanup\Strategies\DefaultStrategy->removeOldBackupsUntilUsingLessThanMaximumStorage(Object(Spatie\Backup\BackupDestination\BackupCollection)) #17 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/Tasks/Cleanup/CleanupJob.php(42): Spatie\Backup\Tasks\Cleanup\Strategies\DefaultStrategy->deleteOldBackups(Object(Spatie\Backup\BackupDestination\BackupCollection)) #18 /var/www/html/www.mydomain.com/vendor/laravel/framework/src/Illuminate/Support/Collection.php(228): Spatie\Backup\Tasks\Cleanup\CleanupJob->Spatie\Backup\Tasks\Cleanup{closure}(Object(Spatie\Backup\BackupDestination\BackupDestination), 1) #19 /var/www/html/www.mydomain.com/vendor/spatie/laravel-backup/src/Tasks/Cleanup/CleanupJob.php(52):

any ideas what could be the issue?

Hi, I am using Laravel and I updated my version to 1.0.13. The sftp adapter connects well, but when trying to read a file from the FTP, I encountered Unidentified index error.

[2017-02-21 23:08:56] local.ERROR: ErrorException: Undefined index: 0000000007f314c30000000000f2f7e7 in /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem/src/SafeStorage.php:30
Stack trace:
#0 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem/src/SafeStorage.php(30): Illuminate\Foundation\Bootstrap\HandleExceptions->handleError(8, 'Undefined index...', '/home/vagrant/c...', 30, Array)
#1 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem/src/Adapter/AbstractFtpAdapter.php(250): League\Flysystem\SafeStorage->retrieveSafely('password')
#2 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem-sftp/src/SftpAdapter.php(246): League\Flysystem\Adapter\AbstractFtpAdapter->getPassword()
#3 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem-sftp/src/SftpAdapter.php(191): League\Flysystem\Sftp\SftpAdapter->getAuthentication()
#4 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem-sftp/src/SftpAdapter.php(172): League\Flysystem\Sftp\SftpAdapter->login()
#5 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem/src/Adapter/AbstractFtpAdapter.php(575): League\Flysystem\Sftp\SftpAdapter->connect()
#6 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem-sftp/src/SftpAdapter.php(491): League\Flysystem\Adapter\AbstractFtpAdapter->getConnection()
#7 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem-sftp/src/SftpAdapter.php(483): League\Flysystem\Sftp\SftpAdapter->getMetadata('20.order/orders...')
#8 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem/src/Filesystem.php(56): League\Flysystem\Sftp\SftpAdapter->has('20.order/orders...')
#9 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem/src/Filesystem.php(384): League\Flysystem\Filesystem->has('20.order/orders...')
#10 /home/vagrant/code/kundan-v4-backend/vendor/league/flysystem/src/Filesystem.php(178): League\Flysystem\Filesystem->assertPresent('20.order/orders...')
#11 /home/vagrant/code/kundan-v4-backend/app/Listeners/EDI/SaveOrders.php(26): League\Flysystem\Filesystem->read('20.order/orders...')
#12 [internal function]: App\Listeners\EDI\SaveOrders->handle(Object(App\Events\EDI\FTPWasChecked))
#13 /home/vagrant/code/kundan-v4-backend/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(354): call_user_func_array(Array, Array)
#14 [internal function]: Illuminate\Events\Dispatcher->Illuminate\Events{closure}(Object(App\Events\EDI\FTPWasChecked))
#15 /home/vagrant/code/kundan-v4-backend/vendor/laravel/framework/src/Illuminate/Events/Dispatcher.php(221): call_user_func_array(Object(Closure), Array)
#16 /home/vagrant/code/kundan-v4-backend/vendor/laravel/framework/src/Illuminate/Foundation/helpers.php(480): Illuminate\Events\Dispatcher->fire('App\Events\EDI\...')
#17 /home/vagrant/code/kundan-v4-backend/routes/console.php(56): event(Object(App\Events\EDI\FTPWasChecked))
#18 [internal function]: Illuminate\Foundation\Console\ClosureCommand->{closure}()
#19 /home/vagrant/code/kundan-v4-backend/vendor/laravel/framework/src/Illuminate/Container/Container.php(508): call_user_func_array(Object(Closure), Array)
#20 /home/vagrant/code/kundan-v4-backend/vendor/laravel/framework/src/Illuminate/Foundation/Console/ClosureCommand.php(55): Illuminate\Container\Container->call(Object(Closure), Array)
#21 /home/vagrant/code/kundan-v4-backend/vendor/symfony/console/Command/Command.php(261): Illuminate\Foundation\Console\ClosureCommand->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#22 /home/vagrant/code/kundan-v4-backend/vendor/laravel/framework/src/Illuminate/Console/Command.php(155): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#23 /home/vagrant/code/kundan-v4-backend/vendor/symfony/console/Application.php(817): Illuminate\Console\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#24 /home/vagrant/code/kundan-v4-backend/vendor/symfony/console/Application.php(185): Symfony\Component\Console\Application->doRunCommand(Object(Illuminate\Foundation\Console\ClosureCommand), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#25 /home/vagrant/code/kundan-v4-backend/vendor/symfony/console/Application.php(116): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#26 /home/vagrant/code/kundan-v4-backend/vendor/laravel/framework/src/Illuminate/Foundation/Console/Kernel.php(121): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#27 /home/vagrant/code/kundan-v4-backend/artisan(35): Illuminate\Foundation\Console\Kernel->handle(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#28 {main}

Currently League\Flysystem\Sftp\SftpAdapter::getPrivateKey() does not check the bool result of RSA::loadKey() where false indicates error.

I hit this due to phpseclib/phpseclib#1372 where by regular OpenSSH generated keys wouldn't load when generated with recent versions.

Is it fair to say that this should be detected and a LogicException raised to ease identification of the problem? At present you end up with the general "Could not login ..." exception.

public function getPrivateKey()
$key->loadKey($this->privatekey);
Method 'loadKey' not found in \phpseclib\Crypt\RSA

\phpseclib\Crypt\RSA -> public function load($key, $type = false)

Hi,

I have an application which gets the data from CSV as a collection and send to a SFTP path. I am using this flysystem-sftp and everything was working fine and suddenly started to get 'Connection closed prematurely' error for my job which send the file to SFTP. Nothing is changed on my code .

SFTP server user a private key for authentication. Can some one help me figure out the problem

I was going to use this plugin and then realized I would have to create a service provider in order to integrate with Laravel.

Questions:

1. Would it be better to create a seperate flysystem-laravel repo to house all the custom serviceproviders? Or maybe a flysystem-sftp-laravel repo to hold this particular one (similiar to glide-laravel)
2. Is there any suggested/recommended file/repo structure to use to add this?

This adapter has a configuration option called $root.
It is not documented but I think it is supposed to do something like $pathPrefix.

In practice however it doesn't really do this.

• When listing directory contents the path is prefix with $root.
• When doing anything else the path is not prefix with $root.

We've recently had the contents of private keys emailed to us by cron when a connection fails, with the email contents looking something like the following:

PHP Notice:  Connection closed prematurely in /path/to/app/vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 2886
PHP Notice:  Connection closed by server in /path/to/app/vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 3205
PHP Notice:  Expected SSH_FXP_STATUS in /path/to/app/vendor/phpseclib/phpseclib/phpseclib/Net/SFTP.php on line 1960
PHP Notice:  Connection closed prematurely in /path/to/app/vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 3396
PHP Warning:  is_file(): open_basedir restriction in effect. File(-----BEGIN RSA PRIVATE KEY-----
REDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTED in /path/to/app/vendor/league/flysystem-sftp/src/SftpAdapter.php on line 168
PHP Notice:  Cannot connect to hostname.com:22. Error 111. Connection refused in /path/to/app/vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 1011
PHP Warning:  is_file(): open_basedir restriction in effect. File(-----BEGIN RSA PRIVATE KEY-----
REDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTEDREDACTED in /path/to/app/vendor/league/flysystem-sftp/src/SftpAdapter.php on line 168
PHP Notice:  Cannot connect to hostname.com:22. Error 111. Connection refused in /path/to/app/vendor/phpseclib/phpseclib/phpseclib/Net/SSH2.php on line 1011

Traced it down to the is_file() check in getPrivateKey() and wondered if there might be a better way to check?

Two options we came up with were:

1. check whether $this->privatekey starts with '-----' and only do the is_file check if it doesn't
2. have some kind of class flag to differentiate, rather than having to is_file

Any thoughts, or preference on approach?

Hello

Your protected $configurable property is wrong for your SftpAdater. You are declaring a 'privateKey' value (with uppercase K), but you are using a protected $privatekey property (with lowercase k)

This make the private key parameter unusable, when you instanciate your adapter.
I tried with a lowercase K in the protected $configurable declarations and in the configuration key when instanciating, and it works better.

You should use the same lowercase/uppercase naming in declaration/property/documentation.
Could you fix it please ?
Best Regards

this is my config:

'ssh-machine' => [
            'driver'        => 'sftp',
            'host'          => env('CMS_SSH_HOST'),
            'root'          => env('CMS_SSH_STORAGE_PATH'),
            'port'          => 22,
            'username'      => env('CMS_SSH_USER'),
            'password'      => env('CMS_SSH_PASSWORD', null),
            'privateKey'    => env('CMS_SSH_KEYFILE'),
            'keytext'       => '',
            'keyphrase'     => '',
            'directoryPerm' => 0760,
            'permPublic'    => 0660,
            'visibility'    => 'public',
            //            'agent'      => '',
            'timeout'       => 30
        ]

I would expect drwxrwx--- after a folder is created, but instead I get drwxr----- . Is there something I misunderstood or did I do something wrong?

This is what I do with Laravel

// in a provider

   Storage::extend('sftp', function ($app, $config) {
            return new Filesystem(new SftpAdapter($config));
        });

// in 
Storage::disk('ssh-machine')
                                   ->put($attachmentFileName, $rawFile, config('filesystems.disks.ssh-machine'));

//$attachmentFileName is something like /var/www/uploads/attachments/blabla.blub.bla.ext 

Hi,

we are trying our Laravel app to run on machine with PHP7. Now when some file transfer should be done it ends with this error:

Methods with the same name as their class will not be constructors in a future version of PHP; Net_SFTP has a deprecated constructor

Is it related to this package or phpseclib/phpseclib and the version of it flysystem-sftp depends ?

Looks like phpseclib 2.0 branch should have fixed it see this

Hi ,
How can I rename a Directory on the remote server.
In the api there is a method rename, but it is for files only and not for directories.

I'm not really sure where to file this issue, but this is the package we're using, so I start here.

What is wrong?

When I use this SftpAdapter with the Laravel Storage facade in a Job, things blow up after a day or so.
The worker process starts, connects to the sFTP-server, and pushes files to it. At night, there is no work, so the connection times out. The first files in the morning are then failing with a Connection closed prematurely. Since it's a Job in a worker-process, the Job fails, but the connection stays 'open' in the process. The only thing that helps is to restart the worker.

Debugging the problem

Before trying to upload, the SftpAdapter calls ->getConnection(), which in turn calls ->isConnected(), which has the following logic:

    public function isConnected()
    {
        if ($this->connection instanceof SFTP && $this->connection->isConnected()) {
            return true;
        }

        return false;
    }

... where SFTP is \phpseclib\Net\SFTP, which inherits ->isConnected() from \phpseclib\Net\SSH2.

It looks like this:

    /**
     * Is the connection still active?
     *
     * @return bool
     * @access public
     */
    function isConnected()
    {
        return (bool) ($this->bitmap & self::MASK_CONNECTED);
    }

And when I log it, it actually returns true, right before the Connection closed prematurely. The text in the docblock seems to tell us this is the right function to call, but the docblock on ->bitmap tells a different story:

    /**
     * Execution Bitmap
     *
     * The bits that are set represent functions that have been called already.  This is used to determine
     * if a requisite function has been successfully executed.  If not, an error should be thrown.
     *
     * @var int
     * @access private
     */
    var $bitmap = 0;

It seems to me that indeed, the ->connect() function has already been called, but the connection has gone away, so we are not connected anymore.

There is a closed issue on phpseclib which suggest we should send a ping to the host every so often, so we don't lose the connection.

Fix it here, then?

I don't think doing a ping is the right answer, but I think a solution could be to improve the SftpAdapter to have a better check for this connection.

Maybe it should detect / catch this Connection closed prematurely Exception and call ->disconnect(), setting it's ->connection to null? That could work.

Or do you think that this is in fact an issue in phpseclib, and I should open it there?

License file missing

Hi,
Could we get an option to disable the stat cache from phpseclib?
I need to know the size of a file without any caching.
See other issue here: thephpleague/flysystem#949 (comment)

I tested this locally and it worked:

    /**
     * Connect.
     */
    public function connect()
    {
        $SFTP = new SFTP($this->host, $this->port, $this->timeout);
        $SFTP->disableStatCache();
        $this->connection = $this->connection ?: $SFTP;
        $this->login();
        $this->setConnectionRoot();
    }

I receive the following error with this code.

Warning: Attempt to assign property of non-object in vendor\phpseclib\phpseclib\phpseclib\Net\SFTP.php on line 1152

Warning: Attempt to assign property of non-object in vendor\phpseclib\phpseclib\phpseclib\Net\SFTP.php on line 1155

$sftp = new Filesystem(new SftpAdapter([
            'host' => $this->config['ftpServer'],
            'port' => $this->config['ftpPort'],
            'username' => $this->config['ftpUser'],
            'password' => $this->config['ftpPass'],
            'timeout' => 10,
        ]));

$manager = new MountManager([
            'sftp' => $sftp,
        ]);

$contents = $manager->read('sftp://'.$file['path']); // Read contents
$manager->delete('sftp://'.$file['path']); // Delete file

The file does not delete from the remote SFTP site.

Versions
PHP: 7.0.11
Flysystem-SFTP: 1.0

I tried everything on this post :

$adapter = new SftpAdapter([
    'host' => '45.55.88.88',
    'port' => 22,
    'username' => 'root',
    'password' => '',
    'privateKey' => '/Users/bheng/.ssh/id_rsa',
    'root' => '/home/forge/bunlong-web-app/public/img/wedding/albums/',
    'timeout' => 10,
    'directoryPerm' => 0755
]);

$filesystem = new Filesystem($adapter);
dd($filesystem);

I got this

Filesystem {#292 ▼
  #adapter: SftpAdapter {#290 ▼
    #port: 22
    #hostFingerprint: null
    #privatekey: "/Users/bheng/.ssh/id_rsa"
    #useAgent: false
    -agent: null
    #configurable: array:14 [▼
      0 => "host"
      1 => "hostFingerprint"
      2 => "port"
      3 => "username"
      4 => "password"
      5 => "useAgent"
      6 => "agent"
      7 => "timeout"
      8 => "root"
      9 => "privateKey"
      10 => "permPrivate"
      11 => "permPublic"
      12 => "directoryPerm"
      13 => "NetSftpConnection"
    ]
    #statMap: array:2 [▼
      "mtime" => "timestamp"
      "size" => "size"
    ]
    #directoryPerm: 493
    #connection: null
    #host: "45.55.88.88"
    #ssl: false
    #timeout: 10
    #passive: true
    #separator: "/"
    #root: "/home/forge/bunlong-web-app/public/img/wedding/albums/"
    #permPublic: 484
    #permPrivate: 448
    #systemType: null
    #alternativeRecursion: false
    #safeStorage: SafeStorage {#291 ▶}
    #pathPrefix: null
    #pathSeparator: "/"
  }
  #plugins: []
  #config: Config {#293 ▼
    #settings: []
    #fallback: null
  }
}

What is this output mean ?

How do I know my connection/configuration is successfully configured ?

How do I upload a simple file to my server using this package ?

Can someone please elaborate a little bit more on the usage area ?

Thanks,

Hi,

When I am trying to get a list of files in a directory, I found that all files are read (streamed) in order to get the mimes types maybe, which make connection to be very slow or timeout (this is noticeable when you have files more than 10Mb).

Here is the trace we got when the adaptor is supposed to give back the list of files under a directory, you can see it is reading a 24Mb files.

Any idea? Thanks.

In response to: #79

This change shouldn't be a minor version because this class isn't final.

In other word after upgrading we occure an error becasue we overrides this class.

When I do $sftpAdapter->readStream($path) it returns an array that contains the stream under the stream key. Is that necessary? Shouldn't it return an actual stream?

In PR #19 we forgot to add a trailing directory Separator:

$this->root = $this->connection->pwd() . $this->separator;

When I try to write a stream via writeStream() it results in a zero byte file on the SFTP-Server.
With the Flysystem FTP adapter it works correctly.

        $stream = fopen($path_to_file, 'r', false, $context_i_set_somewhere);

        $storage = new Filesystem(new SftpAdapter($configuration_array_i_have_set_somewhere));

        $storage->writeStream($path_and_file_name_i_set_somewhere, $stream);

        fclose($stream);

"laravel/framework": "5.3.*"
"league/flysystem-sftp": "^1.0"

Any ideas? Stack trace as below

in Ftp.php line 263
at HandleExceptions->handleError('2', 'ftp_fput(): Syntax error', '/home/sites/1a/e/e3f0512ea8/app/vendor/league/flysystem/src/Adapter/Ftp.php', '263', array('path' => '', 'resource' => resource, 'config' => object(Config))) in ErrorHandler.php line 34
at Raven_Breadcrumbs_ErrorHandler->handleError('2', 'ftp_fput(): Syntax error', '/home/sites/1a/e/e3f0512ea8/app/vendor/league/flysystem/src/Adapter/Ftp.php', '263', array('path' => '', 'resource' => resource, 'config' => object(Config)))
at ftp_fput(resource, '', resource, '2') in Ftp.php line 263
at Ftp->writeStream('', resource, object(Config)) in Ftp.php line 243
at Ftp->write('', '/home/sites/1a/e/e3f0512ea8/app/app/Http/Controllers/test.txt', object(Config)) in Filesystem.php line 102
at Filesystem->put('', '/home/sites/1a/e/e3f0512ea8/app/app/Http/Controllers/test.txt', object(Config)) in FilesystemAdapter.php line 91
at FilesystemAdapter->put('/', '/home/sites/1a/e/e3f0512ea8/app/app/Http/Controllers/test.txt') in JobController.php line 44
at JobController->testFTP()
at call_user_func_array(array(object(JobController), 'testFTP'), array()) in Controller.php line 55
at Controller->callAction('testFTP', array()) in ControllerDispatcher.php line 44
at ControllerDispatcher->dispatch(object(Route), object(JobController), 'testFTP') in Route.php line 189
at Route->runController() in Route.php line 144
at Route->run(object(Request)) in Router.php line 653
at Router->Illuminate\Routing{closure}(object(Request)) in Pipeline.php line 53
at Pipeline->Illuminate\Routing{closure}(object(Request)) in CheckActive.php line 26
at CheckActive->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in SubstituteBindings.php line 41
at SubstituteBindings->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in Authenticate.php line 43
at Authenticate->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in VerifyCsrfToken.php line 65
at VerifyCsrfToken->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in ShareErrorsFromSession.php line 49
at ShareErrorsFromSession->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in StartSession.php line 64
at StartSession->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in AddQueuedCookiesToResponse.php line 37
at AddQueuedCookiesToResponse->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in EncryptCookies.php line 59
at EncryptCookies->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in Pipeline.php line 104
at Pipeline->then(object(Closure)) in Router.php line 655
at Router->runRouteWithinStack(object(Route), object(Request)) in Router.php line 629
at Router->dispatchToRoute(object(Request)) in Router.php line 607
at Router->dispatch(object(Request)) in Kernel.php line 268
at Kernel->Illuminate\Foundation\Http{closure}(object(Request)) in Pipeline.php line 53
at Pipeline->Illuminate\Routing{closure}(object(Request)) in CheckForMaintenanceMode.php line 46
at CheckForMaintenanceMode->handle(object(Request), object(Closure)) in Pipeline.php line 137
at Pipeline->Illuminate\Pipeline{closure}(object(Request)) in Pipeline.php line 33
at Pipeline->Illuminate\Routing{closure}(object(Request)) in Pipeline.php line 104
at Pipeline->then(object(Closure)) in Kernel.php line 150
at Kernel->sendRequestThroughRouter(object(Request)) in Kernel.php line 117
at Kernel->handle(object(Request)) in index.php line 53

Why does the library use the phpseclib instead of php built-in functions of ssh2.sftp (http://php.net/manual/de/function.ssh2-sftp.php)?

At the moment, reading a stream via the sftp adapter is consuming a lot of memory. Perhaps, it's better with the php built-in functions (not tried yet)?

Hey, i updated to v1.0.13 because of the "fixed" issue here:
1.0.12...1.0.13

This still doesn't work out for me because when type is not set, the object seems to be a file, not a directory. The check must be the following from my point of view:

$type = isset($object['type']) && ($object['type'] === 2) ? 'dir' : 'file' ;

print_r($object) returns me for my folder content the following:
Array ( [size] => 0 [uid] => 0 [gid] => 0 [permissions] => 16822 [mode] => 16822 [type] => 2 [atime] => 1481803905 [mtime] => 1483371669 [filename] => OLD ) Array ( [size] => 120 [uid] => 0 [gid] => 0 [permissions] => 438 [mode] => 438 [atime] => 1483542521 [mtime] => 1483542521 [filename] => test1.csv ) Array ( [size] => 0 [uid] => 0 [gid] => 0 [permissions] => 16822 [mode] => 16822 [type] => 2 [atime] => 1483373074 [mtime] => 1483542511 [filename] => tmp_process-later )

test1.csv is obviously a file and type is not set. I didn't check further right now, but currently i assume this should do the trick. I will dive more into it and provide hopefully a fix.

Hi,

Tested this connector and it is working fine but very slow to do the transfert. When using WinSCP no problem but with this connector it is very slow to get the files / folders.

Any option to add somewhere that I am missing?

Hello,

flysystem's documentation does not clearly state which values must be included but I've seen that other adapters are returning the path. The SftpAdapter does include it in the result of the write and writeStream methods but it's missing in read and readStream.

Flysystem-sftp depends on phpseclib, which already have a DSA implementation. There is any limitation preventing to update the phpseclib and using DSA?

Setting "'agent' => true" inside the Config array (as stated in the Readme) leads to the following error message:

TypeError: Argument 1 passed to League\Flysystem\Sftp\SftpAdapter::setAgent() must be an instance of phpseclib\System\SSH\Agent, boolean given, called in /src/vendor/league/flysystem/src/Adapter/AbstractFtpAdapter.php on line 112

I want to preface by saying I am not sure this is an issue with this adapter, but I am wondering how I can further diagnose this to be sure of that...

I am occasionally running into multiple "File not found at path" issues. At first I thought it was legitimate and the hook that was running a download job was firing before the file was finished writing or something. However, after closer inspection and checking the timestamps of the files versus the timestamps of my error logs, I am seeing the files are actually there. I can later repeat the job (manually in a tinker session) and it downloads fine. The main thing I have noticed is that this problem seems to be grouped with a series of jobs for a while (maybe 6-7 jobs error out), then runs fine again for a while. The crux of what I am doing is trying to download cloud meeting recordings for local processing.

Here is a typical error, the rest of the stack trace is all callback and queue stuff:

[2018-10-02 22:21:42] production.ERROR: File not found at path: opt/zoom/cmr-storage/replay/2018/10/02/XXXXX/YYYYY/GMT20181002-221928_Untitled---10-02-2018-05-18-pm_1280x720.mp4 {"exception":"[object] (League\\Flysystem\\FileNotFoundException(code: 0): File not found at path: opt/zoom/cmr-storage/replay/2018/10/02/XXXXX/YYYYY/GMT20181002-221928_Untitled---10-02-2018-05-18-pm_1280x720.mp4 at /home/forge/myapp.com/releases/20180928151417/vendor/league/flysystem/src/Filesystem.php:388)
[stacktrace]
#0 /home/forge/myapp.com/releases/20180928151417/vendor/league/flysystem/src/Filesystem.php(194): League\\Flysystem\\Filesystem->assertPresent('opt/zoom/cmr-st...')
#1 /home/forge/myapp.com/releases/20180928151417/app/Jobs/ZoomProcessing/DownloadMeetingRecordings.php(42): League\\Flysystem\\Filesystem->readStream('opt/zoom/cmr-st...')
#2 /home/forge/myapp.com/releases/20180928151417/vendor/laravel/framework/src/Illuminate/Support/Collection.php(397): App\\Jobs\\ZoomProcessing\\DownloadMeetingRecordings->App\\Jobs\\ZoomProcessing\\{closure}('/opt/zoom/cmr-s...', 0)
#3 /home/forge/myapp.com/releases/20180928151417/app/Jobs/ZoomProcessing/DownloadMeetingRecordings.php(47): Illuminate\\Support\\Collection->each(Object(Closure))
#4 [internal function]: App\\Jobs\\ZoomProcessing\\DownloadMeetingRecordings->handle(Object(App\\Services\\Meeting\\ZoomRecordingProcessingService))

Here is what I am seeing on the SFTP server:

[root@redacted]# pwd   
/opt/zoom/cmr-storage/replay/2018/10/02/XXXXX/YYYYY
[root@redacted]# ls -al --time-style=full-iso
total 3888
drwxr-xr-x 2 root root    4096 2018-10-02 22:21:34.000000000 +0000 .
drwxr-xr-x 3 root root    4096 2018-10-02 22:21:34.000000000 +0000 ..
-rw-r--r-- 1 root root      70 2018-10-02 22:21:34.000000000 +0000 GMT20181002-221928_Untitled---10-02-2018-05-18-pm.vtt
-rw-r--r-- 1 root root 3964667 2018-10-02 22:21:34.000000000 +0000 GMT20181002-221928_Untitled---10-02-2018-05-18-pm_1280x720.mp4

The files timestamp is set roughly 8 seconds prior to attempting the download. All the paths appear correct.

And here is the job:

public function handle(ZoomRecordingProcessingService $processingSvc)
    {
        $this->filePaths->each(function ($filePath) use ($processingSvc) {
            // open a stream to the file on the recording server
            $fileStream = $processingSvc->remoteRecordings()->getDriver()->readStream($filePath);

            // copy the file to local storage under 'meeting-id/filename'
            $filename = basename($filePath);
            $processingSvc->localRecordings()->put("{$this->meeting->id}/{$filename}", $fileStream);
        });
        \Log::info('Downloaded ' . $this->filePaths->count() . ' recordings for meeting: ' . $this->meeting->id);
    }

remoteRecordings() just returns the storage disk being handled by the SFTP adapter. And localRecordings() is just a local disk.

I feel like this is a pretty straightforward scenario, and it works great 90% of the time. One odd thing I noticed was that Filesystem appears to be stripping the lead '/', at least from what I seeing in the logs, but I am clearly passing it as part of the $filePath. I also even tried a manual test without the '/' and works anyway... I also have 3 retries set for this job (2 seconds apart), when one fails they all fail.

Could the connection be getting stale or something? Any ideas of how I can further debug this?

I've uploaded an file on an sftp server which I can get the content from like:

$content = Storage::disk('sftp')->get($remoteFile);

However instead of the following content:
Apprendre à lire très vite, c'est possible, mais ça n'a aucun sens
I get:
b"Apprendre ê lire trÅs vite, c'est possible, mais ìa n'a aucun sens"

Is there a way to enforce utf8 encoding?

This might be something I'm doing wrong, but here goes:

I've got a user on a server I can ssh into - everything works, can see all the files I need to, no problem.

I can FileZilla (using SFTP, all the same creds, password auth is turned off), again everything works, can see all the files I need to, no problem.

Connect with this package, same creds, connection succeeds and I can see only the root folder (/).
Everything is empty and I can't see either the user directory(~) or the mounted disk contents ('/mnt/').

I've tried re-chowning and re-chmodding but that doesn't help, I don't think it's permissions because I can see the files on other SFTP clients and using SSH as the same user.

It seems this adapter fully ignores a configured path prefix.

SFTP Fails login when SFTP/SSH server requires double authentication: publickey and password.

Reproduce:

• Validate login works independently with password and ssh key.
• Set AuthenticationMethods to publickey,password on server.
• Set both password and privatekey values for SFTP Adapter.

Login fails.

Independently, login works successfully. It's only when both authentication methods are required does the login fail.