Plugin curl-ftp: SFTP issues

Hi,

we need to upload files using curl-ftp with SFTP and public key authentication. As stated in https://forums.4d.com/Post/EN/28813934/1/28813935#28813935 we’re facing the same problems - error 2 and/or crashes. In detail:

Version 2.12 32-Bit
macOS 10.13.6 - 4D v17.2 Build 240047: OK
Windows 10 1903 - 4D v17.2 Build 240047: Failure. Starts upload, but never ends, Application hangs

Version 2.12 64-Bit
macOS 10.13.6 - 4D v17 R4 Build 235715: OK
Windows 10 1903 - 4D v17 R4 Build 235715: OK

Version 2.18 32-bit
macOS 10.13.6 - 4D v17.2 Build 240047: Error code 2
Windows 10 1903 - 4D v17.2 Build 240047: Application crash

Version 2.18 64-bit
macOS 10.13.6 - 4D v17 R4 Build 235715: Error code 2
Windows 10 1903 - 4D v17 R4 Build 235715: Application crash

Any ideas? We need to upload from both platforms.

Thanks and regards, Jörg

NB: Test structure can be supplied if needed.

Hello

the plugin is open source so anyone is welcome to contribute :mrgreen:

also, GitHub has its own issue tracking system (but I don’t insist I also follow forums)

as for me, I am sorry I can never help without a reproducible minimal DB.

Hi,

thanks for your reply! Here’s the simple structure: https://forums.4d.com/4DBB_Main/x_User/3979/files/31508988.zip . Enclosed are zip’s of the two versions of cURL ftp (2.12 and 2.18).

To reproduce, you must have access to an SFTP server that supports public key authentication. An easy way - that’s what I did - is to setup Rumpus for macOS. It gives you a trial period for a couple of weeks and after a few minutes the SFTP server was up and running.

Thanks for your help, Jörg

https://github.com/miyako/4d-plugin-curl-ftp/releases/tag/3.0

Hello,

the error code 2 was returned because of type mismatch.

in your code you had

$options.SSL_VERIFYPEER:=‘0’
$options.SSL_VERIFYHOST:=‘0’

but numeric values were expected for these options.
anyway, I changed the code so that either type is acceptable.

as for sftp:

I tested password auth and public key auth, both worked on mac/win (64-bit).

c.f.

$CURLSSH_AUTH_PUBLICKEY:=(1 << 0)
$CURLSSH_AUTH_PASSWORD:=(1 << 1)

$options.SSH_AUTH_TYPES:=$CURLSSH_AUTH_PUBLICKEY // | $CURLSSH_AUTH_PASSWORD


but I did not look into the large file issue on windows.

Hi Keisuke,

shame on me :oops: I did copy/paste from the cURL-Plugin calls where all parameters were in a text array…

Thanks for pointing that out and thanks for extending the types, so the plugin doesn’t crash when passing a text value. SFTP now works on all platforms mentioned above. Topic solved :smiley:

Keep up your great work supplying the 4D community with your plugins!

Regards, Jörg

Hi Miyako,

could you help me with this:

<code 4D>
C_OBJECT($options)
C_LONGINT($error;$CURLSSH_AUTH_PUBLICKEY;$CURLSSH_AUTH_PASSWORD)
C_TEXT($dir)

$CURLSSH_AUTH_PUBLICKEY:=(1 << 0)
$CURLSSH_AUTH_PASSWORD:=(1 << 1)

$options:=New object

$options.URL:=‘sftp://blog.adamov.rs/’
$options.USERNAME:=‘myusername’
$options.CURLOPT_DEBUG:=Get 4D folder(Logs folder)
$options.CURLOPT_USE_SSL:=1
$options.CURLOPT_SSL_VERIFYPEER:=0
$options.CURLOPT_SSL_VERIFYHOST:=0
$options.CURLOPT_SSH_AUTH_TYPES:=$CURLSSH_AUTH_PUBLICKEY //| $CURLSSH_AUTH_PASSWORD
$options.CURLOPT_SSH_PRIVATE_KEYFILE:=Convert path POSIX to system(’/Volumes/Radni/madamov/.ssh/id_rsa’)
$options.CURLOPT_KEYPASSWD:=‘myprivatekeyphrase’

$error:=cURL_FTP_GetDirList (JSON Stringify($options);$dir)

</code 4D>

I am always getting error 67 (permission denied). If I use the same key and passphrase in Terminal (sftp -i /Volumes/Radni/madamov/.ssh/id_rsa myusername@blog.adamov.rs), Cyberduck or Transmit, I can login without problems.

It is on macOS Mojave, 64-bit v17.3, curl-ftp version is 3.4.

Am I doing something wrong?

what if you use a POSIX path instead of system?

Hi Milan,

Also, your path should maybe be :

‘/Users/madamov/.ssh/id_rsa’

  1. the name ‘Users’ is translated in the Finder (‘Radni’ in serb ?), but not sure you use the translation in path.
  2. ‘/Volumes’ should be used in front of ‘mounted’ volume names, the ‘Users’ dir is usually on the boot disk (unless you have an unusual configuration).

HTH

: Keisuke MIYAKO

what if you use a POSIX path instead of system?

The same: error 67.

: Bruno LEGAY

Also, your path should maybe be :

‘/Users/madamov/.ssh/id_rsa’

  1. the name ‘Users’ is translated in the Finder (‘Radni’ in serb ?),
    but not sure you use the translation in path.
  2. ‘/Volumes’ should be used in front of ‘mounted’ volume names, the
    ‘Users’ dir is usually on the boot disk (unless you have an unusual
    configuration).

/Volumes/Radni/madamov is my $HOME folder on separate partition (well, separate volume in APFS container) from system partition, nothing unusual about that, it I like that since I started using Mac OS X. madamov is standard user, not administrator. Nevertheless, it should work since everything else works and especially since

sftp -i /Volumes/Radni/madamov/.ssh/id_rsa myusername@blog.adamov.rs

works. Worth trying, though, since I already tried everything I could think of.

BTW, ‘Radni’ means ‘Working’ or ‘Work’ partition. B)

If you could provide a functional sample
(using a public test server or something like that)
I could take a look, but otherwise,
I am afraid I do not have time to prepare a test environment to debug.
(I know it should be easy with macOS but I don’t even have time for that)

Thanks, I sent you email with real credentials and with keys you can use to test.

After the email exchange with Keisuke and some testing I can tell that you can use curl-ftp plugin for password less authentication if you:

Here is the example that worked:

<code 4D>
$CURLSSH_AUTH_PUBLICKEY:=(1 << 0)
$CURLSSH_AUTH_PASSWORD:=(1 << 1)

$options:=New object

$options.URL:=‘sftp://yoursite.com/’
$options.USERNAME:=‘username’
$options.DEBUG:=Get 4D folder(Logs folder) // optional
$options.USE_SSL:=1
$options.SSL_VERIFYPEER:=0
$options.SSL_VERIFYHOST:=0
$options.SSH_AUTH_TYPES:=$CURLSSH_AUTH_PUBLICKEY
$options.SSH_PRIVATE_KEYFILE:=Get 4D folder(Current resources folder)+‘id_in_pem_format’
$options.KEYPASSWD:=‘privatekeypassphrase’

$error:=cURL_FTP_GetDirList (JSON Stringify($options);$dir)
</code 4D>

To generate key pair in PEM format use

ssh-keygen -m PEM

which defaults to 2048-bit key, you can use -b to specify bit length

ssh-keygen -m PEM -b 4096

Bonjour Miyako,

J’ai jeté un oeil à la dernière version du plugin cURL.
Je vois qu’il n’inclut plus curl.exe ni les dll dépendantes. Ma question est : comment fonctionne-t-il du coup ?
Si cURL n’est pas installé sur le système pour une raison x ou y, j’imagine que ça ne fonctionne pas ?

Cordialement.

Sylvain.

il va marcher seul, par lui-même, car les dépendantes (curl, openssl, ssh2, http2…) sont intégrés comme ‘static library’.

Merci pour ta réponse.
Je vais plutôt m’orienter sur celui-ci dans ce cas.

Cordialement.

Sylvain.