Skip to content

File transfers

Jump to bottom
Aaron W Morris edited this page Apr 2, 2025 · 30 revisions

Overview

indi-allsky supports several methods of file transfers. All are implemented using native python features and modules.

Protocol Class Name Port Description
ftp pycurl_ftp 21 FTP via pycurl
python_ftp 21 FTP via ftplib
ftpes pycurl_ftpes 21 FTPS (explicit) via pycurl
python_ftpes 21 FTPS (explicit) via ftplib
ftps pycurl_ftps 990 FTPS (implicit) via pycurl
sftp pycurl_sftp 22 SFTP via pycurl
paramiko_sftp 22 SFTP via paramiko
webdav (https) pycurl_webdav_https 443 HTTPS PUT via pycurl

PycURL Options

File transfer variables

Name Type Info
timestamp datetime https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes
ts datetime Shortcut for timestamp
day_date date Day reference for exposure. Referencing the date when the timelapse set starts.
ext str File extension (no dot)
camera_uuid str Camera UUID
camera_id int Camera ID
timeofday str night or day
tod str Shortcut for timeofday

Atomic file transfers

Atomic file transfers upload the file as a temporary name then renames the file after it is fully uploaded. This prevents images from being partially loaded when viewed while the file is being uploaded.

Atomic support

  • pycurl_sftp
  • pycurl_ftp
  • pycurl_ftpes
  • pycurl_ftps

SFTP

PycURL SFTP

SFTP protocol implemented via libcurl.

  • Does not support relative remote paths. You may use tilde ~ for the remote home directory.
    • ~ shortcut is not compatible with atomic transfers
  • You may have to manually accept the remote hostkey using ssh on the command line before file transfers can occur.
  • Use Private and Public key inputs or PycURL Options for key based authentication
    • Both private and public keys must be specified
    • If you specify the public and private keys, the password field is used for the key password
    • Only supports rsa keys in PEM format when libcurl compiled with libgcrypt (Debian, Raspbian, etc)
      • Does not support dsa, ecdsa, or ed25519 keys
    • Convert keys using ssh-keygen -p -m PEM -f /home/pi/.ssh/id_rsa
      • Correct key: -----BEGIN RSA PRIVATE KEY-----
      • If ssh key starts with -----BEGIN OPENSSH PRIVATE KEY----- IT WILL NOT WORK

Paramiko SFTP

SFTP protocol implemented via the paramiko module.

  • Use Private Key input for key based authentication (public key is not used)
  • Supports relative and full remote paths.
    • Does not support tilde ~ shortcut
  • Automatically detects and uses ssh keys (rsa, dsa, ecdsa, ed25519) if located at default paths
  • Host key validation is disabled by default

FTP

PycURL FTP

FTP protocol implemented via libcurl.

Python FTP

FTP protocol using Python's native ftp support.

PycURL FTPES

FTPES protocol via libcurl.

  • ftpes sessions start like a regular FTP session on port 21, but negotiate TLS encryption for authentication and data transfers. Similar to START_TLS with SMTP or LDAP.
  • Certificate validation is disabled by default

Python FTPES

FTPES protocol via Python's native ftp support.

  • ftpes sessions start like a regular FTP session on port 21, but negotiate TLS encryption for authentication and data transfers. Similar to START_TLS with SMTP or LDAP.
  • Certificate validation is disabled by default

PycURL FTPS

FTPS protocol via libcurl.

  • FTPS is implicitly encrypted for the full session.
  • Certificate validation is disabled by default

HTTP

PycURL WebDAV

HTTP(S) protocol via libcurl.

  • Remote server must support webdav/web disk.
  • Encrypted when endpoint is https://
  • Certificate validation is disabled by default

Notes

Clone this wiki locally