remote_ssh : ERROR: src/utils/file.c:529: proceeds 0 bytes instead of 8

[achix]

Hello,
just started to test remote capabilities and I get :
/pg_probackup/pg_probackup add-instance -B /usr/local/var/lib/pgsql/mypgprobacks --remote-proto=ssh --remote-host=192.168.1.109 --remote-port=5432 --remote-path=/var/lib/postgresql/pg_probackup -D /var/lib/postgresql/datatest --instance testdblin
ERROR: src/utils/file.c:529: proceeds 0 bytes instead of 8

the catalog host is FreeBSD and the remote is linux. Could that pose a problem?

[gsmolk]

Hello!
Can you pull recent changes from remote_ssh, rebuild binaries on host and remote machines and try one more time?

the catalog host is FreeBSD and the remote is linux. Could that pose a problem?

probably, but I`m not sure

[gsmolk]

Oh, I`ve forgot, there is now --remote option, which must be added to cmdline.
--remote-proto is now optional with default value 'ssh'

[knizhnik]

You are also incorrectly using --remote-path options.
I am not sure if it is the reason of the reported error, but if --remote-path options is specified, it is concatenated with name of the program. In you case it is "/pg_probackup/pg_probackup".
So result will be something like: "/var/lib/postgresql/pg_probackup//pg_probackup/pg_probackup".
Not sure that it is what you have intended.,,

[achix]

Hello, thanks, it would have to be just : --remote-path=/var/lib/postgresql . I'll test again.

[gsmolk]

I am not sure if it is the reason of the reported error, but if --remote-path options is specified, it is concatenated with name of the program

I don`t think this is a right way. Remote binary can have a different name, so --remote-path should point to binary, not directory.

[achix]

Ok, I am testing against are large testdb :
I successfully :

• built pg_probackup (remote_ssh) on both the catalog host and the database host, both running linux same version
• on the catalog host :

./pg_probackup/pg_probackup init -B /var/lib/pgbackup/pgprobackups
INFO: Backup catalog '/var/lib/pgbackup/pgprobackups' successfully inited
./pg_probackup/pg_probackup add-instance -B /var/lib/pgbackup/pgprobackups --remote --remote-host="DBHOST" -D /var/lib/pgsql/data --instance testdb
INFO: Instance 'testdb' successfully inited
But when I try to run the first backup I get :
./pg_probackup/pg_probackup backup -B /var/lib/pgbackup/pgprobackups/ --instance testdb -b FULL --stream
INFO: Validating backup 0
ERROR: cannot open "/var/lib/pgbackup/pgprobackups/backups/testdb/0/backup_content.control": No such file or directory

[achix]

hello,
git fetched rebased , in both hosts, now it won't even let me add the instance :
postgres@smadb2cs:~$ ./pg_probackup/pg_probackup add-instance -B /var/lib/pgbackup/pgprobackups -D /var/lib/pgsql/data --remote-proto=ssh --remote-host=10.9.0.77 --remote-path=/var/lib/pgsql/pg_probackup/pg_probackup --instance testdbERROR: src/utils/file.c:537: proceeds 0 bytes instead of 8: No such file or directory

postgres@smadb2cs:~$ ./pg_probackup/pg_probackup -V
pg_probackup 2.0.26 (PostgreSQL 10.6)

[gsmolk]

fixed

[gsmolk]

--remote option is now obsolete

[achix]

./pg_probackup/pg_probackup add-instance -B /var/lib/pgbackup/pgprobackups -D /var/lib/pgsql/data --remote-proto=ssh --remote-host=10.9.0.77 --remote-path=/var/lib/pgsql/pg_probackup/. --instance testdb
INFO: Instance 'testdb' successfully inited

this worked.

also backup with --stream seems to start.

pls tell how's the archive command supposed to look like for remote? I cannot figure it out.

it always asks for local -B. should I create an instance in the PostgreSQL host as well?

[achix]

tried backups like :

./pg_probackup/pg_probackup backup -B /var/lib/pgbackup/pgprobackups/ --instance testdb -b FULL --stream
INFO: Backup start, pg_probackup version: 2.0.26, backup ID: PLSQ78, backup mode: full, instance: testdb, stream: true, remote true
INFO: Start transfering data files
^CERROR: src/utils/file.c:863: proceeds 2741 bytes instead of 8200: File exists

^^^ this seems to work. Just interrupted it.

postgres@smadb2cs:$
postgres@smadb2cs:$
postgres@smadb2cs:~$ ./pg_probackup/pg_probackup backup -B /var/lib/pgbackup/pgprobackups/ --instance testdb -b FULL
INFO: Backup start, pg_probackup version: 2.0.26, backup ID: PLSQ87, backup mode: full, instance: testdb, stream: false, remote true
INFO: Wait for WAL segment /var/lib/pgbackup/pgprobackups/wal/testdb/00000001000009880000003B to be archived

^^^ this I cannot figure out how to setup archive_command for remote

currently testing :

trying with :
./pg_probackup/pg_probackup backup -B /var/lib/pgbackup/pgprobackups/ --instance testdb -b FULL -j 8 --stream

this will take hours..

but pls tell me about archive-push from the pgsql host to the catalog host.

[achix]

still running , I dont see -j 8 to be respected, current speed below what this line can do : 8.2MB/s.
However it seems to work. Thanks! It will more than day for 1.3TB.

[achix]

delete does not delete back up, seems to work but info still shows backup.

[gsmolk]

yes, we will fix that

[gsmolk]

still running , I dont see -j 8 to be respected, current speed below what this line can do : 8.2MB/s.
However it seems to work. Thanks! It will more than day for 1.3TB.

What capacity this line has?
Did you try to use a weaker encryption algorithm?