remote site image upload issues

[shotscot]

Hey guys, before I start. I have tried multiple things which I will list below and I have also searched the GitHub discussions for a solution and read through the all sky wiki.

I managed to finally install the remote site after some ongoing issues. After the install I changed the remote site settings through the WebUI editor. I was initially getting bad 404 & 403 error warnings when trying to view my remote site which I eventually overcome. This was down to my folder layout not having a public_html I believe! once I added that I could see my site. (you can see my folder layout in the image below).

the first issue I noticed is the testUpload.sh.txt and future images are going into the top directory of the site. I am unsure if this is down to the Image URL and Image directory settings in the allsky settings in the WebUI. (see photos for settings) I have chopped and changed these multiple times.
If I change the image directory to anything other than leaving it blank I get a RET1 error. I have tried - public_html/allsky & allsky.

I am unsure if I have changed some setting somewhere that I shouldn't have or if my folder layout is completely wrong? I have no doubt it is a mistake on my behalf, I just don't know what.

[shotscot]

I am struggling! I have got to a new stage where my images are generating in the correct folder, now my only issue is this...

I dont know how to run postdata.sh, im not that savvy

[EricClaeys]

About an hour ago I was able to get to your Website at https://annanallsky.co.uk/allsky/.
Now I can't, but I can get to it at http://annanallsky.co.uk/allsky/.
Are you doing something to the Website?
What URL did you use for the screenshot above?

[shotscot]

I've reinstall the website again to give it a fresh start but I've lost connection to the RPI and I've given up for tonight I think. The URL is what I used for that last screenshot! I'm hopeless with all this

[EricClaeys]

We'll get you through this, I'm confident. I know it can be frustrating when it doesn't work. It's nice once you get it going, though.

Make sure to read this page which may help you determine what to put in the Image Directory and Website URL settings for the server.
The Image URL should almost always be the Website URL with /image.jpg appended to it.

The Image Directory setting tells FTP where to go once it's logged into the server.
The Website URL tells your browser where to go on the server.
The trick is to make sure both setting point to the same place on the server.

Run testUpload.sh --debug --website and then copy/paste the result into your reply. That should tell us where FTP is on the server.

[shotscot]

hi Eric,

Ill have a read through the page you linked.

here is the debug results -

Executing:
        /home/chris/allsky/scripts/upload.sh --debug --remote-web /tmp/testUpload.sh.txt null testUpload.sh.txt testUpload.sh
Test upload to Remote Website succeeded.
        OUTPUT:
Upload output from '/tmp/testUpload.sh.txt:
   ---- Resolving host address...
---- 1 address found: 35.214.92.81
---- Connecting to gukm1024.siteground.biz (35.214.92.81) port 21
<--- 220-##########################################################
<--- 220-Please upload your web files to the public_html directory.
<--- 220-Note that letters are case sensitive.
<--- 220-##########################################################
<--- 220 This is a private system - No anonymous login
---> FEAT
<--- 211-Extensions supported:
<---  UTF8
<---  EPRT
<---  IDLE
<---  MDTM
<---  SIZE
<---  MFMT
<---  REST STREAM
<---  MLST type*;size*;sizd*;modify*;UNIX.mode*;UNIX.uid*;UNIX.gid*;unique*;
<---  MLSD
<---  PRET
<---  AUTH TLS
<---  PBSZ
<---  PROT
<---  TVFS
<---  ESTA
<---  PASV
<---  EPSV
<---  ESTP
<--- 211 End.
---> AUTH TLS
<--- 234 AUTH TLS OK.
---> OPTS UTF8 ON
WARNING: Certificate verification: hostname checking disabled
<--- 504 Unknown command
---> USER chris@annanallsky.co.uk
<--- 331 User chris@annanallsky.co.uk OK. Password required
---> PASS XXXX
<--- 230-Your bandwidth usage is restricted
<--- 230 OK. Current restricted directory is /
---> OPTS UTF8 ON
<--- 504 Unknown command
---> OPTS MLST type;size;modify;UNIX.mode;UNIX.uid;UNIX.gid;
<--- 200  MLST OPTS type;size;sizd;modify;UNIX.mode;UNIX.uid;UNIX.gid;unique;
---> PWD
<--- 257 "/" is your current location
---> PBSZ 0
<--- 200 PBSZ=0
---> PROT P
<--- 200 Data protection level set to "private"
---> TYPE I
<--- 200 TYPE is now 8-bit binary
---> PWD
257 "/" is your current location
---> TYPE A
<--- 200 TYPE is now ASCII
---> PRET LIST
<--- 200 Ready to proceed
---> PASV
<--- 227 Entering Passive Mode (35,214,92,81,135,143)
---- Connecting data socket to (35.214.92.81) port 34703
---> LIST
<--- 150 Accepted data connection
WARNING: Certificate verification: hostname checking disabled
drwx--x--x    9 3099       u2692-91vi       4096 Jan 17 21:47 .
drwx--x--x    9 3099       u2692-91vi       4096 Jan 17 21:47 ..
-rw-r--r--    1 3099       u2692-91vi       3943 Jan 15 14:39 NoThumbnail.png
-rw-r--r--    1 3099       u2692-91vi       2644 Jan 15 14:39 allsky-favicon.png
-rw-r--r--    1 3099       u2692-91vi       1738 Jan 15 14:39 allsky-font.css
-rw-r--r--    1 3099       u2692-91vi      11765 Jan 15 14:39 allsky-logo.png
-rw-r--r--    1 3099       u2692-91vi       4476 Jan 15 14:39 allsky.css
-rw-r--r--    1 3099       u2692-91vi         42 Jan 15 14:39 analyticsTracking.js
-rw-r--r--    1 3099       u2692-91vi      52789 Jan 15 14:39 animate.min.css
-rw-r--r--    1 3099       u2692-91vi          1 Jan 15 14:42 checksums.txt
-rw-r--r--    1 3099       u2692-91vi       5444 Jan 17 18:33 configuration.json
-rw-r--r--    1 3099       u2692-91vi      25884 Jan 15 14:39 controller.js
-rw-r--r--    1 3099       u2692-91vi        173 Jan 17 18:05 data.json
drwxr-xr-x    2 3099       u2692-91vi       4096 Jan 17 18:04 fonts
-rw-r--r--    1 3099       u2692-91vi      14145 Jan 15 14:39 functions.php
-rw-r--r--    1 3099       u2692-91vi       1096 Jan 15 14:39 getForecast.php
-rw-r--r--    1 3099       u2692-91vi    1762312 Jan 17 21:46 image.jpg
-rw-r--r--    1 3099       u2692-91vi       9344 Jan 15 14:39 index.php
drwxr-xr-x    3 3099       u2692-91vi       4096 Jan 17 18:04 keograms
-rw-r--r--    1 3099       u2692-91vi      14784 Jan 15 14:39 loading.jpg
-rw-r--r--    1 3099       u2692-91vi     119183 Jan 15 14:39 moment.js
drwxr-xr-x    2 3099       u2692-91vi       4096 Jan 17 18:04 myFiles
-rw-r--r--    1 3099       u2692-91vi      61201 Jan 15 14:39 ng-lodash.min.js
-rw-r--r--    1 3099       u2692-91vi       7929 Jan 15 14:39 runCommands.php
-rw-r--r--    1 3099       u2692-91vi       1242 Jan 15 14:39 show_thumbnails.php
drwxr-xr-x    3 3099       u2692-91vi       4096 Jan 17 18:04 startrails
-rw-r--r--    1 3099       u2692-91vi         28 Jan 17 21:47 testUpload.sh.txt
drwxr-xr-x    3 3099       u2692-91vi       4096 Jan 17 18:04 videos
drwxr-xr-x    2 3099       u2692-91vi       4096 Jan 17 18:05 viewSettings
-rw-r--r--    1 3099       u2692-91vi       3132 Jan 15 14:39 viewSettings.php
drwxr-xr-x    4 3099       u2692-91vi       4096 Jan 17 18:04 virtualsky
<--- 226-Options: -a -l
<--- 226 31 matches total
---> PRET MLSD
<--- 200 Ready to proceed
---> PASV
<--- 227 Entering Passive Mode (35,214,92,81,135,60)
---- Connecting data socket to (35.214.92.81) port 34620
---> MLSD
<--- 150 Accepted data connection
WARNING: Certificate verification: hostname checking disabled
<--- 226 31 matches total
---> TYPE I
<--- 200 TYPE is now 8-bit binary
---> PRET STOR testUpload.sh-20675
<--- 200 Ready to proceed
---> PASV
<--- 227 Entering Passive Mode (35,214,92,81,135,143)
---- Connecting data socket to (35.214.92.81) port 34703
---> STOR testUpload.sh-20675
<--- 150 Accepted data connection
WARNING: Certificate verification: hostname checking disabled
<--- 226-File successfully transferred
<--- 226 0.031 seconds (measured here), 0.87 Kbytes per second
---> MFMT 20250117214724 testUpload.sh-20675
<--- 213 UTIME OK
---> DELE testUpload.sh.txt
<--- 250 Deleted testUpload.sh.txt
---> TYPE A
<--- 200 TYPE is now ASCII
---> PRET MLSD
<--- 200 Ready to proceed
---> PASV
<--- 227 Entering Passive Mode (35,214,92,81,135,43)
---- Connecting data socket to (35.214.92.81) port 34603
---> MLSD
<--- 150 Accepted data connection
WARNING: Certificate verification: hostname checking disabled
<--- 226 31 matches total
---> TYPE I
<--- 200 TYPE is now 8-bit binary
---> RNFR testUpload.sh-20675
<--- 350 RNFR accepted - file exists, ready for destination
---> RNTO testUpload.sh.txt
<--- 250 File successfully renamed or moved
---> QUIT
<--- 221-Goodbye. You uploaded 1 and downloaded 0 kbytes.
<--- 221 Logout.

Just to add, i have this current error in the WebUI -
ERROR: Image URL 'https://www.annanallsky.co.uk/image.jpg' does not appear to be a valid image.

[EricClaeys]

Thanks for the output. It helps.
The important entries are:

---> USER chris@annanallsky.co.uk
<--- 331 User chris@annanallsky.co.uk OK. Password required
---> PASS XXXX
...
<--- 257 "/" is your current location
...
---> STOR testUpload.sh-20675
<--- 226-File successfully transferred

This says the lftp command on your Pi logged in to the server as chris@annanallsky.co.uk and was put in the / directory.
It then listed the files in that directory, which are the Allsky Website files.
It then successfully transferred the test file.

This tells us the home directory of the chris@annanallsky.co.uk login is where all the Allsky Website files are.

We now need to determine what URL points to that directory.

Every URL I've tried now gives a "404 not found" error so either the URL is wrong or something is wrong with the way the server is processing web requests.

In your original post you included a couple screenshots from Filezilla.
What login did you use?
In Filezilla, please go to File -> Site Manager... and choose the site from the screenshots.
Clear out the User: field and take a screenshot and attach it to your reply.
Mine looks like:

Go to the "Advanced" tab. What's in the Default remote directory field?
Cancel out of the Site Manager so it doesn't save your deleted User.

---

What do you get if you go to `https://www.annanallsky.co.uk/image.jpg` in a browser? I get a "404 not found" message, which is why you are being told it's "not a valid image".

[EricClaeys]

I'm able to get to something at http://annanallsky.co.uk/allsky/, but it's only finding the index.php file. I don't think this is in the same directory lftp logs into - it looks like it's a copy of the file somewhere else on the server.

[shotscot]

Hi Eric, thanks for the detailed replies. I have logged into my siteground ftp account on FileZilla and went into site manager and it shows no sites? I cannot help but laugh to myself how badly I am at this stuff. Clearly my site should be in the site manager tab but there is nothing.

[shotscot]

after transferring my current connection to the site manager and going into the advance tab the Default remote directory field is blank?

[shotscot]

ive been on the website logs for possible errors, this is the current error appearing in the log - File does not exist: /home/u2692-91viwtdm0nhm/www/annanallsky.co.uk/public_html/

[shotscot]

well well well - I got onto my provider and asked them about the error and this was the response - yes, apparently the directory annanallsky.co.uk was deleted and that's why we are seeing this error now. He rebuilt the root and the site is loading now. I will need to go into my webUI setting and change the settings of where the images upload to as they are currently uploading into an allsky folder within the allsky folder

[EricClaeys]

Yea!!!!
When I go to https://www.annanallsky.co.uk your site appears with the "Loading..." image.
If you change the imageName setting in the remote_configuration.json file to image.jpg, you'll see the image. It's currently set to /current/tmp/image.jpg which is what a LOCAL Website uses.
That's a very common mistake and I'm not sure if it's user error or there's a bug in Allsky.

Would you please attach ~/allsky/config/logs/remoteWebsiteInstall.log ? It may give me a clue.

Any idea how the annanallsky.co.uk directory was deleted?

[EricClaeys]

When I go to https://www.annanallsky.co.uk/ it only shows the "Loading..." image.
When I go to http://www.annanallsky.co.uk/ it works.

This implies you have two copies of the "allsky" folder on the server and https goes to one of them and http goes to the other.

[shotscot]

For some reason it's under www.annanallsky.co.uk/index.php

[EricClaeys]

Copy the attached file to your server and rename it to runCommands.php. It should overwrite what's already there.
Then, create a file called commands.txt in the same directory on the server that contains:

set delete 0
info

Then go to these sites:
https://www.annanallsky.co.uk/runCommands.php
http://www.annanallsky.co.uk/runCommands.php

And let me know what you get. One should fail.

runCommands.php.txt

[shotscot]

all done, first screenshot is from HTTPS and second is from HTTP

[EricClaeys]

ok, this tells us:

1. https://www.annanallsky.co.uk/runCommands.php points to the public_html directory which is wrong.
2. http://www.annanallsky.co.uk/runCommands.php points somewhere else, possibly "public_html/allsky" ?
3. I also noticed that https://annanallsky.co.uk/runCommands.php (with no "www.") points to the same place as # 2.

Where did you update runCommands.php and where did you create the commands.txt file?

I suggest working with your service provider to:

1. Have everything point to the "allsky" FTP directory.
2. Have "www.annanallsky.co.uk" and "annanallsky.co.uk", or get rid of "www.annanallsky.co.uk". The "www." isn't used much anymore.
3. Have "http" requests "converted" into "https" requests. At a minimum, both should go to the same location.

[EricClaeys]

the Default remote directory field is blank?

That's fine - so is mine. It totally depends on how the FTP login was set up on the server.

I have several Pi's I use for testing, and each has its own login on my server, each with its own subdirectory.

My production Pi logs into the allsky directory.
My three test Pi's, called allsky-test, allsky-test2, and allsky-test3 log into their own directories.
In each case, the Default remote directory in FileZilla is empty.