io.Copy gives permission denied for get operation and io.ReadAll gives Bad Message for 190kb+ files

[aashakabra]

Hello,

I am doing a simple get operation on a file available on SFTP, but I am not writing the content in a file, I want to hold it in memory and want to pass it as output for futher transformation.

Below code works perfectly fine on our test SFTP server (openssh) but it fails with error on customer's SFTP server (which is very much older openssh and there are permission restrictions , I will list them out in detail).

OS details of sftp server  - Red Hat Enterprise Linux Server release 7.9 (Maipo)

Type of sftp server - MessageWay SFTP Interface Version 6.1, sftp server type : openSSH
OpenBSD: sshd_config,v 1.73
sftp> version : SFTP protocol version 3

Permissions on parent directory & file -  View message , upload message and download message (with other required permissions)
Permissions assigned to user - View message , upload message and download message (with other required permissions)
Restrictions applied on sftp server -  Whitelist public ip to establish connection towards access point.

The code implementation I have is-

// ProcessDataGetOperation downloads base64 binary encoded file content from SFTP server
func ProcessDataGetOperation(sc *sftp.Client, inputParams *InputParam, binary bool, activityName string) (*Output, error) {
	srcFile, err := sc.OpenFile(inputParams.RemoteFileName, (os.O_RDONLY))
	if err != nil {
		return nil, GetError(RemoteFileOpenError, activityName, err.Error())
	}
	defer srcFile.Close()

	// read the content of file hosted on sftp server
	dataBytes := new(bytes.Buffer)
	bytes, err := io.Copy(dataBytes, srcFile)

	//bytes, err := io.ReadAll(srcFile)
	if err != nil {
		return nil, GetError(GetOperationError, activityName, err.Error())
	}
	var fileInfo fileInfo
	fileInfo.FileName = inputParams.RemoteFileName
	fileInfo.NumberOfBytes = bytes

	var output Output
	if binary {
		output.BinaryData = base64.StdEncoding.EncodeToString(dataBytes.Bytes())
	} else {
		output.ASCIIData = dataBytes.String()
	}
	output.FileTransferred = append(output.FileTransferred, &fileInfo)

	return &output, nil
}

With this code we are getting below error on customer SFTP server- Permission denied
The permission detail that customer has highlighted to us is - The file on sftp server has read permission. The directory has read & execute permissions.
However Customer is able to drag and download the file using WinSCP.

I tried the same permission on my local openssh server but I did not face any issue.

Now , when I replace io.Copy with io.ReadAll - the code works fine but for file size less than 190kb. io.ReadAll fails for filesize greater than 190kb and error received is "Bad Message"(SSH_FX_BAD_MESSAGE)

What could be the possible reason to run into this issue? and can you suggest some solution alternate to io.Copy which could possibly fix the issue?

[aashakabra]

Hello @puellanivis , Can you please share some inputs around this?

Customer is able to get the content of a file on SFTP server for their another test instance. It is just above described SFTP server that they run into issue, which seems to be their main SFTP server.

Can you please suggest what could be the possible reason for this error and suggest alternate that I can try instead of io.Copy or io.ReadAll ?

[drakkan]

@aashakabra please avoid mentioning people directly. I understand the issue is urgent for you but you are using open source software for free, you can't expect help as if we were your colleagues working for the same company, we try to help when possible, but we also have our commitments, you opened this issue less than 12 hours ago. Thanks for understanding

[aashakabra]

I understand. Sorry for the inconvenience.

[puellanivis]

What options are you specifying in your sftp.NewClient call?

[aashakabra]

I am not specifying any options.

// Create an SFTP client on top of the SSH connection
logCache.Infof("Creating new SFTP client")
client, err := sftp.NewClient(conn)
if err != nil {
	return nil, fmt.Errorf("failed to create SFTP client: %s", err.Error())
}

I close the client connection in another call back method Stop which is called when engine is stopped.

[puellanivis]

Can you try using sftp.UseConcurrentReads(false) ? I suspect the permission denied is originating from trying to stat the file. That should make the io.Copy and io.ReadAll equivalent. (That is, eventually produce a SSH_FX_BAD_MESSAGE)

[aashakabra]

Thankyou for your reply.

Okay. Should I also add - sftp.UseFstat(true) along with sftp.UseConcurrentReads(false) ?

As per above ping if this will further lead to SSH_X_BAD_MESSAGE - what can be probable solution for this? as on this specific SFTP server they are facing this error for file above 190kb.

In past during development we faced size issue with io.ReadAll (no error but sftp get file content + sftp put file content -> the file size dint match. And then we decided to use io.Copy() instead which solved this issue for bigger file size. We never ran into error. It is this customer SFTP server where they are getting error further with 190kb+ file size - SSH_FX_BAD_MESSAGE

[puellanivis]

I don’t know if the UseFstat(true) will help, but it should be tried in alternative to the UseConcurrentReads(false) as they solve different issues. It could be that fstat works but stat doesn’t, but it’s also probable that neither might work. So UseConcurrentReads(false) will turn off even attempting either.

I’m not entirely sure why you’re getting SSH_FX_BAD_MESSAGE, and was just code diving to see if I could see anything. We definitely don’t generate it, so it’s coming from the service on the other side. I’m unsure what it could possibly be complaining about, but I’m thinking it might be from attempting a too large read maybe. But we’re defaulting to the smallest size that all servers are supposed to support.

It’s interesting that io.ReadAll was giving inconsistent file sizes. 🤔 That’s probably diagnostically significant, but I’m not sure I know what could be wrong.

PS: Which version of this library are you using?

[aashakabra]

I am using version github.com/pkg/sftp v1.13.6

Let me set the flag UseConcurrentReads(false) and give it a try on customer envt.
Can you confirm, if this will stop us from reading two different files concurrently?

[puellanivis]

This does not stop you from reading two different files concurrently. It stops the library from concurrently issuing read requests. It’s kind of a last ditch effort to provide the highest compatibility, but also the slowest speeds.