SMB for Samba > 4.10.18 (possibly) not working

[mmattel]

Steps to reproduce

1. Uninstall smbclient installed from Ubuntu 20.04
2. Compile a newer version > 4.10.18 (does not matter which one, valid up to last stable or samba-master)
3. Check SMB links

Expected behaviour

Existing mounts should be green and new mounts be creatable

Actual behaviour

Existing mounts always red, new mounts can´t be created (red)

Following scenario and testing:
Post upgrading from U18.04 to 20.04, I saw that my SMB shares were no longer accessible. This is due to changes in Samba and an older fileserver. Looking for a solution, I tested following scenarious, where all versions need a samba compilation from source:

1. Samba 4.10.18 (the last one before a bigger change, not patched) - works
2. Samba 4.11.6 (the one installed from OS, patches applied) - does not work
3. Samba 4.13.3 (the lastest stable version, patches applied) - does not work
4. Samba Master (git clone, patches applied) - does not work

Note I regarding patching:
See a bug I filed and the patches provided at https://bugzilla.samba.org/show_bug.cgi?id=14607
For versions where I applied the patches, I can connect in all cases flawless with smbclient -L filer -U domain/user and run successfully the icewind php example.php test.

Note II regarding patching:
I can´t test if the Ubuntu provided smbclient version would work, because I do not have a "younger" SMB fileserver...

Note regarding pecl smbclient:
No differences in behaviour if installed or not!

Note regarding testing:
I cloned https://github.com/icewind1991/SMB and did a check if I can connect via php example.php. I was able to properly connect in ALL cases

Why not taking the "works" version 4.10.18
This ist the last version that allows NT1 compatibility. Newer fileservers do not longer support NT1 - means you are in a deadlock when having both. The patch (and later when included in stable) solves this problem, native SMB2 allowed!

Note regading the Samba versions of Ubuntu
The Ubuntu shipped version of smbclient (Samba) (Ubuntu updates very seldom...).
U18.04: 4.7.6 (2018.03)
U20.04: 4.11.6 (2020.01)
Latest Stable Samba: 4.13.3 (2020.12)
Last version before big change (4.10->4.11): 4.10.18 (2020.9)

My findings:
Because I can connect with smbclient and php example.php, the source issue must be in ownCloud. I stronly believe, that it is a minor thing - else the tests before would not work.

Server configuration

Operating system: Ubuntu 20.04

Web server: nginx

Database: mariadb

PHP version: 7.4

ownCloud version: 10.6

Updated from an older ownCloud or fresh install: updated

Where did you install ownCloud from: tar

Signing status (ownCloud 9.0 and above):

all ok

The content of config/config.php:

{
    "system": {
        "instanceid": "oct7bkicorxf",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "url"
        ],
        "datadirectory": "\/var\/www\/owncloud\/data",
        "overwrite.cli.url": "https:\/\/url",
        "htaccess.RewriteBase": "\/",
        "dbtype": "mysql",
        "version": "10.6.0.5",
        "dbname": "owncloud",
        "dbhost": "localhost",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "logtimezone": "Europe\/Vienna",
        "installed": true,
        "filelocking.enabled": "true",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "\/var\/run\/redis\/redis-server.sock",
            "port": 0,
            "timeout": 0,
            "dbindex": 0
        },
        "filesystem_check_changes": 1,
        "loglevel": 2,
        "mail_smtpmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpsecure": "tls",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "587",
        "maintenance": false,
        "theme": "",
        "preview_max_scale_factor": 1,
        "excluded_directories": [
            ".snapshot",
            "~snapshot"
        ],
        "upgrade.automatic-app-update": true,
        "quota_include_external_storage": false,
        "singleuser": false,
        "apps_paths": [
            {
                "path": "\/var\/www\/owncloud\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/www\/owncloud\/apps-external",
                "url": "\/apps-external",
                "writable": true
            }
        ],
        "show_server_hostname": false,
        "log_rotate_size": "20971520"
    }
}

List of activated apps:

Enabled:
  - activity: 2.6.0
  - audioplayer: 2.8.4
  - comments: 0.3.0
  - configreport: 0.2.0
  - dav: 0.6.0
  - federatedfilesharing: 0.5.0
  - federation: 0.1.0
  - files: 1.5.2
  - files_clipboard: 1.0.3
  - files_external: 0.7.1
  - files_external_dropbox: 1.2.0
  - files_external_ftp: 0.2.1
  - files_mediaviewer: 1.0.3
  - files_pdfviewer: 0.11.2
  - files_sharing: 0.14.0
  - files_texteditor: 2.3.0
  - files_trashbin: 0.9.1
  - files_versions: 1.3.0
  - firstrunwizard: 1.2.0
  - market: 0.6.0
  - notifications: 0.5.2
  - provisioning_api: 0.5.0
  - systemtags: 0.3.0
  - templateeditor: 0.4.0
  - updatenotification: 0.2.1
Disabled:
  - encryption
  - external
  - guests
  - user_external

Are you using external storage, if yes which one: local/smb/gd.

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Logs

Web server error log

No errors

ownCloud log (data/owncloud.log)

I tried with loglevel=0, but there is no usable output ???

No relevant data

@pmaier1 @micbar

[mmattel]

Note: testing is a mess. I have created a U20.04 headless VM with the working compiled version 4.10.18. But because I can´t switch all the time by recompiling the samba environemt, I cloned the server and use there samba 4.13.3. Hope that I can sort it out, but if one has an idea, pls drop a comment!

[mmattel]

@micbar maybe you could assist if you let me know where I can find the code which checks if a storage is available (green) or not (red).

[mmattel]

#38272 (Improve displaying errors in the extexternal storage settings ui)
@janackermann this would help to see the response why it fails...

[micbar]

@jvillafanez @cdamken Please do some research about incompatibilities.

@mmattel We all know, that the official smbclient is a very bad piece of software engineering.

We are using alternatives in our own instances.

[cdamken]

Hi @mmattel so far as a workaround for 20.04 and the php-smbclient I saw that the ondej repository helps perfect:
https://launchpad.net/~ondrej/+archive/ubuntu/php/

Have you tried?

[mmattel]

@cdamken @micbar ondrej/php is for php and default extensions. But this is not the point. You have to sort things out:

1. The U20.04 smbclient has as basis Samba, which is delivered in version 4.11.6. and necessary for any access to SMB file servers.

2. If you use libsmbclient for php, which is mandatory for extended SMB access - in particular for the enterprise WND app, you must use the PECL extension. This compiles a php extension based on the installed samba release and is therefore also bound to! Means, you can´t share the same smbclient.so for php7.2, 7.3, 7.4, 8 ect!

3. The icewind repo used, takes care which kind of php-smb access is available and handles this in the backend quite nice. If pecl/smbclient is installed, it uses that one, else the standard one.

4. As written above, test made with smbclient and icewind/smb test worked flawless - means, it is an ownCloud point. I strongly believe a return code/message from checking accessability has changed a bit and is not covered.

5. The issue is bad in particular, because Samba has rejected NT1 authentication which was necessary in some cases. (The usage was defined with a parameter in smb.conf.) Not only because a smb fileserver was maybe an older model, but also because samba was gold plating some things where Microsoft was more tolerant. See the bug on bugzilla.samba above (including a fix).

Believe me, I have extensively tested over many days +12h a day trying various scenarious. It is hitting me personally if that does not get fixed...

Please also see the docs PR I have created which manages the manual installation scenario for Ubuntu 20.04
You will find a own sub chapter focussing on Samba.

One can say that this may be a neglectable issue - where I strongly disagree. Many companies are running non Microsoft enterprise file servers successfully for a long time, as they are very reliable (dont want to do a praise on NetApp, but it just happened...)

@micbar this: #38278 (comment) would help me a lot supporting finding the cause. I have already two identical VM´s running where I can test working and non working scenarious in parallel.

We are using alternatives in our own instances.

Not really, see: https://doc.owncloud.org/server/10.6/admin_manual/enterprise/external_storage/windows-network-drive_configuration.html#installation Basis is Ubuntu-smbclient + Pecl-smbclient (eduardok) - means you are affected.

[mmattel]

Update:
Test environment, all Ubuntu 20.04, PHP 7.4 native (no ondrej), mariadb:

• Production environment (P1) was upgraded from U18.04 and runs 10.6 production and uses a compiled unpatched Samba version 4.10.18
• 2 identical VM (second server has been cloned). ownCloud base is core-master, fresh installed.
  Server 1 (S1) has installed a compiled unpatched Samba version 4.10.18
  Server 2 (S2) has installed a compiled patched Samba version 4.13.3

Note:
There is a change from 10.6 official to git master. See PR #38151 (Alternative SMB external storage implementation where all users of one share use the same storage id. User specific access is filtered in a cache wrapper by stating the files on SMB)

Test results:

• P1: Using and adding SMB shares is, as we are used to - all fine ✔️
• S1: Adding an SMB mount is possible. You can see and access the mount as usual.
  BUT post changing the password to a non working one, instead the mount gets a red square, it stays green ❌
• S2: Adding an SMB mount not possible, the mount always has a red square ❌

Adding @DeepDiver1975 because of the merged PR #38151

@micbar, could we have a chat on talk?

[micbar]

@micbar, could we have a chat on talk?

done

[mmattel]

I found the root cause with network traces!

Test Setup:

• 2 identical servers (S1 and S2) with a compiled samba version (master) containing a patch that has been accepted and is soon part of the public available stable releases of samba (see bugzilla linka bove). This patch is necessary fo access filer with smbclient not using NT1.
  Ubuntu 20.04, php7.4, mariadb, pecl smbclient, owncloud master.
  Server 2 is a clone from server one and has samba setup as stand alone server.
• an external non samba, non Microsoft SMB file server. The name is filer

First Test:

• S1 connects via smbclient and the icwind/smb test to S2
  OK
• S1 connects via smbclient and the icwind/smb test to filer
  OK

Second Test:

• S1 connects via ownCloud SMB mount to S2
  OK
• S1 connects via ownCloud SMB mount to filer
  FAIL

Third Test:

1. I did a network trace where I tried to create a mount via oC (not working)
2. I did a network trace where I tried to do a directory listing via Icewind/SMB (works)

Comparing the traces showed, that via oC, there is a request called SMB2_FILE_ALL_INFO
This request returns a STATUS_NOT_SUPPORTED response. This is the root cause why oC fails with stat but the Icewind/SMB tests succeeds - as that test does not do a stat when doing a dir listing.

Result:
There is nothing wrong with samba or oC. It is a matter of the storage backend which returns an unsupported response... 🤦🏻‍♂️ Anyway, this can affect your customer base.

This issue shows an underlaying general issue when NT1 is needed

How to get out of the mess (for all customers running into this issue):

This is not limited to the particular fileserver used, but for all customers who have a fileserver which need the NT1 option as described below - and that are many...

1. you mandatory need a samba version on the server which runs ownCloud, which is capable using NT1 as client. Note, that you cant set the client max protocol=NT1 any longer in smb.conf in actual samba versions! This topic is already addressed in the PR in docs. ✔️
2. this is something where ownCloud would need a small change. Two new fields in the mount settings for a smb option for smbclient to restrict to a particular client protocol. The option for the defaut smbclient is:
   --option='client min protocol=<protocol_min>' and --option='client max protocol=<protocol_max>'. eduardok as recently released a smbclient version which is capable defining min/max client protocol for the php library. There is also a issue filed at the Icewind repo to enable parametrisation. To implement a solution at ownCloud, we have to wait until the Icewind/SMB library offers that parametrisation.

Why such a parameter in the mount option?

• because in the mount option, it is restricted to the mount only,
• whereas in smb.conf, it would be for ALL mounts
• that would conflict other SMB servers connected which would then not be accessible anymore!
• client max protocol=NT1 is in actual versions of smb.conf not longer allowed but allowed as option
• Danger of deadlock

[mmattel]

Info: Icewind just released SMB 3.3.0, see https://github.com/icewind1991/SMB/releases/tag/v3.3.0
where you can now set the min/max smb protocol version, example:

$options = new Options();
$options->setMinProtocol(IOptions::PROTOCOL_SMB2);
$options->setMaxProtocol(IOptions::PROTOCOL_SMB3);
$serverFactory = new ServerFactory($options);

to get a list of protocols possible to set, see: https://github.com/icewind1991/SMB/blob/4d078bd0bd2910c80d549272c7e474aae6aabe02/src/IOptions.php#L25