Azure Blob Storage, Azure File Share and SharePoint Online Connector Apps

[IceOnly]

This PR contains three new connector apps, to connect Azure Blob Storage, Azure File Share and SharePoint Online with the New File System Module in the System App.

File System Module PR:
microsoft/BCApps#663

Here are some Screenshots:

Fixes #22691
Fixes AB#559148

[pri-kise]

Some general thoughts:
I think we need some kind of path serparated from the name added to the file account content.
It's often very important to know the path.

Additionally we might need some additional kind of top level selection before List Directories:
I'm thinking about a webservice for local file system access, there we should be able to select the Drive "C:" or "D:".
For Azure File Share we should be able to select the File Share on a top level before searching directories and files.

Further more if I take a look at the current sharepoint client. I have no idea how we should map this to the current provided
interface "File Connector".
https://github.com/microsoft/ALAppExtensions/blob/main/Modules/System/SharePoint/src/SharePointClient.Codeunit.al

Sometimes a path isn't enough and we might need more information to download a file.
Maybe this info could be added to the table 70005 "File Account Content" table and the interface "File Connector" could accept a "File Account Content" as parameter instead of a path....

Maybe someone else with more experience has some ideas how this access can be unified...

[IceOnly]

Some general thoughts: I think we need some kind of path serparated from the name added to the file account content. It's often very important to know the path.

Additionally we might need some additional kind of top level selection before List Directories: I'm thinking about a webservice for local file system access, there we should be able to select the Drive "C:" or "D:". For Azure File Share we should be able to select the File Share on a top level before searching directories and files.

Further more if I take a look at the current sharepoint client. I have no idea how we should map this to the current provided interface "File Connector". https://github.com/microsoft/ALAppExtensions/blob/main/Modules/System/SharePoint/src/SharePointClient.Codeunit.al

Sometimes a path isn't enough and we might need more information to download a file. Maybe this info could be added to the table 70005 "File Account Content" table and the interface "File Connector" could accept a "File Account Content" as parameter instead of a path....

Maybe someone else with more experience has some ideas how this access can be unified...

Why not listing C: and D: as directory on the top level?
The path is part of the Fiel Acount Content table. It is splitted in Parent Folder and Name.

However, I am not sure if the module should define the path separator. So all Services uses the same path structure and the implentation need to translate if needed.

[PeterConijn]

Looks cool! I have a few small performance and security concerns, which I have addressed, and some minor other stuff, but nothing major. I'm excited for this.

[JesperSchulz]

Let me amplify this on Twitter/X to get your initial question answered: "before I go round the whole thing, I'm interested in whether there is any interest in the module at all". Let's see what people have to say! 😊

[tinfister]

I support every effort to improve file handling.

[dNsl9r]

Wonderful! Please merge this 👍🏻

[miljance]

Easily overlooked feature but a needed one. Huge development effort that should not be thrown away.
Perhaps I will find some time to invest in contribution from my side but cannot make promises at the moment.

[TheDoubleH]

Bravo! Love seeing more 'helper' features/functions that eventually will benefit all of us!

[JesperSchulz]

@IceOnly, this PR seems to have stagnated a little (which is perfectly fine). I was just wondering if you want to push this forward at some point, or if you've changed your mind / cannot find the time? There seems to be plenty of interest for this module in the community!

[IceOnly]

@IceOnly, this PR seems to have stagnated a little (which is perfectly fine). I was just wondering if you want to push this forward at some point, or if you've changed your mind / cannot find the time? There seems to be plenty of interest for this module in the community!

It is the time. I will try to find some free time to make the last changes.

This are my open points:

• I am struggling with the wist from pri-kise to split the Main Interface in logic interfaces like one for File, one for Directory and one for Account access. I like the Idea to implement one Interface and you are done.
• I will try to implement a realy simple pagination interface. The Most file services uses realy different methods for getting the next batch. So i will only alow the implemntation to result a pagination codeunit that can store the inforamtion it needs to get the next batch. How big a batch is can only be defined by the implementation.

[JesperSchulz]

Feel free to do it at your own pace. I didn't mean to stress you. I was just curious if you still intend to continue work. If you do, we'll just leave this PR open. No problem whatsoever.

[tinestaric]

Thought; could we add ResponseReasonPhrase to the message if ErrorMessage is empty?

Two cases where I came across this were when I misconfigured the Sharepoint URLs, and I was getting "An error occurred." with no details, but when debugging I saw it returned a 403.

Second one was when I tried to create a file in a subfolder that doesn't exist. Error Message is empty, but 404 Not Found was still in the diagnostics.

On the one hand, I know that's not really informative for the user, but otherwise I always have to debug, to understand what went wrong...

[JesperSchulz]

Could also be an improvement we add later in a follow-up PR. Doesn't strike me as critical to get this PR on the road. But it's a good idea for sure. We should probably use ErrorInfo to shield the normal user from such details, while providing these important pointers in a different dimension of the error message.

[JesperSchulz]

Coming along nicely! I couldn't find anything serious, but would suggest to revisit the permissions.

[JesperSchulz]

Why not use inherent permissions/entitlements on most of these objects. Not ever would a system admin be interested in controlling the execute permissions of e.g. an Impl. codeunit. Let's only have the permissions configurable, which an admin actually could care about!

[JesperSchulz]

Could also be an improvement we add later in a follow-up PR. Doesn't strike me as critical to get this PR on the road. But it's a good idea for sure. We should probably use ErrorInfo to shield the normal user from such details, while providing these important pointers in a different dimension of the error message.

[JesperSchulz]

@IceOnly, we got an issue to fix to make CI work again. ETA is tomorrow or Wednesday. Once we're done, could you pull from main and see if your PR passes too? I'll ping you when we have a successful build again.

[tinestaric]

I tried using the connectors with our apps, and I must again say that this is a great addition!
One thing that I've noticed that error handling could be improved, but as we agreed earlier, that can be addressed in a later PR.

One final thing I'd try to get sorted out for this PR is to make the Blob Storage and File Share tables public as well. The use case is that I'd store files in either of those storage types from BC and then send the file URI to an external service that can pull it (File based integrations). Right now I have to keep Storage Account, Container, and File Share Name in a separate record to build up a URI.

[IceOnly]

I tried using the connectors with our apps, and I must again say that this is a great addition! One thing that I've noticed that error handling could be improved, but as we agreed earlier, that can be addressed in a later PR.

One final thing I'd try to get sorted out for this PR is to make the Blob Storage and File Share tables public as well. The use case is that I'd store files in either of those storage types from BC and then send the file URI to an external service that can pull it (File based integrations). Right now I have to keep Storage Account, Container, and File Share Name in a separate record to build up a URI.

I made the other setup tables global as well.

[tinestaric]

Found one more case that would probably need to be dealt with within the connector app. Copying a production environment to a sandbox (mabye also copy companies?). We don't want changes in a sandbox environment to be reflected in production at all, so during the copy part, the settings or at the very least the credentials should be dropped.

[IceOnly]

Found one more case that would probably need to be dealt with within the connector app. Copying a production environment to a sandbox (mabye also copy companies?). We don't want changes in a sandbox environment to be reflected in production at all, so during the copy part, the settings or at the very least the credentials should be dropped.

Good idea. I added a disabled field to all account tables. It will be set automaticly on enviroment copy.

[JesperSchulz]

I am super eager to get this one the road. Community, could we get some sign-offs? Once we've got those (and we've fixed our uptake issues 🤦‍♂️), we'll move this PR into the next stage! My goal is to have this completed before end of this month 😊

[pri-kise]

I added some comments.
You don't necessary need to change everything, but I wanted to give you my hones opinion on the current PR.

nit: if you have the time or use the known VS Code extensions, then you could sort the properties of the pages.

[pri-kise]

@JesperSchulz do we already have to prepare for this upcoming change when the Images are removed from the MediaResources table?
microsoft/BCApps#2391

[JesperSchulz]

We should definitely try to do things "the new way" where possible.

[pri-kise]

Whats' the use for permission Sets that are Public but not assignable?
Is this really on purpose?

[pri-kise]

Will this permissionset extension allow partners to set this up as delegated admin agent? (@JesperSchulz could you maybe forward this to someone with knowledge on this topic?)