[storage]: Front-end Attach / Detach Namespace Proto-Bufs

[mkalderon]

Need to add support for attaching / detaching controllers from a namespace.
We can take this. Just raising as issue in-case someone has prior thoughts on this ?
@glimchb

[glimchb]

isn't NVMeNamespaceCreate same as Attach ? we can rename it if that helps...

[glimchb]

I opened #178 as well for enable/disable NS...

[mkalderon]

Not quite. When a namespace is created it isn't attached to a controller. We're providing api to create subsystem/namespace/controllers etc.. so it's not like backend that you get this information and attachement information as well.
We need a way to attach between namespace and controller. Same namespace can also be attached to several controllers.

Having said that, looking at your additional issue with enable/disable - maybe it's somehow related and provides another way to make this attachment. in enable/disable are you referring to enable a namespace for a specific controller ?

[glimchb]

in enable/disable are you referring to enable a namespace for a specific controller ?

yes

[benlwalker]

A controller is a "view" or a "session" between a host and a subsystem. A subsystem contains the namespaces. I don't think it makes any sense in the NVMe model to attach a namespace to a controller. Can you outline the use case here? Is this for private namespaces? Or is this some dual port device case where the PCI endpoints see different namespaces?

Also, it does make sense for NVMe over fabrics to set up visibility rules based on the hostnqn via a call we don't have in the API yet. But I don't think that's what this issue is talking about.

[glimchb]

if we have 1 subsystem with 1 namespace and 10 controllers each on a separate PF/VF that we want to pass-through to 10 VMs...
how would you describe the operation of exposing/attaching Namespace to Controller (PF/VF) ?

[benlwalker]

The best approach is to create 1 subsystem containing 1 namespace for each PF/VF. You do not want the VMs to share state of any kind, and if they share a subsystem there is shared state in there, even if you filter out shared namespaces from view.

[mkalderon]

There may be cases where you would want the host "view" to match a backend, with shared namespaces etc...
I think the proto-bufs should provide this functionality, best approach limits the administrator.

[benlwalker]

Unfortunately, that's in direct opposition to the defined OPI view of separate "front", "middle", and "back" ends. The front-end is entirely virtualized and emulated, and back/middle end volumes are added as namespaces. The front-end and the host never see anything about back-end subsystems or controllers, and in fact the back-end may not even be NVMe. Things like NVMe admin commands that come into the front-end are not simply passed through to some "middle" or "back" end object because there isn't middle/back-end object that corresponds to a controller/subsystem - only volumes.

In order to think through solutions, I think we'll need a lot more detail on the motivating use case for the host "view" to match some backend. Are you able to provide that?

[glimchb]

The best approach is to create 1 subsystem containing 1 namespace for each PF/VF. You do not want the VMs to share state of any kind, and if they share a subsystem there is shared state in there, even if you filter out shared namespaces from view.

I'm not arguing the best approach. I think API should allow and not limit if user wants to share subsystem. Even if we don't see today the use case, it is reasonable to allow this

[sburla-marvell]

A subsystem can have namespaces that are not attached to any controller and similarly a subsystem can have a namespace that is attached to multiple controllers. This is all still front end view(nothing do with backend). This means that there should be an ability to attach/detach a namespace to/from a controller. There is also an ability to attach or detach namespaces from nvme controllers using async event at runtime (namespace attribute changed) . This event causes host to reenumerate the namespaces. This is a feature most of our customers use.

[benlwalker]

I think there may be some mixing of very different concepts going on here. These API definitions are not doing the equivalent thing of the NVMe Namespace Attach/Detach commands. Those NVMe commands are looking at a list of namespaces already attached to a subsystem, and then adding a reference, via an NSID, to a controller's view. That's not OPI's job - OPI is the thing that put the namespaces into the subsystem to beginwith.

So again, can you clarify what the actual use case is for this feature? An example flow and justification of some kind, even very high level? I'm certainly not advocating that we don't support all sorts of dynamic hot insert/removal of namespaces with all of the appropriate async events, nor am I advocating that we preclude support for any specific NVMe namespace management commands. OPI is definitely going to have to support all of that stuff. I'm really focused just on the need to set up mappings between controllers and namespaces during provisioning, based on the controller identity itself (as opposed to based on a hostnqn), and only in the front-end for NVMe devices. It's a very narrow thing I'm advocating against currently.

The reason I'd like to avoid adding support for this is relatively straightforward:

• Anyone sharing subsystems between tenants has to be very careful not to expose shared state. You have to disable NSSR, likely disable namespace management (because the NSIDs are global and you can run into conflicts), and possibly other things. Especially dangerous are new things added to the spec later on.
• There's an easy alternative design (make more subsystems) that's safer and better.
• We should try to avoid adding APIs just because the spec supports it or we'll end up with 100+ calls and too much complexity. We should add calls whenever there's a real use case for an xPU that someone presents. I don't even propose we be very strict or scrutinize the use cases all that closely - just some reasonable justification is probably a good enough bar for me.

[sburla-marvell]

This is not about the optional Namespace management support from the host side. I am not actually we should let the host do namespace management in this case anyway, as we want DPU to be in control. This is about how the DPU provisioning new namespaces to a controller at runtime. Unfortunately, the only way the spec supports this is through the host side (using namespace management commands). But the same mechanism lets DPU side also do this by issuing an async event. The high level use case for this feature is provide additional storage to a tenant without the need to burn another VF/PF resource.

[benlwalker]

Can't what you're describing be accomplished by adding a namespace to a subsystem instead of adding it to the controller? I believe it can if the default behavior of a controller is to automatically see all namespaces in the subsystem.

This is really the core of this discussion. What you outlined above can be done with the calls that already exist. What's the new/extra functionality needed that warrants an extra step to attach/detach from specific controllers?