Codecov Report

Attention: Patch coverage is 78.16092% with 38 lines in your changes missing coverage. Please review.

Project coverage is 70.46%. Comparing base (96c06be) to head (5faec1c).
Report is 224 commits behind head on main.

@@           Coverage Diff            @@
##             main    #5669    +/-   ##
========================================
  Coverage   70.45%   70.46%            
========================================
  Files         217      219     +2     
  Lines       36016    36177   +161     
========================================
+ Hits        25376    25493   +117     
- Misses       9135     9164    +29     
- Partials     1505     1520    +15     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

1. EEP is probably not the best place for this, since a bad module can take out the entire proxy, not just the route
2. Can you share a workflow of how a user will fetch a module into the pod

Some options that I can think of:

• use a docker image to download dynamic modules. This docker image can be deployed as a side car in the Envoy pod and the lib can be loaded to Envoy via a shared volume.
• use a customized envoy docker image provided by users. The libs for dynamic modules are already loaded to that image.

The first approach can solve the same issues for golang filter, so I'm +1 for this.

Second one needs some prerequisites for features like golang/dynamic module, use envoyproxy to customize the image before using EEP for dm/go filter

Some options that I can think of:

• use a docker image to download dynamic modules. This docker image can be deployed as a side car in the Envoy pod and the lib can be loaded to Envoy via a shared volume.
• use a customized envoy docker image provided by users. The libs for dynamic modules are already loaded to that image.

this experience is not seamless, so maybe lets first document this in docs post v1.4, and then get feedback, and then we can eliminate the need for the EnvoyPatchPolicy with an API field (maybe in EnvoyProxy) in the future

FYI: Dynamic modules require the exact version match (at least for now), so for example, the modules working with Envoy v1.50.0 are not guaranteed to work with v1.51.0 (if the compatibility check fails, then HTTP filter configuration xds request will be rejected). So, coupling Envoy images would be the safest way; either using the Envoy container itself or with init container downloading (with some version argument). I believe the same restriction applies to golang filter.

So either way, the life cycle of downloading into Envoy container should match the one of Envoy container itself. that's my 2p

@mathetake Thanks for clarifying how dynamic modules need to match the version of Enovy Proxy.

This would be a problem for upgrade - since the ABI version is defined in the abi_version.h, EG can't check if a dynamic module will still be compatible or not after upgrade. Users can only find out after something is broken.

Is there a way for EG to detect an incompatibility between Envoy and a dynamic module, and then surfaced that issue in the status of EG resources?

There's no mechanism to check that without actually loading Envoy. Like i suggested, the current best way is to tie the Envoy container with modules as in the examples or EG introduce some module management init container like the container receives the Envoy version and downloads the specific dynamic modules for that version before Envoy running. To be clear, the same problem exists for golang filter already, and nothing specific to dynamic modules.

so i think we can go with this as-is leaving the lifecycle of dynamic modules to users for now, and then we can later add the "managed" installation of remote dynamic modules like in EnvoyProxy API or EnvoyGateway API maybe

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-ai-gateway
spec:
  dynamic_modules:
  - name: mymodule
    location:
      url_template: https://foo.bar.com/path/to/${ENVOY_VERSION}/libmymodule.so

yah above approach looks good @mathetake, this is maybe hard to predict, but any guesstimate on average and max sizes of the module ?

but any guesstimate on average and max sizes of the module ?

shared library is almost the same as normal application binary, so the question would be how large a normal application would be; a few MB at least for Rust and Go, and tens of MB if it contains debug info:

~/dynamic-modules-examples/rust$ ls -hl target/debug/librust_module.so 
-rwxr-xr-x 2 mathetake mathetake 23M Apr 16 01:58 target/debug/librust_module.so

so i think we can go with this as-is leaving the lifecycle of dynamic modules to users for now, and then we can later add the "managed" installation of remote dynamic modules like in EnvoyProxy API or EnvoyGateway API maybe

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-ai-gateway
spec:
  dynamic_modules:
  - name: mymodule
    location:
      url_template: https://foo.bar.com/path/to/${ENVOY_VERSION}/libmymodule.so

Why not just directly add this to DM fields EEP API? Like what we did in WASM

so i think we can go with this as-is leaving the lifecycle of dynamic modules to users for now, and then we can later add the "managed" installation of remote dynamic modules like in EnvoyProxy API or EnvoyGateway API maybe

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: envoy-ai-gateway
spec:
  dynamic_modules:
  - name: mymodule
    location:
      url_template: https://foo.bar.com/path/to/${ENVOY_VERSION}/libmymodule.so

Why not just directly add this to DM fields EEP API? Like what we did in WASM

That will allow a route owner to access another route or even the listener violating its scope of extension

So currently a dynamic module's capability is as same as Lua and Wasm so I don't have that concern right now. However yes indeed it can expand to have more broader capability plus a module won't be limited to HTTP filter (might end up having Load balancer extension etc). So loading/distribution of modules should be a separate API vs EEP which purely should focus on the configuration of HTTP filters. That's my 2c

That will allow a route owner to access another route or even the listener violating its scope of extension

Get it, thanks for clarifying @arkodg

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. Please feel free to give a status update now, ping for review, when it's ready. Thank you for your contributions!

This pull request has been automatically marked as stale because it has not had activity in the last 30 days. Please feel free to give a status update now, ping for review, when it's ready. Thank you for your contributions!