Skip to content

Commit f59e8bf

Browse files
authored
Update papers/pasteur_labs/main.md
component -> pipeline
1 parent 5cabc1b commit f59e8bf

1 file changed

Lines changed: 1 addition & 1 deletion

File tree

papers/pasteur_labs/main.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ Building pipelines that propagate gradients effortlessly across components intro
2020
To the best of our knowledge, modern scientific computing tools do not provide native support for service-level AD, limiting all autodiff processing to a single component. However it is certainly possible to combine multiple tools to create AD-capable components that can communicate over network. Typically such solution would involve an AD-native framework (e.g., JAX, TensorFlow, PyTorch, Julia) and a service layer that exposes gradient-specific endpoints. Service layer can be implemented using general-purpose web application frameworks (e.g. Flask or Django) or specialised serving solutions (e.g. Ray Serve, AWS SageMaker, Goolge Vertex AI). Crucially, there is no standardized gradient APIs, no REST/gRPC protocols to follow, so all of these solutions for service-level AD require large effort to define AD-relevant endpoints, input and output schemas. Additionally, it is necessary to factor in orchestration of distributed gradient calculation jobs and workflows.
2121

2222
We are not the first to recognise this gap, and there are several examples in academic literature of proposed solutions for distributed AD. @baker2021peering proposed a technique for dstributed training of deep neural networks that leverages
23-
the outer-product structure of the gradient of a network layer. @rush2024federated adopted AD to the context of federated learning of ML models. @tang2023auto considered the problem of differentiating computations expressed in relational databases. Notwithstanding these selected examples, even experimental support for component-level AD is nascent, and it is remarkable that recent surveys on the state of AD do not consider it being an important research direction (@van2018automatic, @baydin2018automatic).
23+
the outer-product structure of the gradient of a network layer. @rush2024federated adopted AD to the context of federated learning of ML models. @tang2023auto considered the problem of differentiating computations expressed in relational databases. Notwithstanding these selected examples, even experimental support for pipeline-level AD is nascent, and it is remarkable that recent surveys on the state of AD do not consider it being an important research direction (@van2018automatic, @baydin2018automatic).
2424

2525
## Tesseracts enable differentiable physics programming at scale
2626

0 commit comments

Comments
 (0)