Paper: NODEFit: Fit time-series data with a Neural Differential Equation - #1208
Paper: NODEFit: Fit time-series data with a Neural Differential Equation#1208gpavanb1 wants to merge 27 commits into
Conversation
for more information, see https://pre-commit.ci
|
Curvenote Preview
|
Co-authored-by: Rowan Cockett <rowanc1@gmail.com>
Co-authored-by: Rowan Cockett <rowanc1@gmail.com>
for more information, see https://pre-commit.ci
|
Dear Pavan (@gpavanb1), I am Jennifer Yoon (@JennEYoon), co-chair of the SciPy 2026 Proceedings Committee. I am honored to serve as the Editor for this paper. Please feel free to reach out to me with any questions about the Proceedings review process. Reviewers, please introduce yourselves here. Important upcoming dates:
Authors, please try to respond to all substantive reviewer comments during each round of reviews. Feel free to ask the reviewers for clarification. Because this is an iterative process, please check in every few days to see if there are any open issues. And feel free to revise your paper at any time up until the final submission date. |
|
Reviewers, here are additional information. Reviewers -- readme: https://github.com/scipy-conference/scipy_proceedings/blob/2026/README.md#instructions-for-reviewers Reviewers -- How to add comments on GitHub Pull Request: |
|
Hi @gpavanb1, @JennEYoon, I am Mayank Nigam, and I'll be reviewing this paper. I received the reviewer invitation today (July 27), so my initial review will follow shortly. I'll aim to have full first-round feedback posted within the next few days. Looking forward to the discussion! |
|
Hi Mayank (@manigam00), Thank you for agreeing to review this paper! To see how the review process works in practice, take a look at the completed reviews in PR1250 and PR1238.
Feedback is provided in two ways: inline comments attached to line numbers in the Keep in mind that the target reader is a typical SciPy Conference attendee. The content should be broadly understandable to non-experts, while still offering rigorous technical presentation—including comparisons to standard methods or software packages—for domain experts.. See above links to reviewers readme and reviewers - how to add comments to a GitHub PR. Jennifer |
|
Dear Ivan (@iperezav), |
|
Hi! My name is Lou Scheffer, and I'll be one of the reviewers. I'm a EE by training and a biologist by experience. You can see more about me here: http://lscheffer.com/resume.html. I'm not an expert on solving stochastic differential equations, nor on Python ODE libraries (my background is C++) but the methods here (reconstructing both state and gradient working backwards) seem plausible and well motivated. The library seems well constructed for ease of use. One thing I think is missing is motivation. A paper proposing new methods often starts with an example where existing technique fail, and a quick discussion of why they fail. Then the new method is introduced, often with a informal description of why it should work better, then the technical details, then a plot/figure showing that the new technique works where the existing techniques fail. This is especially powerful if the author was trying to solve some real world problem, could not with existing libraries, and hence this library is the result. An extension of this is a paragraph or two in the conclusions, meant for the user, describing the cases where these new techniques may be appropriate. This might take the form of (very vaguely) if you try to fit your data using the standard library/methods XXX, and you don't get a good fit because of YYY, then you might have better results using the methods here. You will want to use the smallest neural network that fits the underlying process, informed either by a knowledge of the underlying process, or perhaps by experiment when the underlying process is unknown. (This is a purely hypothetical example of how this might look.) |
|
|
||
| ## Introduction | ||
|
|
||
| Physical phenomena are commonly governed by differential equations. Traditional methods for time-series analysis often rely on discrete-time models, which may fail to capture the underlying continuous-time dynamics. Neural Ordinary Differential Equations (Neural ODEs) [@chen2018torchdiffeq] represent a paradigm shift by modeling the latent state evolution as a continuous process. |
There was a problem hiding this comment.
The introduction states that discrete-time models "may fail to capture the underlying continuous-time dynamics" but no example, figure, or reference is given. A brief concrete case (or citation) showing where a discrete method falls short would make the motivation much more convincing.
There was a problem hiding this comment.
The introduction states that discrete-time models "may fail to capture the underlying continuous-time dynamics" but no example, figure, or reference is given. A brief concrete case (or citation) showing where a discrete method falls short would make the motivation much more convincing.
Thank you @manigam00 for the valuable comment!
This has been addressed in the newer introductory section, where we add two new citations that talk about the shortcoming of discrete models in the context of irregular sampling.
There was a problem hiding this comment.
Thanks — I've read the revised introduction and the added citations address this well. Resolving from my side.
|
|
||
| Physical phenomena are commonly governed by differential equations. Traditional methods for time-series analysis often rely on discrete-time models, which may fail to capture the underlying continuous-time dynamics. Neural Ordinary Differential Equations (Neural ODEs) [@chen2018torchdiffeq] represent a paradigm shift by modeling the latent state evolution as a continuous process. | ||
|
|
||
| NODEFit provides a simple Python interface for fitting ODEs and SDEs to measured data, allowing researchers to fit their data directly to the governing physical laws. |
There was a problem hiding this comment.
Since NODEFit builds on torchdiffeq/torchsde (per the Implementation section), it would help to state here what gap NODEFit fills relative to using those libraries directly. (This echoes @LouScheffer's point about motivation — I arrived at the same gap as a general reader.)
As a reader with a software/ML background but new to differential equation modeling, I had to read this introduction two or three times. It assumes familiarity with discrete-time models, neural ODEs, and SDEs without a sentence introducing each. Since the target audience is a general SciPy attendee, consider adding brief intuitive definitions. Also, "SDE" appears here in abbreviated form only — per the review criteria, consider spelling it out at first use in the body text.
There was a problem hiding this comment.
Dear Pavan @gpavanb1,
Re: Mayank's review comment:
As a reader with a software/ML background but new to differential equation modeling, I had to read this introduction two or three times. It assumes familiarity with discrete-time models, neural ODEs, and SDEs without a sentence introducing each. Since the target audience is a general SciPy attendee, consider adding brief intuitive definitions. Also, "SDE" appears here in abbreviated form only — per the review criteria, consider spelling it out at first use in the body text.
I agree. Per the SciPy writing guidelines, acronyms must be spelled out upon first use in both the abstract and the main text—for example, 'stochastic differential equations (SDE)'. Feel free to repeat the full term occasionally if it helps readability.
I also suggest adding a footnote on the first page briefly outlining the history of Itô's lemma, SDEs, the Black-Scholes(-Merton) models, and a short biography of mathematician Kiyoshi Itô.
Although quant finance has strong ties to SciPy Conference Community—given sponsors like Two Sigma and Jane Street, as well as Two Sigma's early funding of pandas—it is still a niche topic for our general attendees. A brief primer will go a long way in making the paper more accessible.
There was a problem hiding this comment.
Dear Pavan @gpavanb1,
Re: Mayank's review comment:
As a reader with a software/ML background but new to differential equation modeling, I had to read this introduction two or three times. It assumes familiarity with discrete-time models, neural ODEs, and SDEs without a sentence introducing each. Since the target audience is a general SciPy attendee, consider adding brief intuitive definitions. Also, "SDE" appears here in abbreviated form only — per the review criteria, consider spelling it out at first use in the body text.
I agree. Per the SciPy writing guidelines, acronyms must be spelled out upon first use in both the abstract and the main text—for example, 'stochastic differential equations (SDE)'. Feel free to repeat the full term occasionally if it helps readability.
I also suggest adding a footnote on the first page briefly outlining the history of Itô's lemma, SDEs, the Black-Scholes(-Merton) models, and a short biography of mathematician Kiyoshi Itô.
Although quant finance has strong ties to SciPy Conference Community—given sponsors like Two Sigma and Jane Street, as well as Two Sigma's early funding of pandas—it is still a niche topic for our general attendees. A brief primer will go a long way in making the paper more accessible.
Dear Jennifer, Thank you for your suggestions. I will follow up with a full response within 7 days.
There was a problem hiding this comment.
Dear Pavan @gpavanb1,
Re: Mayank's review comment:
As a reader with a software/ML background but new to differential equation modeling, I had to read this introduction two or three times. It assumes familiarity with discrete-time models, neural ODEs, and SDEs without a sentence introducing each. Since the target audience is a general SciPy attendee, consider adding brief intuitive definitions. Also, "SDE" appears here in abbreviated form only — per the review criteria, consider spelling it out at first use in the body text.
I agree. Per the SciPy writing guidelines, acronyms must be spelled out upon first use in both the abstract and the main text—for example, 'stochastic differential equations (SDE)'. Feel free to repeat the full term occasionally if it helps readability.
I also suggest adding a footnote on the first page briefly outlining the history of Itô's lemma, SDEs, the Black-Scholes(-Merton) models, and a short biography of mathematician Kiyoshi Itô.
Although quant finance has strong ties to SciPy Conference Community—given sponsors like Two Sigma and Jane Street, as well as Two Sigma's early funding of pandas—it is still a niche topic for our general attendees. A brief primer will go a long way in making the paper more accessible.
Thank you for the valuable comments @JennEYoon!
We have made the following changes as per your suggestions.
-
Abstract — Brief gloss for discrete-time regressors; Neural ODEs/SDEs explained intuitively; bare
ODEs/SDEsspelled out on first use (ordinary differential equations (ODEs),stochastic differential equations (SDEs)). -
Introduction — Short primer for a general SciPy reader:
- Discrete-time models: next-step predictors from a finite history
-
ODE / Neural ODE: vector field + neural
$f_\theta$ - SDE / Neural SDE: drift + diffusion driven by Brownian motion
-
First-page footnote (
[^ito-primer]) — Kiyosi Itô (1915–2008), stochastic integral / Itô’s lemma, Black–Scholes–Merton, and a one-line career note (Kyoto / founder of modern stochastic analysis).
Later intro paragraphs now use the acronyms without re-expanding them as well.
There was a problem hiding this comment.
Thanks to you both — the expanded acronyms, the introductory primer on discrete-time models and ODEs/SDEs, and the Itô footnote address my original comment well. Resolving from my side.
| Physical phenomena are commonly governed by differential equations. Traditional methods for time-series analysis often rely on discrete-time models, which may fail to capture the underlying continuous-time dynamics. Neural Ordinary Differential Equations (Neural ODEs) [@chen2018torchdiffeq] represent a paradigm shift by modeling the latent state evolution as a continuous process. | ||
|
|
||
| NODEFit provides a simple Python interface for fitting ODEs and SDEs to measured data, allowing researchers to fit their data directly to the governing physical laws. | ||
|
|
There was a problem hiding this comment.
The transition into Methods felt abrupt — I wasn't sure what "Method" was about to be described or why. A short roadmap sentence at the end of the introduction (e.g., "Section 2 presents the mathematical formulation") would orient the reader.
Dear Mayank, Thank you for your review. I will follow up with a full response within 7 days. |
Dear Lou, Thank you for your review. I will follow up with a full response within 7 days. |
…o address LouScheffer's comments in introduction, results and conclusion
for more information, see https://pre-commit.ci
Thank you for the valuable comments and the positive feedback! Please find below the revisions made based on your suggestions.
We expanded the Introduction with a concrete saturating-kinetics example, explained why template-based
We added Conclusion paragraphs on when to prefer NODEFit over polynomials/splines/curve_fit (poor extrapolation, unknown dynamics, coupled states), when to use Neural SDE vs ODE (intrinsic vs measurement noise), and guidance to use the smallest network that fits, which are to be validated on held-out time segments when domain knowledge is limited. We hope these revisions make the paper’s motivation and intended use cases clearer. Thank you again for the constructive feedback and it materially improved the manuscript. |
manigam00
left a comment
manigam00
left a comment
There was a problem hiding this comment.
Thank you for this submission — NODEFit addresses a genuinely useful problem, and the paper is well organized overall.
A note on my perspective: I'm a software engineering practitioner rather than a domain expert in differential equations. My comments therefore focus on structure, narrative, accessibility for a general SciPy audience, and verifiability, and I did not attempt to verify the mathematical derivations.
What works well. The revised Introduction now motivates the work clearly — the exponential-kinetics example with scipy.optimize.curve_fit, and the explanation of why a fixed functional template falls short, make the case for a continuous-time approach concrete. Figure 1 is the strongest evidence in the paper: showing the cubic baseline inflecting upward past t=5 while the Neural ODE levels off, with the training boundary marked, demonstrates the central claim directly. I also appreciated the memory comparison table for honestly stating the adjoint method's computational cost rather than only its advantages, and the sample code is compact and well commented throughout. The Conclusion's practical guidance — when a Neural ODE is unnecessary, when to prefer a Neural SDE, and how to think about network capacity — is particularly valuable for practitioners.
Main themes in my comments.
Quantitative evidence. Several claims rest on visual or conceptual support rather than measurement: the memory efficiency section compares properties without an empirical data point, the performance optimization subsection describes an optimized implementation without a speedup figure, and the results are demonstrated visually without fit metrics. A small number of concrete measurements would strengthen the paper considerably.
Accessibility. The paper's heavier mathematical passages assume more background than a typical SciPy attendee may have. Extending the symbol definitions that appear elsewhere in the paper to the remaining equations would help, as would a brief roadmap at the end of the Introduction.
Structure. The stochastic adjoint derivation currently sits inside the torchsde dependency subsection, while its deterministic counterpart is in Methods. Consolidating these would make both easier to find.
Completeness. PyTorch doesn't appear to be formally cited, and the installation instructions would benefit from stating version requirements.
Please feel free to reply on any individual comment before making changes — several of these are questions rather than requests, and I'd rather understand your reasoning than have you change something unnecessarily.
Disclosure: I used a generative AI assistant (Claude) to help me understand background concepts in this area and to refine the wording of my comments. All evaluations and judgments in this review are my own.
These comments refer to the paper as of commit 2087cfa (August 1, 2026).
|
|
||
| ```{math} | ||
| :label: eq:node | ||
| \frac{dy(t)}{dt} = f_\theta(y(t), t) |
There was a problem hiding this comment.
Suggest defining y and t alongside f_θ here, similar to how other symbols are defined after equations elsewhere in the paper.
| ``` | ||
|
|
||
| This enables gradient computation with constant memory cost by solving this equation backwards in time. | ||
|
|
There was a problem hiding this comment.
A general note: symbol definitions are somewhat inconsistent across the paper's equations. Many are nicely defined where they appear (e.g., f_θ, W_t, a(t), ϵ), but some notation goes unexplained — for example, in the adjoint equation, the transpose notation a(t)^T and the partial-derivative term are not described in words. Since the target reader is a general SciPy attendee, a brief plain-language note for these would help readers follow the more technical equations.
|
|
||
| ## Implementation | ||
|
|
||
| NODEFit is implemented as an open-source Python package built on top of the PyTorch ecosystem. It leverages specialized libraries to handle the numerical integration and gradient computation required for training Neural ODEs and SDEs. By abstracting these complexities, NODEFit makes it remarkably easy to fit complex time-series data to governing differential equations. All plots in this paper were generated using Matplotlib [@matplotlib]. |
There was a problem hiding this comment.
PyTorch is central to NODEFit but doesn't appear to be formally cited anywhere in the paper, while Matplotlib, torchdiffeq, and torchsde all are. Suggest adding a citation for PyTorch here, consistent with the review criteria on citing software.
| #### torchsde | ||
| For stochastic systems, NODEFit integrates `torchsde` [@li2020scalable]. Stochastic Differential Equations present unique challenges, particularly in ensuring consistent Brownian motion across multiple steps and handling the nuances of stochastic calculus. | ||
|
|
||
| To derive the stochastic adjoint from first principles, we consider the SDE in Stratonovich form (denoted by the $\circ$ operator). The Stratonovich integral evaluates the integrand at the midpoint of the interval, $g(y, t) \circ dW_t \approx g(y_{t+\Delta t/2}, t+\Delta t/2) \Delta W_t$. This choice ensures that the SDE obeys the standard rules of calculus: |
There was a problem hiding this comment.
This subsection is titled as being about the torchsde dependency, but a substantial part of it is a from first principles derivation of the stochastic adjoint. Structurally, would this derivation fit better in the Methods section (alongside the ODE adjoint derivation in the Backpropagation subsection, and near "Stochastic dynamics" which currently covers the SDE case only briefly) or in the Appendix with the rest of the stochastic calculus? That would keep this subsection focused on what torchsde provides and how NODEFit uses it.
| | **Intermediate States** | Stored in memory | Reconstructed on the fly | | ||
| | **Memory Scaling** | $O(N)$ (Linear with steps) | $O(1)$ (Constant with steps) | | ||
| | **Noise (SDEs)** | Must be stored for every step | Regenerated via Virtual Brownian Tree | | ||
| | **Trade-off** | Faster (no reconstruction) | Slower (requires solving backwards) | |
There was a problem hiding this comment.
This table is well done — accurate, clearly phrased for a general reader, and I especially appreciate that the Trade-off row honestly states the adjoint method's computational cost rather than showing only its advantages. One suggestion: since the section is titled "Memory Efficiency Comparison," I expected an empirical element — was a measurement made? Even one small data point (e.g., peak memory for a short vs. long trajectory under both modes) would turn the O(1) claim from stated to demonstrated. If that's not feasible before the deadline, consider retitling to something like "Memory Efficiency Trade-offs" so the title matches the (useful) conceptual content.
|
|
||
| ### Performance Optimizations | ||
|
|
||
| To handle larger datasets and more complex trajectories, we utilized an optimized implementation that inherits from the base `NeuralSDE` and `SDE` classes. This version leverages faster tensor operations for state-time concatenation and an efficient training loop. For the results presented in this paper, we tuned the following hyperparameters: a `batch_size` of 20 for improved gradient estimates and a fixed time step `dt` of 0.1 to balance numerical stability with computational speed. |
There was a problem hiding this comment.
This section mentions an optimized implementation with faster tensor operations and an efficient training loop — was the speedup measured? Even a rough figure would substantiate 'optimized.'
| NODEFit can be installed via pip: | ||
|
|
||
| ```bash | ||
| pip install nodefit |
There was a problem hiding this comment.
The installation instructions are refreshingly simple, but a bit more context would help practitioners: (1) minimum Python and PyTorch versions; (2) whether a GPU is required or optional (and whether CPU-only usage is supported);
As a prospective user, these were the first things I looked for after the pip command.
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
|
Thank you for the invitation @JennEYoon and thank you @gpavanb1 for submitting your work. The paper introduces the Python package NEUROFit. It fits ODEs and SDEs using neural networks and differential equation solver. The script is well written, it contains an introduction to the topic, a section with the description and performance of the method, a code example, results and conclusions. Overall, the script meets the goal of the proceedings by providing a way in which Python helped solved a scientific computational problem in the area of differential equations and neural networks. The work is relevant and here are my recommendations: Recommendations:
|
for more information, see https://pre-commit.ci
|
Very much appreciate the concise and insightful comments @iperezav
This seems to be based on the default template. I have tried removing labels for equations that are not cited, but they still show up with numbering.
This is a good point and to the author's knowledge, PySINDY comes close and has been added to the list of citations along with mentioning it in the introduction and methods section. The libraries
We clarified that the memory discussion concerns the theoretical scaling of the adjoint method rather than a benchmarking study. To improve reproducibility, we added the computer specifications and wall-clock times for the example scripts in a separate section and table.
We have added to the introduction and citations for transformers, ARIMA and signatures, and contrast them with the current approach.
This has been appropriately added to improve the readability of the paper.
We now list specific applications in the Introduction (ecology, epidemiology, chemical kinetics, robotics, and finance) and added a Lotka–Volterra reconstruction in the Results. NODEFit fits a structured Neural ODE of the classical prey–predator form from observations through t=6, recovers the four rates, and extrapolates the cycle to t = 12. |
7 participants
If you are creating this PR in order to submit a draft of your paper, please name your PR with
Paper: <title>. An editor will then add adraftlabel; this will trigger GitHub Actions to run automated checks on your paper and build a preview. You may then work to resolve failed checks and ensure the preview build looks correct. If you have any questions, please tag the proceedings team in a comment on your PR with@scipy-conference/2026-proceedings.See the project readme for more information.