[DistributionLoss] Add a "return samples" option

[Antoine-Schwartz]

Description

As already proposed by DeepAR on Sagemaker, it would be interesting to be able to retrieve the output samples when using a DistributionLoss.

Use case

Retrieving these trajectories directly in outputs allows for greater independence of use cases that come after forecasting. Typically, being able to recalculate different quantiles without having to relaunch a forecast job is a must for many supply chain optimization modules.

[elephaint]

Hi, apologies for the late reply. You can achieve what you want by specifying a large number of quantiles, for example:

loss = DistributionLoss(distribution='StudentT', quantiles=np.linspace(0, 1, 100))

This will effectively allow you to obtain the sampled values for the distribution per timestep in the forecast.

Does that answer your question?

[carlocav]

Hello! Is this still the only way to get sample paths? The problem of this approach is that it only works to get the marginal distribution of the outcome at single points in time independently and doesn't allow to get the joint forecast distribution at multiple future time periods taking temporal autocorrelation into account.

[elephaint]

As of 3.0.0, you can specify a quantiles argument in predict, i.e.

nf.predict(...., quantiles=[0.1, 0.2,....])

this works for DistributionLosses and point losses with conformal intervals. So it's similar but slightly easier implementation wise. Hence, the full distribution is available without a need for recalculation. Let me know if this works for you.

[carlocav]

Thank you for the answers. This still is returning quantiles rather than actual sample paths over time to approximate the joint forecast distribution.

Sample paths are generated so to compute quantiles and it would be nice to add the option to simply return them. This feature is available in gluonts library and it’s probably the only reason to keep using it.

[elephaint]

Sure, we could add a sample endpoint, don't think that's too hard. I'll reopen the issue.

Although not sure if you sample and do it e.g. 200 times to compute a distribution, you get the same as just outputting the quantiles. I mean I somewhat understand the desire to avoid max one line of additional code but it's also just one line? Or am I misunderstanding what it is that you seek?

[carlocav]

distributionloss generates samples paths that are then used by neuralforecast to compute and return quantiles. What I seek is the option to get them directly rather than the quantiles. In fact if we have a forecast horizon > 1 and we want to approximate the joint distribution over time we need the raw sample paths as we can’t use the quantiles of outcomes at single points in time because of temporal autocorrelation e.g if we have a very high value in t+1 it might be likely to have it also in t+2, etc. If I have sample paths I can compute all the quantities of interest, as an example we can compute quantiles of the cumulative sum of the outcomes in t+1 , … t + n adding up the quantities of time 1 to n of each sample and only then compute the quantiles, and it would not be possible to compute this quantity by adding the quantiles at the n points in time directly because quantiles are not additive.

You can find other possible use cases where sample paths rather than quantiles are needed e.g. here https://minimizeregret.com/post/2022/07/25/use-sample-paths/

[github-actions]

This issue has been automatically closed because it has been awaiting a response for too long. When you have time to to work with the maintainers to resolve this issue, please post a new comment and it will be re-opened. If the issue has been locked for editing by the time you return to it, please open a new issue and reference this one.

[elephaint]

distributionloss generates samples paths that are then used by neuralforecast to compute and return quantiles. What I seek is the option to get them directly rather than the quantiles. In fact if we have a forecast horizon > 1 and we want to approximate the joint distribution over time we need the raw sample paths as we can’t use the quantiles of outcomes at single points in time because of temporal autocorrelation e.g if we have a very high value in t+1 it might be likely to have it also in t+2, etc. If I have sample paths I can compute all the quantities of interest, as an example we can compute quantiles of the cumulative sum of the outcomes in t+1 , … t + n adding up the quantities of time 1 to n of each sample and only then compute the quantiles, and it would not be possible to compute this quantity by adding the quantiles at the n points in time directly because quantiles are not additive.

You can find other possible use cases where sample paths rather than quantiles are needed e.g. here https://minimizeregret.com/post/2022/07/25/use-sample-paths/

Ok, I get what you mean now. It's not how most of the NF models normally work when used with a DistributionLoss, they produce marginal distributions. And the parameters describing these marginal distributions are used to sample and produce quantiles. Consequently, there is no difference between sampling and producing quantiles. We don't use generated sample paths to produce the quantiles. We use the marginal distributions for those.

Hence, I think your request is to allow direct models to produce their forecasts recursively, so that 'true' sample paths can be generated, which generates the joint forecast distribution.

I think it's a good ask and something that should be incorporated in NF