Next scheduled date is not set correctly when recurring contribution fails

[larssandergreen]

When a recurring contribution fails, the next scheduled date appears to always be set to one frequency interval in the future, no matter what the failure count is. I followed this through in the debugger and can see that the ContributionRecur is being correctly set to have the same next scheduled date as before the failed attempt, but then later when the failed Contribution is added, the next scheduled date is set to one frequency interval in the future by core.

This Contribution API call here has the result of the next scheduled date being pushed ahead, which happens after the ContributionRecur has been adjusted, so it overwrites the next scheduled date set earlier. I believe this is the code in core that is setting the next scheduled date ahead with the 48 hours test (the next scheduled is today, which is within 48 hours of now, so it is set to one frequency unit in the future).

So I think the solution here might be to reset the ContributionRecur next scheduled date after creating the failed Contribution. Or alternately, maybe this should be changed in core.

[adixon]

Ah, yes, I recall a discussion about core getting involved with the update of the next scheduled date, that used to only happen in the extension.

I'd say core shouldn't be quite so cavalier in deciding on the next scheduled date, but if core is resistant to change, then (re)setting the next scheduled date after creating the failed contribution isn't a terrible work around.

[larssandergreen]

The strange thing is that it did work as intended until maybe 1-3 months ago for us, so I think something changed in the extension that let core do its thing when it previously could not - but it's not clear to me what that is.

I agree with you that core shouldn't be so cavalier, but if the change has been there since 5.29, might be hard to argue to change it back now.

Honestly, the cases when someone's credit card doesn't work Monday and does work on Tuesday are rare-to-non-existent in my experience. It might make more sense to retry in a week or something like that, which would avoid this issue with core, but that would add complexity. I think an ideal setting would be retry once per week for N tries, then once per month for M tries, then stop trying (not having to manually stop the processing for people that we can't get a hold of would be one less task). But other folks might have different needs.

[adixon]

Yes, the strategy for failures is complicated. I'd note that there's more than one reason for failure, not just cards going bad. Lately, it's been about the iATS server not responding to requests (i.e. "down"), and I've also seen the occasional network routing type issue. And highly monitored installs want different strategies compared to those that aren't monitored.

My general approach is to make it work reasonably well in relatively common worst case scenarios, and provide tools and documentation for tweaking for more monitored installs.

re: what might have changed - yes, the latest iATS release made some changes that might have impacted how this works, trying to avoid the multiple charging of cards that was happening under some failure circumstances (mostly related to failures within in the api calls to generate the recurring contribution).

Conclusion: sounds like I should add in the next scheduled date 'force set' after all the api calls are done. I think I'm already force-setting a few fields anyway so it shouldn't be too much of a hack. But the logic is definitely complicated, so it might be time to look at the big picture again and make sure it all makes sense.

Here's where I think the behaviour you're seeing comes from:
https://github.com/iATSPayments/com.iatspayments.civicrm/blob/aef234581146c2bd276d5f6769c83d082f392d13/CRM/Iats/Transaction.php#LL221C19-L221C19

[larssandergreen]

Sounds quite reasonable to me, thanks. Agreed, that's were it comes from.