Negative value of good passes (Atomic-SPADL)

[chlosains]

I created a VAEP model based on women's football data cross European leagues and then I have been looking at games in the W Euros right now and came across this. VAEP seems to values passes negatively, and I am not sure if it's a me issue on the basis of my modelling or if this is a common VAEP thing?
The England - Wales games yesterday had nice build up play in the goal Russo scored, where James passes the ball into the box and Toone delivers a cutback that Russo gets on the end of and scores. The pass is a cutback assist from a very dangerous location, yet the pass itself is valued negatively while the reception is positive. Is this usual?

The figures are as follows. James pass 0.003. Tonne receival 0.07. Toone pass (cutback) -0.09. Russo receival 0.17. Russo shot 0.17.

[archit-manek]

I’ve been exploring VAEP recently too, I did some digging into why cutbacks from dangerous positions are undervalued and from what I’ve read, it’s actually a known quirk of VAEP models: actions like cutbacks or line breaking passes sometimes get undervalued, especially if the model is conservative or doesn’t have context features (like pass pressure, body orientation, etc.).

I found this research paper to be insightful: Actions Speak Louder Than Goals: Valuing Player Actions in Soccer

[chlosains]

Thank you for sharing this, Archit! I will take a read.

[probberechts]

In general, it's normal that VAEP values some passes negatively. These should be passes that decrease the probability of scoring and/or increase the probability of conceding. However, I assume this question is specifically about cutback passes.

@archit-manek is absolutely right that VAEP struggles to evaluate specific actions accurately due to a lack of contextual information. In particular, standard event data doesn't capture defender positioning, which can lead to undervaluing actions like a through ball that puts a player one-on-one with the goalkeeper because the model cannot know that there are no defenders anymore.

Thus, you shouldn't spend too much attention to individual action scores when evaluating a VAEP model. These single-action assessments are often noisy. However, over a larger sample (around 10 games), the model's aggregate evaluation tends to provide a reliable measure of a player's overall contribution.

That said, I think the issue you're encountering is somewhat different. VAEP should, in principle, recognize that a successful cutback pass tends to increase scoring probability. I suspect two things might be going wrong here:

1. If your dataset contains few (or no) examples of cutbacks that lead to shots or goals, the model will likely generalize from the more common pattern that backward passes reduce scoring probability. Without enough positive cutback examples, it won't learn the exception that applies in certain zones.

2. If you're using a tree-based model and haven't trained with enough trees, the model may lack the complexity required to capture the distinction between a generic backward pass and a high-value cutback. Gradient boosting works by sequentially training decision trees, where each new tree focuses on correcting the errors made by the previous ones. Early trees typically learn broad, general trends (like “backward passes reduce scoring chance”), while the later trees refine the model by learning exceptions and edge cases (like cutbacks). If you're using too few trees, the model may never reach the stage where it starts learning these subtle patterns. Increasing the number of trees (with appropriate regularization) can help the model capture it.

[chlosains]

I had a look at how many cutbacks show up in my training set. I have 696 cutback attempts according to my current (and imperfect but to the low end) cutback definition in 2022 and 2023 NWSL. Thats only two out of the 14 seasons of data I am using for training.
From my understanding the issue is actually not whether the model is learning cutbacks (I think it is), I think when you add together the Toone pass (-0.09) and Russo’s receiving value (0.17) then the cutback is probably valued more reasonably and so the issue is actually in how the model splits between passer and receiver. This is my hypothesis anyway.
The split between passer and receiver I assume is something the model also learns?
In my data i have 36% of passes have higher passing VAEP while 64% have higher receiving VAEP. How does the model learn which to lean towards?

[probberechts]

Atomic-VAEP assigns value to passes without explicitly knowing the immediate outcome of the pass. Instead, it learns the expected outcome based on historical data, and the pass’s value reflects this expected result.

To put it concretely: suppose a pass moves the ball from location A to location B. The value of the pass is essentially the difference between the value of possessing the ball at location A and the value at location B. Importantly, the value at location B depends heavily on whether the ball is successfully received by a teammate or intercepted by an opponent. While the model doesn’t know the actual outcome of the pass at that moment, it has learned from historic data the probability of each possible outcome and incorporates this into its valuation.

Thus, if a player receives a pass in a game situation where most players would struggle to control it (such as a long ball) the majority of the credit is attributed to the receiver. Conversely, if the pass is made in a context where successful receptions are very common (e.g., a pass between two central defenders), most of the credit goes to the passer. This approach is similar to how g+ values passes, though in Atomic-VAEP this value attribution is implicit within the modelling framework rather than explicitly computed in a postprocessing step.

It could be that cutback passes have an extremely low probability of being received successfully, which would imply that they are generally a bad option and thus valued negatively. However, that would surprise me.