Inclusion of P/E curves into evaluation

[barnabasharris]

Hi,
I was wondering whether it would make sense to generate and return P/E curves (see Hirzel et. al. 2006, section 4.2) as part of the ENMevaluate() function? I'm not super up to date with the research in this area so am unsure whether they have been superseded by more recent evaluation approaches (?) but the ability to partition the continuous predictions of the models into sensible classifications seems appealing:

If you felt it was valuable, I think I could incorporate it into the package code.

Edit: just found this recent article, which indicates it could be useful https://onlinelibrary.wiley.com/doi/10.1002/ece3.11208

Cheers,

[barnabasharris]

Just thought I should expand on this a little bit for clarity. The proposed change would incorporate something like the following at around line 95 of tune.enm.R:

  bins <- seq(0, 1, by=.05)  # Deciles from 0 to 1
    
    # 4. Calculate P/E ratios for each bin
    pe_ratios <- lapply(1:(length(bins) - 1), function(i) {
      # Proportion of presences in bin i
      presence_in_bin <- sum(occs.val.pred >= bins[i] & occs.val.pred < bins[i + 1]) / length(occs.val.pred)
      
      # Proportion of background points in bin i
      background_in_bin <- 
        sum(c(bg.val.pred,bg.train.pred) >= bins[i] & c(bg.val.pred,bg.train.pred) < bins[i + 1]) / length(c(bg.val.pred,bg.train.pred))
      
      # Calculate P/E ratio
      pe <- presence_in_bin / background_in_bin
      
      data.frame(pe = pe, bin = bins[i], nk=nk)
    })
    
    pe_ratios_df <- 
      do.call('rbind',pe_ratios)

With the idea being that you calculate the ratio of predicted to expected presences / absences across all bins for predictions for each partition. These data can then be used to construct a confidence interval around the final predictions, as in Hirzel et. al. 2006, Figs. 5B, 5D, 5F)

Predictions can then be partitioned logically:

"The optimal number of classes may be defined by looking at the confidence interval around the continuous P/E curve (e.g. Fig. 5B), the goal being of finding how many HS classes (on the horizontal axis) may be defined while minimizing their overlap in P/E ratio (vertical axis). The continuous P/E curves also allow choosing the class boundaries objectively (Fig. 6). A first, natural boundary is defined by the P/E = 1 line: where P/E confidence interval is lower than 1, the model is predicting less presences than expected by chance, and the opposite when it is greater than 1; this may be used to distinguish unsuitable, marginal (random and uncertain) and suitable habitat (Fig. 6). Another natural threshold is the HS below which no presence ever occurs, suggesting uninhabitable conditions, which is equivalent to the threshold defining the minimal predicted area (MPA100) of Engler et al. (2004). Additional thresholds may be placed at the steps of the curve."

[jamiemkass]

@barnabasharris So sorry it took so long for me to respond. I think this is a cool idea. If you have already done this and want to submit a PR, I can take a look at how the implementation might work, but it would likely go in a dev version of the pkg. I am just now updating to 2.0.5 on CRAN, so it may be some time before another update. Would like to see where you are with this now though.

[barnabasharris]

No worries. I will submit (my first) PR, so bear with me if I do it wrong! The way I've bundled the cross-validation data up might not be preferable to you as it introduces some (possibly unnecessary) dependencies but I don't think it would be difficult to remedy. I've developed a nice thresholding routine too which I'll include.

[barnabasharris]

Don't think it worked. My intended changes are visible here:
https://github.com/barnabasharris/ENMeval/tree/pe-curve
Let me know if I need to do anything further to facilitate the pull request or I can amend the changes if you think they need revising etc.

[jamiemkass]

Barnabas, thanks again for this great suggestion. Unfortunately, it did not make it into the latest version because I had so many other changes to make. I will consider it for a future update!

[barnabasharris]

No problem and sorry— I had intended to submit a proper merge request but didn't get round to it!