June 15, 2021 Preseason Forecast Meeting

[fssem1]

Conclusions from June 15, 2021 Preseason Forecast Meeting:
• Bootstrap CI versus prediction interval on the forecast?
o The decision was made to go with the prediction interval.
• Use model averaging (what models to include) or just go with the top model based on a performance metric as AICc, MASE etc. and not use model averaging?
o The decision was made to go with model averaging but to weight the models based on a different metric than AIC (e.g., MAPE_LOOCV, MAPE_one_step_ahead, wMAPE but weight the historical years in a decreasing fashion)
o Additional references (thanks to Rich):
 https://www.cs.cmu.edu/~schneide/tut5/node42.html
 https://www.adfg.alaska.gov/static/applications/dcfnewsrelease/1232415165.pdf
 https://www.stat.umn.edu/geyer/5421/slides/glmbb.html
 https://uoftcoders.github.io/rcourse/lec09-model-selection.html
 https://cran.r-project.org/web/packages/AICcmodavg/vignettes/AICcmodavg.pdf
 https://rdrr.io/cran/MuMIn/man/model.avg.html
 https://esajournals.onlinelibrary.wiley.com/doi/10.1002/ecm.1309
 https://theoreticalecology.wordpress.com/2018/05/14/model-averaging-in-ecology-a-review-of-bayesian-information-theoretic-and-tactical-approaches-for-predictive-inference/
o Rich and Sara will see if they can find any support for one performance metric over another.
• Move to just the 20 m ISTI variable for the SECM variables for the 2022 forecast?
o The decision was made to just stick with ISTI_20m_MJJ variable as a possible variable in the models; The other ISTI variables will not be included as possible variables in the model averaging process for the 2022 preseason forecast.
• Logical set of environmental variables (space/time) to assess?—25 models currently in the document
o Reduced set of variables that includes: