BERTCGAForcaster Class

[sonqt]

Description

This PR introduces several enhancements and new features, including:

• Forecaster Class: Implements support for auto-encoding transformer models such as BERT, RoBERTa, SpanBERT, DeBERTa, and ELECTRA.
• New Metric for forecaster.summarize(): Adds an additional evaluation metric to improve summarization capabilities.
• BERTCGAModel Class: Implements a new model class specifically designed for forecasting with BERT-based architectures.
• Jupyter Notebooks:
  • Demonstration notebooks showcasing how to use BERTCGAModel.
  • Notebooks for reproducing the performance results reported in the paper.
• Enhanced Download Configuration: Modifies download_config to enable downloading fine-tuned BERT-base and RoBERTa-large models from the Zissou server.

[jpwchang]

In addition to the comments above, one additional open point of discussion: so far, we have no documentation pages for subclasses of ForecasterModel (ForecasterModel itself has documentation, located at the bottom of the Forecaster documentation page). This is mainly because that class exists somewhat outside the ConvoKit class hierarchy (it is not a Transformer subclass). In the past, this was alright since we didn't have many concrete ForecasterModel subclasses (just CRAFT and the BoW baseline). But now that we're getting more and more models, you may imagine a user wanting to quickly understand the difference between the many options they have for the forecaster_model parameter in Forecaster, and right now there is nowhere they can go to in the docs to understand this.

It seems to me we definitely need documentation pages now for every ForecasterModel subclass. The discussion to be had is exactly where on the website these pages would go (since they are not part of the Transformer hierarchy, which is the main organizational system for the documentation site right now). Two ideas I can think of are (1): as a new subpage under the Forecaster documentation page, and (2) as a standalone header on the site, similar to how "Utilities" is currently treated (that page also being outside the Transformer hierarchy). But I am also open to other ideas.

[seanzhangkx8]

I will go ahead and merge the new BERTForecaster, and Son will continue working on integrating Forecaster with other LLMs (Gemma/LlaMA) and documentation.