Skip to content

Cross validation mods#321

Open
elmartinj wants to merge 14 commits intoTimeCopilot:mainfrom
elmartinj:cross_valdation_mods
Open

Cross validation mods#321
elmartinj wants to merge 14 commits intoTimeCopilot:mainfrom
elmartinj:cross_valdation_mods

Conversation

@elmartinj
Copy link
Contributor

This pull request contains updates to the MASE cross validation processes in agent, forecaster and experiment handler scripts. These involve the tokenization reduction additions as well. The idea is to allow different forms of cross validation to be available to user. in the analyze function in order to determine the best statistic approach to a specific dataset.

New argument to function analyze will allow user or api to determine cross validation folds as follows:

  • 1 fold - quick version
  • 3 fold - std version
  • 5 fold - robust version.

In addition to the calculations of the median of these folds in order to decide which is the best modeling approach, std deviation is provided to the LLM ; thus allowing for more clarity regarding the stability of all fold adjustments. final addition is regarding the pretify function, in order to display these results if requested.

elmartinj and others added 13 commits March 13, 2026 11:25
This tutorial contains an example with multiple indexes and subsequent tampering to the data in order to show resiliency and a real life use case of TC applied on cryptocurrency prices up to 2021.
Fix to the issue raised on empty dataframe, resulting from an inner merge where the existing dataframe that accumulated results and the newer one had diferent indices. A subsequent issue must be raised to either:
1 report a single model failure (on index matching)
2 fix the moirai discrepancy (only model that showed this issue)
Fix to the issue raised on empty dataframe, resulting from an inner merge where the existing dataframe that accumulated results and the newer one had diferent indices. A subsequent issue must be raised to either:
1 report a single model failure (on index matching)
2 fix the moirai discrepancy (only model that showed this issue)
…for nfl bets. Processor will take the mode of the last 3 days to select a spread consensus, spread consensus prices will later be fetched accross all data in order to have the most complete timeseries. Final result of processor file execution is a csv that contains timestamps, spread consensus information, and who's playing who. Y column will contain odds for that specific ratio. This file is compatible with timecopilot and interpretation of the final forecast will be determined by the spread registered in the id's and the final price forecasted on markets by timecopilot. This can be an average for the next day or high/low. I leave that choice for the crew's choosing.
…okenization run, first trial version. felt cute, might delete later
…cripts in order to refine the possibilites available for cross validation. This will allow user through the analyze function to determine the amount of folds desired within a process. This should be flexible because folds obv imply more comp/time costs
…ms of cross validation to be available to user. new argument to function analyze will allow user or api to determine cross validation folds as follows: 1 fold - quick version, 3 fold - std version, 5 fold - robust version. in addition to the calculations of the median of these folds in order to decide which is the best modeling approach, std deviation is provided to the llm; thus allowing for more clarity regarding the stability of all fold adjustments. final addition is regarding the pretify function, in order to display these results if requested.
@CLAassistant
Copy link

CLAassistant commented Mar 13, 2026

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants