feat(feature): add auto time features

Author: LongxingTan

CHANGELOG.md

@@ -2,6 +2,7 @@
 
 ## v.0.0.15
 - fix classification/anomaly detection
+- fix from_pretrained
 
 
 ## v0.0.13

docker/Dockerfile

@@ -1,7 +1,7 @@
 From tensorflow/tensorflow:2.16.1-gpu
 
 RUN apt-get update
-RUN apt-get install -y libgl1-mesa-dev wget vim python3.8
+RUN apt-get install -y libgl1-mesa-dev wget vim python3.9
 
 RUN pip install --no-cache-dir tfts
 

docs/source/index.rst

@@ -9,23 +9,37 @@ TFTS Documentation
 
    <a class="github-button" href="https://github.com/LongxingTan/Time-series-prediction" data-icon="octicon-star" data-size="large" data-show-count="true" aria-label="Star LongxingTan/Time-series-prediction on GitHub">GitHub</a>
 
-TFTS (TensorFlow Time Series) supports state-of-the-art deep learning time series models for both business cases and data competitions. The package provides:
+TFTS (TensorFlow Time Series) supports state-of-the-art deep learning time series models for production, research and data competitions. Specifically, the package provides:
 
-* Flexible and powerful design for time series task
-* Advanced SOTA deep learning models
-* TFTS documentation lives at `time-series-prediction.readthedocs.io <https://time-series-prediction.readthedocs.io>`_
+* Flexible and powerful modular design for time series task
+* Easy-to-use advanced SOTA deep learning models
+* Allow training on CPUs, single and multiple GPUs, TPU
 
 
 Quick Start
 -----------------
-The tfts could accept any time series data of 3D data format as model input: ``(num_examples, train_sequence_length, num_features)``,
-and the model supported by tfts outputs 3D data as model output: ``(num_examples, predict_sequence_length, num_outputs)``
 
+1. Requirements
+~~~~~~~~~~~~~~~~~~
 
-Visit :ref:`Quick start <quick-start>` to learn more about the package.
+To get started with `tfts`, follow the steps below:
+
+* Python 3.7 or higher
+* `TensorFlow 2.x <https://www.tensorflow.org/install/pip>`_ installation instructions
+
+
+2. Installation
+~~~~~~~~~~~~~~~~~~
+Now you are ready, proceed with
+
+.. code-block:: shell
 
-- :ref:`detailed installation instructions<installation>`
-- :ref:`how to use it<usage>`
+    $ pip install tfts
+
+2. Learn more
+~~~~~~~~~~~~~~~~~~
+
+Visit :ref:`Quick start <quick-start>` to learn more about the package.
 
 
 Tutorials
@@ -38,18 +52,16 @@ The :ref:`Tutorials <tutorials>` section provides guidance on
 
 Models
 ---------
-The tfts library supports the SOTA deep learning models for time series.
 
-- `TFTS BERT model <https://github.com/LongxingTan/KDDCup2022-Baidu>`_ wins the 3rd place in `Baidu KDD Cup 2022 <https://aistudio.baidu.com/aistudio/competition/detail/152/0/introduction>`_
-- `TFTS Seq2Seq model <https://github.com/LongxingTan/Data-competitions/tree/master/tianchi-enso-prediction>`_ wins the 4th place in `Alibaba Tianchi ENSO prediction <https://tianchi.aliyun.com/competition/entrance/531871/introduction>`_
-- :ref:`Learn more models <models>`
+1. Design a Custom Model with TFTS
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
 
 .. code-block:: python
 
    import tensorflow as tf
    from tfts import AutoConfig, AutoModel
 
-
    def build_model(use_model, input_shape):
       inputs = tf.keras.layers.Input(input_shape)
       config = AutoConfig.for_model(use_model)
@@ -64,13 +76,22 @@ The tfts library supports the SOTA deep learning models for time series.
       model.compile(optimizer, loss_fn)
       return model
 
-
    model = build_model(use_model="bert", input_shape=(24, 3))
    model.summary()
 
 
+2. More highlights
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+The tfts library supports the SOTA deep learning models for time series.
+
+- `TFTS BERT model <https://github.com/LongxingTan/KDDCup2022-Baidu>`_ — 3rd place in `Baidu KDD Cup 2022 <https://aistudio.baidu.com/aistudio/competition/detail/152/0/introduction>`_
+- `TFTS Seq2Seq model <https://github.com/LongxingTan/Data-competitions/tree/master/tianchi-enso-prediction>`_ — 4th place in `Alibaba Tianchi ENSO prediction <https://tianchi.aliyun.com/competition/entrance/531871/introduction>`_
+- :ref:`Learn more models <models>`
+
+
 Tricks
--------
+----------
 Visit :ref:`Tricks <tricks>` if you want to know more tricks to improve the prediction performance.
 
 

docs/source/models.rst

@@ -11,7 +11,7 @@ Some experiments of tfts in Kaggle Dataset
 Models supported
 ------------------
 
-You can you below models in ``AutoModel``
+You can use below models with ``AutoModel``
 
 * RNN
 * Seq2seq
@@ -23,3 +23,8 @@ You can you below models in ``AutoModel``
 * NBeats
 * AutoFormer
 * Informer
+
+.. code-block:: python
+
+    config = AutoConfig.for_model("seq2seq")
+    model = AutoModel.from_config(config, predict_sequence_length=predict_sequence_length)

docs/source/quick-start.rst

@@ -74,6 +74,9 @@ The general setup for training and testing a model is
 
 3.1 Prepare the data
 ~~~~~~~~~~~~~~~~~~~~~~~~
+The tfts could accept any time series data of 3D data format as model input: ``(num_examples, train_sequence_length, num_features)``,
+and the model supported by tfts outputs 3D data as model output: ``(num_examples, predict_sequence_length, num_outputs)``
+
 Before training, ensure your raw data is preprocessed into a 3D format with the shape ``(batch_size, train_steps, features)``. Perform any necessary data cleaning, normalization, or transformation steps to ensure the data is ready for training.
 
 
@@ -121,11 +124,8 @@ Run with pretrained weights
     model = AutoModel.from_pretrained("tfts-model")
 
 
-3.3 Evaluate the model
-~~~~~~~~~~~~~~~~~~~~~~~
-
 
-3.4 Serve the model
+3.3 Serve the model
 ~~~~~~~~~~~~~~~~~~~~~~~
 Once the model is trained and evaluated, deploy it for inference. Ensure the model is saved in a format compatible with your serving environment (e.g., TensorFlow SavedModel, ONNX, etc.). Set up an API or service to handle incoming requests, preprocess input data, and return predictions in real-time.
 

docs/source/tutorials.rst

@@ -44,6 +44,25 @@ Feed the input data into the model
 - array for single variable prediction
 
 
+Features
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- datetime features
+- static features
+- dynamic features
+
+.. code-block:: python
+
+    from tfts.features import feature_registry, registry
+
+    feature_registry = feature_registry
+    feature_registry.register(["some features"])
+
+    @registry
+    def add_custom_features():
+        return
+
+
 .. _train_models:
 
 Train the models

examples/notebooks/multi_steps_sales_prediction.ipynb

@@ -72,10 +72,12 @@
    "source": [
     "import logging\n",
     "from typing import List, Optional, Union\n",
+    "\n",
     "import numpy as np\n",
     "import pandas as pd\n",
     "import tensorflow as tf\n",
-    "from tfts import AutoModel, AutoConfig, KerasTrainer"
+    "\n",
+    "from tfts import AutoConfig, AutoModel, KerasTrainer"
    ]
   },
   {
@@ -103,7 +105,7 @@
     "class CFG:\n",
     "    input_dir = \"/kaggle/input/china-vehicle-sales-data/china_vehicle_sales_data.csv\"\n",
     "    train_sequence_length = 12\n",
-    "    predict_sequence_length = 3\n"
+    "    predict_sequence_length = 3"
    ]
   },
   {
@@ -316,7 +318,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 5,
+   "execution_count": null,
    "id": "ad6327cc",
    "metadata": {
     "execution": {
@@ -340,6 +342,7 @@
     "\n",
     "logger = logging.getLogger(__name__)\n",
     "\n",
+    "\n",
     "def add_lagging_feature(\n",
     "    data: pd.DataFrame,\n",
     "    groupby_column: Union[str, List[str]],\n",
@@ -364,9 +367,6 @@
     "        for lag in lags:\n",
     "            feature_col_name = f\"{column}_lag{lag}\"\n",
     "            feature_columns.append(feature_col_name)\n",
-    "            logger.debug(\n",
-    "                f\"Creating lagging feature: {feature_col_name} for column '{column}' with lag {lag} and groupby '{groupby_column}'.\"\n",
-    "            )\n",
     "            data[feature_col_name] = data.groupby(groupby_column)[column].shift(lag)\n",
     "    return data"
    ]
@@ -759,7 +759,13 @@
    "source": [
     "feature_columns = []\n",
     "\n",
-    "data = add_lagging_feature(data, groupby_column=[\"provinceId\", \"model\"], value_columns=[\"salesVolume\"], lags=list(range(1, 12)), feature_columns=feature_columns)\n",
+    "data = add_lagging_feature(\n",
+    "    data,\n",
+    "    groupby_column=[\"provinceId\", \"model\"],\n",
+    "    value_columns=[\"salesVolume\"],\n",
+    "    lags=list(range(1, 12)),\n",
+    "    feature_columns=feature_columns,\n",
+    ")\n",
     "\n",
     "data"
    ]
@@ -854,7 +860,9 @@
    ],
    "source": [
     "grouped_sequence = data.groupby([\"provinceId\", \"model\"]).apply(\n",
-    "    lambda x: x.sort_values('Date')[[\"salesVolume\", \"salesVolume_lag1\", \"salesVolume_lag2\", \"salesVolume_lag3\"]].to_numpy()\n",
+    "    lambda x: x.sort_values(\"Date\")[\n",
+    "        [\"salesVolume\", \"salesVolume_lag1\", \"salesVolume_lag2\", \"salesVolume_lag3\"]\n",
+    "    ].to_numpy()\n",
     ")\n",
     "\n",
     "data_3d = np.stack(grouped_sequence.values)\n",
@@ -902,27 +910,25 @@
     "        self.total_samples = self.num_ids * self.samples_per_id\n",
     "\n",
     "        # Precompute all valid (id, start_idx) pairs\n",
-    "        self.indices = [\n",
-    "            (i, j)\n",
-    "            for i in range(self.num_ids)\n",
-    "            for j in range(self.samples_per_id)\n",
-    "        ]\n",
-    "    \n",
+    "        self.indices = [(i, j) for i in range(self.num_ids) for j in range(self.samples_per_id)]\n",
+    "\n",
     "    def __getitem__(self, index):\n",
-    "        # batch-wise item        \n",
-    "        batch_indices = self.indices[index * self.batch_size:(index + 1) * self.batch_size]\n",
-    "        \n",
+    "        # batch-wise item\n",
+    "        batch_indices = self.indices[index * self.batch_size : (index + 1) * self.batch_size]\n",
+    "\n",
     "        x_batch = []\n",
     "        y_batch = []\n",
     "\n",
     "        for id_idx, start_idx in batch_indices:\n",
-    "            x = self.data[id_idx, start_idx:start_idx + self.train_seq_len, 1:]\n",
-    "            y = self.data[id_idx, start_idx + self.train_seq_len:start_idx + self.train_seq_len + self.pred_seq_len, 0]\n",
+    "            x = self.data[id_idx, start_idx : start_idx + self.train_seq_len, 1:]\n",
+    "            y = self.data[\n",
+    "                id_idx, start_idx + self.train_seq_len : start_idx + self.train_seq_len + self.pred_seq_len, 0\n",
+    "            ]\n",
     "            x_batch.append(x)\n",
     "            y_batch.append(y)\n",
     "\n",
     "        return np.nan_to_num(np.array(x_batch)), np.nan_to_num(np.array(y_batch))\n",
-    "    \n",
+    "\n",
     "    def __len__(self):\n",
     "        # depends on how many samples you want to extract from 1 ID\n",
     "        return int(np.ceil(len(self.indices) / self.batch_size))"
@@ -1086,14 +1092,14 @@
    "source": [
     "def build_model():\n",
     "    inputs = tf.keras.Input(shape=(CFG.train_sequence_length, 3))\n",
-    "    \n",
+    "\n",
     "    config = AutoConfig()(\"rnn\")\n",
     "    config.rnn_type = \"lstm\"\n",
     "    backbone = AutoModel.from_config(config=config)\n",
-    "    \n",
+    "\n",
     "    outputs = backbone(inputs)\n",
     "    model = tf.keras.Model(inputs=inputs, outputs=outputs)\n",
-    "    model.compile(loss=tf.keras.losses.MeanAbsoluteError(), optimizer=tf.keras.optimizers.Adam(), metrics = ['mae'])\n",
+    "    model.compile(loss=tf.keras.losses.MeanAbsoluteError(), optimizer=tf.keras.optimizers.Adam(), metrics=[\"mae\"])\n",
     "    return model\n",
     "\n",
     "\n",
@@ -1165,8 +1171,8 @@
     }
    ],
    "source": [
-    "history = model.fit(train_dataset, validation_data=valid_dataset, epochs=10) \n",
-    "model.save_weights('./sales_model.weights.h5')"
+    "history = model.fit(train_dataset, validation_data=valid_dataset, epochs=10)\n",
+    "model.save_weights(\"./sales_model.weights.h5\")"
    ]
   }
  ],

tests/test_datasets/__init__.py tests/test_data/__init__.pytests/test_datasets/__init__.py renamed to tests/test_data/__init__.py

@@ -1,6 +1,6 @@
 import unittest
 
-from tfts.datasets.get_data import get_air_passengers, get_data, get_sine
+from tfts.data.get_data import get_air_passengers, get_data, get_sine
 
 
 class GetDataTest(unittest.TestCase):