Snowflake integration by gee-senbong · Pull Request #722 · Nixtla/nixtla

gee-senbong · 2026-01-03T05:05:40Z

This PR revises previous PR (#648) to add snowflake integration support.

Command to trigger the script to install Snowflake stored procedure:

python -m nixtla.scripts.snowflake_install_nixtla

CLAassistant · 2026-01-03T05:05:46Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ gee-senbong
❌ senbong

senbong seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

github-actions · 2026-01-03T05:08:27Z

Experiment Results

Experiment 1: air-passengers

Description:

variable	experiment
h	12
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	12.6793	11.0623	47.8333	76
mape	0.027	0.0232	0.0999	0.1425
mse	213.936	199.132	2571.33	10604.2
total_time	1.5605	0.6944	0.0046	0.0036

Plot:

Experiment 2: air-passengers

Description:

variable	experiment
h	24
season_length	12
freq	MS
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	58.1031	58.4587	71.25	115.25
mape	0.1257	0.1267	0.1552	0.2358
mse	4040.21	4110.79	5928.17	18859.2
total_time	0.6584	0.6448	0.0037	0.0041

Plot:

Experiment 3: electricity-multiple-series

Description:

variable	experiment
h	24
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	178.293	268.13	269.23	1331.02
mape	0.0234	0.0311	0.0304	0.1692
mse	121589	219485	213677	4.68961e+06
total_time	0.7406	1.1659	0.005	0.0045

Plot:

Experiment 4: electricity-multiple-series

Description:

variable	experiment
h	168
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	465.497	346.972	398.956	1119.26
mape	0.062	0.0436	0.0512	0.1583
mse	835021	403760	656723	3.17316e+06
total_time	0.9005	0.6468	0.0052	0.0046

Plot:

Experiment 5: electricity-multiple-series

Description:

variable	experiment
h	336
season_length	24
freq	H
level	None
n_windows	1

Results:

metric	timegpt-1	timegpt-1-long-horizon	SeasonalNaive	Naive
mae	558.673	459.757	602.926	1340.95
mape	0.0697	0.0565	0.0787	0.17
mse	1.22723e+06	739114	1.61572e+06	6.04619e+06
total_time	0.8129	0.7469	0.0049	0.0044

Plot:

gee-senbong · 2026-01-03T13:44:21Z

nixtla/scripts/snowflake_install_nixtla.py

+        MOD(HASH(unique_id), :MAX_BATCHES) AS gp,
+        unique_id,
+        ds,
+        OBJECT_CONSTRUCT_KEEP_NULL(*) AS data_obj


OBJECT_CONSTRUCT_KEEP_NULL is used here to handle the case when the historical exog variable has null value. Previous implementation:

nixtla/nixtla/scripts/snowflake_install_nixtla.py

Line 112 in e43d352

(SELECT MOD(HASH(unique_id), :MAX_BATCHES) AS gp, unique_id, ds, y, object_construct(*) AS obj FROM TABLE(:INPUT_DATA)) a,

gee-senbong · 2026-01-03T13:47:28Z

nixtla/scripts/snowflake_install_nixtla.py

+
+╔══════════════════════════════════════════════════════════════════════════════╗
+║                    🎉 Ready to start forecasting!                            ║
+╚══════════════════════════════════════════════════════════════════════════════╝


Sample scripts are generated to show user how to call the stored procedures using the generated sample tables.

JQGoh

Great to know that your demo video replicates the earlier behavior, a good start!

JQGoh · 2026-01-05T04:09:00Z

Makefile

+
+format:
+	@echo "Running black formatter on staged files..."
+	@git diff --cached --name-only --diff-filter=ACMR | grep '\.py$$' | xargs -r uv run black


Suggestion: Deployment execution command like
python -m nixtla.scripts.snowflake_install_nixtla can be registered as part of Makefile command.

JQGoh · 2026-01-05T04:12:09Z

nixtla/scripts/snowflake_install_nixtla.py

+CREATE OR REPLACE NETWORK RULE {ds_prefix}nixtla_network_rule
+MODE = EGRESS
+TYPE = HOST_PORT
+VALUE_LIST = ('api.nixtla.io');


Please help to make this an option that user can customize the domain name, in case we want to use a value other than 'api.nixtla.io'

JQGoh · 2026-01-05T04:24:59Z

nixtla/scripts/snowflake_install_nixtla.py

+```
+"""
+
+PACKAGES = [


Suggest that the essential packages can be read from the pyproject.toml and on top of that, we add the additional snowflake required dependencies.

The objective is that if Nixtla client has newly introduced dependency, we can automatically consider them than updating this manually for snowflake integration.

This is exactly the thing that didn't work in snowflake. Snowflake running environment is extremely picky on libraries, it doesn't support many of them, so we have to carefully pick the ones. I spent the most time to figure out this list when I worked on this. But @gee-senbong feel free to try what @JQGoh suggests. Maybe it's much easier to work with now.

goodwanghan · 2026-01-06T01:22:49Z

Makefile

+
+format:
+	@echo "Running black formatter on staged files..."
+	@git diff --cached --name-only --diff-filter=ACMR | grep '\.py$$' | xargs -r uv run black


i think we should move to uv format but these two new commands are good to add.

Will update this to use uv format.

gee-senbong · 2026-01-09T14:16:52Z

pyproject.toml

 dependencies = [
    "annotated-types",
    "httpx[zstd]",
+    "narwhals>=2.11.0",


Update this to fix the new integration test error when running in python 3.9 environment:

> sf_df, client_df = self._execute_and_compare( snowflake_session, deployed_with_api_endpoint, example_dataframes, "evaluation_metrics", compare_evaluate, ) nixtla_tests/snowflake/test_snowflake_deployment.py:379: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ nixtla_tests/snowflake/test_snowflake_deployment.py:218: in _execute_and_compare client_df = comparison_fn(test_case, example_data[test_case.input_table]) nixtla_tests/snowflake/test_snowflake_deployment.py:364: in compare_evaluate result = evaluate(data, metrics=metrics, models=forecasters, train_df=data) .venv/lib/python3.9/site-packages/utilsforecast/evaluation.py:314: in evaluate result = metric(**kwargs) .venv/lib/python3.9/site-packages/utilsforecast/losses.py:331: in mape return _nw_agg_expr( .venv/lib/python3.9/site-packages/utilsforecast/losses.py:76: in _nw_agg_expr nw.from_native(df) .venv/lib/python3.9/site-packages/narwhals/dataframe.py:1519: in select return super().select(*exprs, **named_exprs) .venv/lib/python3.9/site-packages/narwhals/dataframe.py:232: in select return self._with_compliant(self._compliant_frame.select(*compliant_exprs)) .venv/lib/python3.9/site-packages/narwhals/_pandas_like/dataframe.py:421: in select new_series = self._evaluate_into_exprs(*exprs) .venv/lib/python3.9/site-packages/narwhals/_compliant/dataframe.py:360: in _evaluate_into_exprs return tuple( .venv/lib/python3.9/site-packages/narwhals/_compliant/dataframe.py:361: in <genexpr> chain.from_iterable(self._evaluate_into_expr(expr) for expr in exprs) # pyright: ignore[reportArgumentType] .venv/lib/python3.9/site-packages/narwhals/_compliant/dataframe.py:375: in _evaluate_into_expr result = expr(self) .venv/lib/python3.9/site-packages/narwhals/_compliant/expr.py:236: in __call__ return self._call(df) .venv/lib/python3.9/site-packages/narwhals/_compliant/expr.py:675: in <lambda> lambda df: [series.alias(name) for series in self(df)], .venv/lib/python3.9/site-packages/narwhals/_compliant/expr.py:236: in __call__ return self._call(df) .venv/lib/python3.9/site-packages/narwhals/_compliant/expr.py:375: in _reuse_series_inner **{ .venv/lib/python3.9/site-packages/narwhals/_compliant/expr.py:376: in <dictcomp> name: df._evaluate_expr(value) if self._is_expr(value) else value .venv/lib/python3.9/site-packages/narwhals/_compliant/dataframe.py:352: in _evaluate_expr result: Sequence[EagerSeriesT] = expr(self) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = PandasThen(depth=0, function_name=whenthen) df = <narwhals._pandas_like.dataframe.PandasLikeDataFrame object at 0x137e37730> def __call__(self, df: EagerDataFrameT) -> Sequence[EagerSeriesT]: > return self._call(df) E AttributeError: 'PandasThen' object has no attribute '_call'

pyproject.toml

review-notebook-app · 2026-01-19T05:25:28Z

Check out this pull request on

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

goodwanghan · 2026-02-02T07:47:52Z

nbs/docs/deployment/3_snowflake.ipynb

goodwanghan

We have had a lot of offline discussion, and the PR reflects the design decisions based on our discussion.

gee-senbong added 4 commits December 28, 2025 10:05

Add snowflake_install_nixtla.py script from feat/snowflake branch

b668e99

Refactor the script & fix errors

9610e16

Add anomaly dections & update sample datasets

219b70e

Update forecast with hist exog variables support

24cd280

Cleanup

c9b61cf

gee-senbong commented Jan 3, 2026

View reviewed changes

JQGoh reviewed Jan 5, 2026

View reviewed changes

goodwanghan reviewed Jan 6, 2026

View reviewed changes

senbong and others added 12 commits January 7, 2026 20:17

Update Makefile

eff9378

Store base_url as secret & customize api_host

60ab059

Update Makefile

09ba29a

Add confidence intervals & integration tests

ab8e210

Cleanup codes

f68cc4a

Fix integration tests

9a9f001

Revise snowflake integration tests

af13a60

Added Snowflake environment variables to .github/workflows/pytest.yml

8712f80

Fix integration tests

527c186

Add uuid to snowflake assets

0e58251

Use different nixtla API key

176ee90

Resolve long paths in Windows

245981b

gee-senbong force-pushed the feat/snowflake-v2 branch from af579ec to 245981b Compare January 8, 2026 12:22

gee-senbong added 2 commits January 8, 2026 22:08

Constraint narwhals>=2.11.0

0b174cd

Update pandas version constraint

5b1f7b4

gee-senbong commented Jan 9, 2026

View reviewed changes

pyproject.toml Outdated Show resolved Hide resolved

gee-senbong force-pushed the feat/snowflake-v2 branch from a148893 to 5b1f7b4 Compare January 10, 2026 01:40

Add future exog support

d28015f

gee-senbong force-pushed the feat/snowflake-v2 branch from c2955a9 to d28015f Compare January 12, 2026 02:19

senbong added 2 commits January 18, 2026 20:56

Revise forecast to use single table

bf6d950

Merge branch 'main' into feat/snowflake-v2

1546368

Fix notebook test

cdc59f9

gee-senbong force-pushed the feat/snowflake-v2 branch from 8eb2361 to cdc59f9 Compare January 19, 2026 06:19

gee-senbong and others added 3 commits January 27, 2026 14:30

Add explain for feature contribution

c8be4c7

Merge branch 'main' into feat/snowflake-v2

771e8b9

Pin pandas < 3.0.0

3f3463b

gee-senbong force-pushed the feat/snowflake-v2 branch from 9f47f80 to 3f3463b Compare January 28, 2026 05:37

gee-senbong and others added 6 commits January 28, 2026 15:19

Revert changes in nbs/docs/tutorials/15_missing_values.ipynb

8ccb377

Update distributed extra requirements

8c4c387

Update README.md

0b015c2

Cleanup

d3dfd00

Merge branch 'main' into feat/snowflake-v2

46cf2bb

Add snowflake documentation

fc9e1f1

gee-senbong force-pushed the feat/snowflake-v2 branch from 1ac0b3e to fc9e1f1 Compare February 1, 2026 08:51

goodwanghan reviewed Feb 2, 2026

View reviewed changes

nbs/docs/deployment/3_snowflake.ipynb

Copy link

Contributor

goodwanghan Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

beautiful!

gee-senbong reacted with rocket emoji

goodwanghan approved these changes Feb 2, 2026

View reviewed changes

Merge branch 'main' into feat/snowflake-v2

047e2e7

gee-senbong merged commit da7bbb9 into main Feb 4, 2026
17 of 18 checks passed

gee-senbong deleted the feat/snowflake-v2 branch February 4, 2026 05:35

Conversation

gee-senbong commented Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

CLAassistant commented Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Jan 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Experiment 1: air-passengers

Description:

Results:

Plot:

Experiment 2: air-passengers

Description:

Results:

Plot:

Experiment 3: electricity-multiple-series

Description:

Results:

Plot:

Experiment 4: electricity-multiple-series

Description:

Results:

Plot:

Experiment 5: electricity-multiple-series

Description:

Results:

Plot:

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

JQGoh left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

review-notebook-app bot commented Jan 19, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

goodwanghan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

gee-senbong commented Jan 3, 2026 •

edited

Loading

CLAassistant commented Jan 3, 2026 •

edited

Loading

github-actions bot commented Jan 3, 2026 •

edited

Loading

JQGoh left a comment •

edited

Loading