Skip to content

TabPFN updates #1598

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
bgruening merged 19 commits into from
Mar 26, 2025
Merged

TabPFN updates #1598

Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
03a2d99
add license and comparison with xgboost and catboost
anuprulez Mar 25, 2025
64e2dd5
update tests
anuprulez Mar 25, 2025
166f825
fix method
anuprulez Mar 25, 2025
ed5c01d
update tool
anuprulez Mar 25, 2025
b51f7ac
update
anuprulez Mar 25, 2025
91490cd
Merge branch 'bgruening:master' into tabpfn_license
anuprulez Mar 25, 2025
1744d96
fix formatting
anuprulez Mar 25, 2025
3cbf64e
formatting
anuprulez Mar 25, 2025
f32b0fa
fix formatting
anuprulez Mar 25, 2025
650b6c6
fix formatting
anuprulez Mar 25, 2025
ded15b4
fix errors
anuprulez Mar 26, 2025
bc74fde
fix linting in help
anuprulez Mar 26, 2025
653a189
Update tools/tabpfn/main.py
anuprulez Mar 26, 2025
504325f
Update tools/tabpfn/main.py
anuprulez Mar 26, 2025
d745cf3
Update tools/tabpfn/main.py
anuprulez Mar 26, 2025
84fa20d
fix review comments
anuprulez Mar 26, 2025
03e7b4c
fix review comments
anuprulez Mar 26, 2025
38e0d0b
fix linting
anuprulez Mar 26, 2025
606bf12
Update tools/tabpfn/tabpfn.xml
anuprulez Mar 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 15 additions & 9 deletions tools/tabpfn/main.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,9 @@ def classification_plot(y_true, y_scores):
plt.plot(
recall, precision, linestyle="--", color="black", label="Micro-average"
)
plt.title("Precision-Recall Curve (Multiclass Classification)")
plt.title(
"Precision-Recall Curve (Multiclass Classification)"
)
plt.xlabel("Recall")
plt.ylabel("Precision")
plt.legend(loc="lower left")
Expand Down Expand Up @@ -85,21 +87,25 @@ def train_evaluate(args):
# prepare train data
tr_features, tr_labels = separate_features_labels(args["train_data"])
# prepare test data
if args["testhaslabels"] == "haslabels":
if args["testhaslabels"] == "true":
te_features, te_labels = separate_features_labels(args["test_data"])
else:
te_features = pd.read_csv(args["test_data"], sep="\t")
te_labels = []
s_time = time.time()
if args["selected_task"] == "Classification":
classifier = TabPFNClassifier()
classifier = TabPFNClassifier(random_state=42)
classifier.fit(tr_features, tr_labels)
y_eval = classifier.predict(te_features)
pred_probas_test = classifier.predict_proba(te_features)
if len(te_labels) > 0:
classification_plot(te_labels, pred_probas_test)
te_features["predicted_labels"] = y_eval
te_features.to_csv(
"output_predicted_data", sep="\t", index=None
)
else:
regressor = TabPFNRegressor()
regressor = TabPFNRegressor(random_state=42)
regressor.fit(tr_features, tr_labels)
y_eval = regressor.predict(te_features)
if len(te_labels) > 0:
Expand All @@ -112,14 +118,14 @@ def train_evaluate(args):
"True values",
"Predicted values",
)
te_features["predicted_labels"] = y_eval
te_features.to_csv(
"output_predicted_data", sep="\t", index=None
)
e_time = time.time()
print(
"Time taken by TabPFN for training and prediction: {} seconds".format(
e_time - s_time
)
f"Time taken by TabPFN for training and prediction: {e_time - s_time} seconds"
)
te_features["predicted_labels"] = y_eval
te_features.to_csv("output_predicted_data", sep="\t", index=None)


if __name__ == "__main__":
Expand Down
143 changes: 73 additions & 70 deletions tools/tabpfn/tabpfn.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,112 +5,115 @@
<token name="@VERSION_SUFFIX@">1.1</token>
</macros>
<creator>
<organization name="European Galaxy Team" url="https://galaxyproject.org/eu/" />
<person givenName="Anup" familyName="Kumar" email="kumara@informatik.uni-freiburg.de" />
<person givenName="Frank" familyName="Hutter" email="fh@cs.uni-freiburg.de" />
<organization name="European Galaxy Team" url="https://galaxyproject.org/eu/"/>
<person givenName="Anup" familyName="Kumar" email="kumara@informatik.uni-freiburg.de"/>
<person givenName="Frank" familyName="Hutter" email="fh@cs.uni-freiburg.de"/>
</creator>
<requirements>
<requirement type="package" version="@TOOL_VERSION@">tabpfn</requirement>
<requirement type="package" version="2.2.2">pandas</requirement>
<requirement type="package" version="3.9.2">matplotlib</requirement>
<requirement type="package" version="@TOOL_VERSION@">tabpfn</requirement>
<requirement type="package" version="2.2.2">pandas</requirement>
<requirement type="package" version="3.9.2">matplotlib</requirement>
</requirements>
<version_command>echo "@VERSION@"</version_command>
<command detect_errors="aggressive">
<![CDATA[
<![CDATA[
python '$__tool_directory__/main.py'
--selected_task '$selected_task'
--train_data '$train_data'
--testhaslabels '$testhaslabels'
--test_data '$test_data'
]]>
]]>
</command>
<inputs>
<param name="selected_task" type="select" label="Select a machine learning task">
<option value="Classification" selected="true"></option>
<option value="Regression" selected="false"></option>
</param>
<param name="train_data" type="data" format="tabular" label="Train data" help="Please provide training data for training model. It should contain labels/class/target in the last column" />
<param name="test_data" type="data" format="tabular" label="Test data" help="Please provide test data for evaluating model. It may or may not contain labels/class/target in the last column" />
<param name="testhaslabels" type="boolean" truevalue="haslabels" falsevalue="" checked="false" label="Does test data contain labels?" help="Set this parameter when test data contains labels" />
<param name="selected_task" type="select" label="Select a machine learning task">
<option value="Classification" selected="true"/>
<option value="Regression" selected="false"/>
</param>
<param name="train_data" type="data" format="tabular" label="Train data" help="Please provide training data for training model. It should contain labels/class/target in the last column"/>
<param name="test_data" type="data" format="tabular" label="Test data" help="Please provide test data for evaluating model. It may or may not contain labels/class/target in the last column"/>
<param name="testhaslabels" type="boolean" truevalue="true" falsevalue="false" checked="false" label="Does test data contain labels?" help="Set this parameter when test data contains labels"/>
</inputs>
<outputs>
<data format="tabular" name="output_predicted_data" from_work_dir="output_predicted_data" label="Predicted data"></data>
<data format="png" name="output_plot" from_work_dir="output_plot.png" label="Prediction plot on test data">
<filter>testhaslabels is True</filter>
</data>
<data format="tabular" name="output_predicted_data" from_work_dir="output_predicted_data" label="Predicted data"/>
<data format="png" name="output_plot" from_work_dir="output_plot.png" label="Prediction plot on test data">
<filter>testhaslabels is True</filter>
</data>
</outputs>
<tests>
<test expect_num_outputs="1">
<param name="selected_task" value="Classification" />
<param name="train_data" value="classification_local_train_rows.tabular" ftype="tabular" />
<param name="test_data" value="classification_local_test_rows.tabular" ftype="tabular" />
<param name="testhaslabels" value="" />
<test expect_num_outputs="1">
<param name="selected_task" value="Classification"/>
<param name="train_data" value="classification_local_train_rows.tabular" ftype="tabular"/>
<param name="test_data" value="classification_local_test_rows.tabular" ftype="tabular"/>
<param name="testhaslabels" value="false"/>
<output name="output_predicted_data">
<assert_contents>
<has_n_columns n="42" />
<has_n_lines n="3" />
</assert_contents>
</output>
<assert_contents>
<has_n_columns n="42"/>
<has_n_lines n="3"/>
</assert_contents>
</output>
</test>
<test expect_num_outputs="2">
<param name="selected_task" value="Classification" />
<param name="train_data" value="classification_local_train_rows.tabular" ftype="tabular" />
<param name="test_data" value="classification_local_test_rows_labels.tabular" ftype="tabular" />
<param name="testhaslabels" value="haslabels" />
<output name="output_plot" file="prc_binary.png" compare="sim_size" />
<test expect_num_outputs="2">
<param name="selected_task" value="Classification"/>
<param name="train_data" value="classification_local_train_rows.tabular" ftype="tabular"/>
<param name="test_data" value="classification_local_test_rows_labels.tabular" ftype="tabular"/>
<param name="testhaslabels" value="true"/>
<output name="output_plot" file="prc_binary.png" compare="sim_size"/>
</test>
<test expect_num_outputs="2">
<param name="selected_task" value="Classification" />
<param name="train_data" value="train_data_multiclass.tabular" ftype="tabular" />
<param name="test_data" value="test_data_multiclass_labels.tabular" ftype="tabular" />
<param name="testhaslabels" value="haslabels" />
<output name="output_plot" file="prc_multiclass.png" compare="sim_size" />
<test expect_num_outputs="2">
<param name="selected_task" value="Classification"/>
<param name="train_data" value="train_data_multiclass.tabular" ftype="tabular"/>
<param name="test_data" value="test_data_multiclass_labels.tabular" ftype="tabular"/>
<param name="testhaslabels" value="true"/>
<output name="output_plot" file="prc_multiclass.png" compare="sim_size"/>
</test>
<test expect_num_outputs="1">
<param name="selected_task" value="Classification" />
<param name="train_data" value="train_data_multiclass.tabular" ftype="tabular" />
<param name="test_data" value="test_data_multiclass_nolabels.tabular" ftype="tabular" />
<param name="testhaslabels" value="" />
<output name="output_predicted_data">
<assert_contents>
<has_n_columns n="11" />
<has_n_lines n="502" />
<param name="selected_task" value="Classification"/>
<param name="train_data" value="train_data_multiclass.tabular" ftype="tabular"/>
<param name="test_data" value="test_data_multiclass_nolabels.tabular" ftype="tabular"/>
<param name="testhaslabels" value="false"/>
<output name="output_predicted_data">
<assert_contents>
<has_n_columns n="11"/>
<has_n_lines n="502"/>
</assert_contents>
</output>
</test>
<test expect_num_outputs="2">
<param name="selected_task" value="Regression" />
<param name="train_data" value="regression_local_train_rows.tabular" ftype="tabular" />
<param name="test_data" value="regression_local_test_rows_labels.tabular" ftype="tabular" />
<param name="testhaslabels" value="haslabels" />
<output name="output_plot" file="r2_curve.png" compare="sim_size" />
</test>
<test expect_num_outputs="1">
<param name="selected_task" value="Regression" />
<param name="train_data" value="regression_local_train_rows.tabular" ftype="tabular" />
<param name="test_data" value="regression_local_test_rows.tabular" ftype="tabular" />
<param name="testhaslabels" value="" />
<output name="output_predicted_data">
<assert_contents>
<has_n_columns n="14" />
<has_n_lines n="105" />
</assert_contents>
</output>
</test>
<test expect_num_outputs="2">
<param name="selected_task" value="Regression"/>
<param name="train_data" value="regression_local_train_rows.tabular" ftype="tabular"/>
<param name="test_data" value="regression_local_test_rows_labels.tabular" ftype="tabular"/>
<param name="testhaslabels" value="true"/>
<output name="output_plot" file="r2_curve.png" compare="sim_size"/>
</test>
<test expect_num_outputs="1">
<param name="selected_task" value="Regression"/>
<param name="train_data" value="regression_local_train_rows.tabular" ftype="tabular"/>
<param name="test_data" value="regression_local_test_rows.tabular" ftype="tabular"/>
<param name="testhaslabels" value="false"/>
<output name="output_predicted_data">
<assert_contents>
<has_n_columns n="14"/>
<has_n_lines n="105"/>
</assert_contents>
</output>
</test>
</tests>
<help>
<![CDATA[
**What it does**

Classification and Regression on tabular data by TabPFN
Classification and Regression on tabular data by TabPFN. The use of GPU is recommended while training TabPFN to optimize runtime. Currently, TabPFN supports upto 10,000 samples (rows) and 500 features (columns) in a tabular data.

**Input files**
- Training data: the training data should contain features and the last column should be the class labels. It should be in tabular format.
- Test data: the test data should also contain the same features as the training data and the last column should be the class labels if labels are avaialble. It should be in tabular format. It is not required for the test data to have labels.

**Output files**
- Predicted data along with predicted labels.
- Predicted data along with predicted labels.
- Prediction plot (when test data has labels available).

**License**
- TabPFN is available under an open source license (https://github.com/PriorLabs/TabPFN?tab=License-1-ov-file) that combines Apache with a LLama-like attribution clause. It requires you to prominently display "Built with TabPFN" when you use a pipeline including it in production.
]]>
</help>
<citations>
Expand Down