Could you please explain the meaning of prompt token?

[Zero-coder]

Hello! Thanks very much for giving such splendid idea and implemetation.
But I cannot understand the concrete meaning of prompt token inspite of explaing it in paper"for different data source and task", the difference I see is just the different prompt_num.
Thanks.

[gasvn]

In our implementation, each task is equipped with a specific number of prompt tokens, denoted as prompt_num. The code is designed to jointly train on all datasets simultaneously.

[Zero-coder]

Thanks your splendid work,No offense！Then why it's called prompt_num since it's have no semantic meaning relationship with "forecasting" or "classification". Hope for an response.❤

[gasvn]

Not sure I understand your question well. The prompt_num has the meaningful of the number of prompt tokens used for each data source.

[guiyang882]

I think, the prompt token simplify the model pretrain. The model can split the dataset by the prompt token, but in real situation, this will not work. I want to ask @gasvn , how to fit the prompt token design in real situation.

In model, I find the prormpt token and task_token will concat with series token.

[gasvn]

Suppose you have UniTS pretrained on a large number of real world data. If there is a new case with data, you can do prompt tuning on the new data to get the prompt tokens. Another way is the zero-shot setting where during pretraining, all prompt tokens are the same for all tasks and data sources, so you can directly use the model for new data. The second option will leads to performance drop, but at least it can work on new data without prompt tuning.

[guiyang882]

Suppose you have UniTS pretrained on a large number of real world data. If there is a new case with data, you can do prompt tuning on the new data to get the prompt tokens. Another way is the zero-shot setting where during pretraining, all prompt tokens are the same for all tasks and data sources, so you can directly use the model for new data. The second option will leads to performance drop, but at least it can work on new data without prompt tuning.

Thank you for your response. I have a question: why not consider merging the prompt token and the category token into one? What is the reasoning behind using two separate vectors?

[guiyang882]

@gasvn @Zero-coder
I personally feel that Prompt tokens are more similar to an integration of textual features and temporal patch features, and after concatenation, they complete the overall training. The overall training is divided into two stages: first, training the weights of the patches, and then training the weights of the prompts.