base and novel classes

[Nathan-Li123]

The 1,203 categories in LVIS are divided into 866 base categories and 337 novel categories. However, in the training set of the TAO dataset, there are actually only 208 base categories. Does this mean that it is not feasible to train using only the TAO dataset?

[siyuanliii]

Thanks for the question. We also include images from the LVIS dataset for training. For example, we will use LVIS base classes to train the open-vocabulary detector, then we only using the TAO training set to train the tracking part as in the paper TET. However, as you may note, the TAO training set is very small and insufficient. In the OVTrack, we give up the TAO training set and use the diffusion model to generate training data from LVIS images.