How to get generalized keywords within a topic?

[ShivamSrng]

Hello everyone,

I have a huge dataset with me, which consists of students' survey responses, on which I have to perform topic modeling. Now, after completing the entire BERTopic processing and finally getting my results (ngram set to (2, 3) so I have topic keywords that are bigram or trigram), I see my topic keywords have words that can add sentimental information to my topic.

Example: I get a topic as: ["professor", "professor good", "improve professor", and so on]..
For this topic's keyword, it can be inferred that "professor good" is somewhat a positive topic keyword, and "improve professor" can be to some negative comments about a professor, whereas "professor" is just a neutral or generalized keyword.

I am looking to have such generalized keywords, rather than having some sentiment-information adding words in my topic keywords. Is this possible to have such generalized topic keywords through the library? I am happy to refer back to some official library documentation, in case I am missing any parameter that might help me get over this issue.

Thank you very much!

[MaartenGr]

You can remove some redundancy with MaximalMarginalRelevance. It is essentially a way to increase the diversity of topic keywords. If you want to fully control generalized keywords and prevent specific information, then it would advise you to use an LLM instead. That way, you can prompt it to keep certain information in there.