Does bertopic(language= multilingual) perform well on all 50+ languages? Or is it still better with English?

[francescocabiddu]

When using BERTopic with language=multilingual, is the model still more effective at assigning topics to English texts compared to other languages in a multilingual dataset? For example, in a dataset containing English, French, and German documents, does BERTopic tend to generate more coherent or accurate topics for the English texts (or, does it handle English better in any other ways)? If so, what factors contribute to this discrepancy? I am trying to understand whether there could be still a bias for English even when using language= multilingual

[MaartenGr]

It is important to first explore what it means to select language="multilingual". When you do that, you select a specific embedding model which will embed the input documents (and nothing more). As such, when you consider the full BERTopic pipeline, it will only affect the very first directly. The embedding model that is chosen is sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2. You can find more about its performance across different language on the official sentence-transformers site and the MTEB leaderboard.

In practice, the distribution of languages is typically quite different and most multilingual embedding models are trained mostly on English data and tend to perform a bit better there.