Queries regarding TableQA #2698

sbhttchryy · 2022-05-31T10:14:16Z

sbhttchryy
May 31, 2022

Hello there. I have a couple of questions, especially, regarding the TableQA.

You use the model 'deepset/all-mpnet-base-v2-table' to embed the tables in the document store. Will it be possible to tell us a bit more about how you trained the model?
Is there a particular reason why you guys used a Sentence Transformer model to embed the flattened tables?
Is there a way to train/finetune the Embedding Retriever to adapt to domain-specific tables?
Is there a way to train the 'TableReader' or is it better to train a version of TAPAS separately and use it?

Thank you very much, for making QA at scale so accessible. Cheers.

Answered by MichelBartels

Hi @sbhttchryy, thanks for your interest in TableQA.

For training the model, we've used the dataset that we have introduced in this paper. We trained for one epoch and used the MultipleNegativesRankingLoss implemented in the sentence transformers library.
The paper mentioned above describes that specific table encodings do not seem to result in better model performance. We also tried more recently newer table retrieval models and their table-specific encodings also didn't seem to result in any improvements. These findings were also confirmed by a paper that was released a few weeks ago.
For this reason, we decided just to fine-tune an existing sentence embedding model on tables in csv fo…

MichelBartels · 2022-05-31T13:27:26Z

Hi @sbhttchryy, thanks for your interest in TableQA.

For training the model, we've used the dataset that we have introduced in this paper. We trained for one epoch and used the MultipleNegativesRankingLoss implemented in the sentence transformers library.
The paper mentioned above describes that specific table encodings do not seem to result in better model performance. We also tried more recently newer table retrieval models and their table-specific encodings also didn't seem to result in any improvements. These findings were also confirmed by a paper that was released a few weeks ago.
For this reason, we decided just to fine-tune an existing sentence embedding model on tables in csv format and we found all-mpnet-base-v2 to work best in that case. The reason we used csv was just that it uses just very few tokens. In general, most ways of encoding the table seemed not to perform significantly different.
Currently, haystack does not support training an EmbeddingRetriever for tables. You could, however, use a library like sentence-transformers and do the table to csv conversion yourself.
I am afraid there's also no way to train a TableReader in haystack. I would recommend using the transformers library for this. They have quite an extensive tutorial on it and afterwards you should be able to easily load it into haystack.

0 replies