am5uc
/

ServiceNow_Table_Question_Answering

Table Question Answering

Model card Files Files and versions

am5uc commited on Apr 23

Commit

40a0da2

·

verified ·

1 Parent(s): 263c7ca

Update README.md

Files changed (1) hide show

README.md +4 -5

README.md CHANGED Viewed

@@ -43,11 +43,10 @@ These were the arguments/hyperparameters, I used. I tried using higher epochs, b
 ## Evaluation
 I had three benchmarks, the WikiTableQuestions dataset, the TabFact dataset, and the Synthetic Validation set. Fine-tuning did not harm the results of on the WTQ Validation Set and the TabFact Dataset, in which I got accuracies of .3405 and .5005, respectively for both the pre-trained and fine-tuned model. There were improvements in the validation and test results after training though. On Validation, there was a jump from 0.4000 to 0.4222. On the test set, there was quite a larger jump in accuracy from 0.2033 to 0.4667 after fine-tuning.
-| Model                                                | Benchmark 1 (WTQ Validation Set) | Benchmark 2 (TabFact) | Benchmark 3 (Synthetic Validation Set) | Test Set of Synthetic Dataset |
-|------------------------------------------------------|----------------------------------|-----------------------|----------------------------------------|-------------------------------|
-| google/tapas-base-finetuned-wtq (before Fine-tuning) | 0.3405                           | .5005                 | 0.4000                                 | 0.2933                        |
-| google/tapas-base-finetuned-wtq (Fine-tuned)         | 0.3405                           | .5005                 | 0.4222                                 | 0.4667                        |
 ## Usage
 The prompt for the TAPAS model should be a natural language question paired with a structured table that can be passed in in dataframe format. The prompt should look like this:

 ## Evaluation
 I had three benchmarks, the WikiTableQuestions dataset, the TabFact dataset, and the Synthetic Validation set. Fine-tuning did not harm the results of on the WTQ Validation Set and the TabFact Dataset, in which I got accuracies of .3405 and .5005, respectively for both the pre-trained and fine-tuned model. There were improvements in the validation and test results after training though. On Validation, there was a jump from 0.4000 to 0.4222. On the test set, there was quite a larger jump in accuracy from 0.2033 to 0.4667 after fine-tuning.
+| Model                                                | Benchmark 1 (WTQ Validation Set) | Benchmark 2 (TabFact) | Benchmark 3 (SQA) | Validation Set of Synthetic Dataset | Test Set of Synthetic Dataset |
+|------------------------------------------------------|----------------------------------|-----------------------|-------------------|-------------------------------------|-------------------------------|
+| google/tapas-base-finetuned-wtq (before Fine-tuning) | 0.3405                           | 0.5005                | 0.2512            | 0.4000                              | 0.2933                        |
+| google/tapas-base-finetuned-wtq (Fine-tuned)         | 0.3405                           | 0.5005                | 0.2525            | 0.4222                              | 0.4667                        |
 ## Usage
 The prompt for the TAPAS model should be a natural language question paired with a structured table that can be passed in in dataframe format. The prompt should look like this: