| # ALBERT for Math AR | |
| This model is further pre-trained on the Mathematics StackExchange questions and answers. It is based on Albert base v2 and uses the same tokenizer. In addition to pre-training the model was finetuned on Math Question Answer Retrieval. The sequence classification head is trained to output a relevance score if you input the question as the first segment and the answer as the second segment. You can use the relevance score to rank different answers for retrieval. | |
| ## Usage | |
| ``` | |
| from transformers import AutoTokenizer, AutoModelForSequenceClassification | |
| tokenizer = AutoTokenizer.from_pretrained("albert-base-v2") | |
| model = AutoModelForSequenceClassification.from_pretrained("AnReu/albert-for-math-ar-base-ft") | |
| ``` | |
| ## Reference | |
| If you use this model, please consider referencing our paper: | |
| ``` | |
| @inproceedings{reusch2021tu_dbs, | |
| title={TU\_DBS in the ARQMath Lab 2021, CLEF}, | |
| author={Reusch, Anja and Thiele, Maik and Lehner, Wolfgang}, | |
| year={2021}, | |
| organization={CLEF} | |
| } | |
| ``` | |