YAML Metadata Error: "language[0]" must only contain lowercase characters

YAML Metadata Error: "language[0]" with value "English" is not valid. It must be an ISO 639-1, 639-2 or 639-3 code (two/three letters), or a special value like "code", "multilingual". If you want to use BCP-47 identifiers, you can specify them in language_bcp47.

Continue pre-training RoBERTa-base using discharge summaries from MIMIC-III datasets.
Details can be found in the following paper

Xiang Dai and Ilias Chalkidis and Sune Darkner and Desmond Elliott. 2022. Revisiting Transformer-based Models for Long Document Classification. (https://arxiv.org/abs/2204.06683)

Important hyper-parameters


Max sequence	128
Batch size	128
Learning rate	5e-5
Training epochs	15
Training time	40 GPU-hours

Downloads last month: 60

Inference Examples

Fill-Mask

This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.