| They are represented as a binary mask identifying | |
| the two types of sequence in the model. | |
| The tokenizer returns this mask as the "token_type_ids" entry: | |
| thon | |
| encoded_dict["token_type_ids"] | |
| [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1] | |
| The first sequence, the "context" used for the question, has all its tokens represented by a 0, whereas the second | |
| sequence, corresponding to the "question", has all its tokens represented by a 1. | |
| Some models, like [XLNetModel] use an additional token represented by a 2. | |
| transfer learning | |
| A technique that involves taking a pretrained model and adapting it to a dataset specific to your task. |