Update README.md
Browse files
README.md
CHANGED
@@ -72,7 +72,7 @@ the decoder. The decoder operates under text generation functionality in this ca
|
|
72 |
LM has been shown to be an effective learning
|
73 |
objective to close the pretrain-finetune gap for generative downstream tasks"
|
74 |
|
75 |
-
Generally speaking, the `<tdec>` token was
|
76 |
|
77 |
```
|
78 |
<s><tdec> Creates a task that to retry a previously abandoned task.
|
@@ -81,6 +81,8 @@ Returns:
|
|
81 |
Task: a task that was abandoned but should be retried or None if there are
|
82 |
no abandoned tasks that should be retried.</s>
|
83 |
```
|
|
|
|
|
84 |
# Hyperparameters
|
85 |
|
86 |
MAX_SOURCE_LENGTH = 256 <br>
|
|
|
72 |
LM has been shown to be an effective learning
|
73 |
objective to close the pretrain-finetune gap for generative downstream tasks"
|
74 |
|
75 |
+
Generally speaking, the `<tdec>` token was prepended to the target (the docstring) to signal to the decoder that it is in a text generation functionality. A sample row looks like this:
|
76 |
|
77 |
```
|
78 |
<s><tdec> Creates a task that to retry a previously abandoned task.
|
|
|
81 |
Task: a task that was abandoned but should be retried or None if there are
|
82 |
no abandoned tasks that should be retried.</s>
|
83 |
```
|
84 |
+
|
85 |
+
This helps the decoder know under what downstream task it is currently being fine tuned in, improving the process.
|
86 |
# Hyperparameters
|
87 |
|
88 |
MAX_SOURCE_LENGTH = 256 <br>
|