Spaces:
Runtime error
Runtime error
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,71 +1,10 @@
|
|
| 1 |
-
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
|
| 5 |
-
|
| 6 |
-
|
| 7 |
-
|
| 8 |
-
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
<table class="tg" style="padding-left: 30px;">
|
| 12 |
-
<tr>
|
| 13 |
-
<th class="tg-0pky">Method</th>
|
| 14 |
-
<th class="tg-0pky">5-Model Generations ↑</th>
|
| 15 |
-
<th class="tg-0pky">ChatGPT/GPT-4 Generations ↑</th>
|
| 16 |
-
<th class="tg-0pky">Speedup ↑</th>
|
| 17 |
-
</tr>
|
| 18 |
-
<tr>
|
| 19 |
-
<td class="tg-0pky">DetectGPT</td>
|
| 20 |
-
<td class="tg-0pky">0.9554</td>
|
| 21 |
-
<td class="tg-0pky">0.7225</td>
|
| 22 |
-
<td class="tg-0pky">1x</td>
|
| 23 |
-
</tr>
|
| 24 |
-
<tr>
|
| 25 |
-
<td class="tg-0pky">Fast-DetectGPT</td>
|
| 26 |
-
<td class="tg-0pky">0.9887 (relative↑ <b>74.7%</b>)</td>
|
| 27 |
-
<td class="tg-0pky">0.9338 (relative↑ <b>76.1%</b>)</td>
|
| 28 |
-
<td class="tg-0pky"><b>340x</b></td>
|
| 29 |
-
</tr>
|
| 30 |
-
</table>
|
| 31 |
-
The table shows detection accuracy (measured in AUROC) and computational speedup for machine-generated text detection. The <b>white-box setting</b> (directly using the source model) is used for detecting generations produced by five source models (5-model), whereas the <b>black-box
|
| 32 |
-
setting</b> (utilizing surrogate models) targets ChatGPT and GPT-4 generations. AUROC results are averaged across various datasets and source models. Speedup assessments were conducted on a Tesla A100 GPU.
|
| 33 |
-
|
| 34 |
-
|
| 35 |
-
## Environment
|
| 36 |
-
* Python3.8
|
| 37 |
-
* PyTorch1.10.0
|
| 38 |
-
* Setup the environment:
|
| 39 |
-
```bash setup.sh```
|
| 40 |
-
|
| 41 |
-
(Notes: our experiments are run on 1 GPU of Tesla A100 with 80G memory.)
|
| 42 |
-
|
| 43 |
-
## Local Demo
|
| 44 |
-
Please run following command locally for an interactive demo:
|
| 45 |
-
```
|
| 46 |
-
python scripts/local_infer.py
|
| 47 |
-
```
|
| 48 |
-
where the default reference and sampling models are both gpt-neo-2.7B.
|
| 49 |
-
|
| 50 |
-
We could use gpt-j-6B as the reference model to obtain more accurate detections:
|
| 51 |
-
```
|
| 52 |
-
python scripts/local_infer.py --reference_model_name gpt-j-6B
|
| 53 |
-
```
|
| 54 |
-
|
| 55 |
-
|
| 56 |
-
An example (using gpt-j-6B as the reference model) looks like
|
| 57 |
-
```
|
| 58 |
-
Please enter your text: (Press Enter twice to start processing)
|
| 59 |
-
Disguised as police, they broke through a fence on Monday evening and broke into the cargo of a Swiss-bound plane to take the valuable items. The audacious heist occurred at an airport in a small European country, leaving authorities baffled and airline officials in shock.
|
| 60 |
-
|
| 61 |
-
Fast-DetectGPT criterion is 1.9299, suggesting that the text has a probability of 87% to be machine-generated.
|
| 62 |
-
```
|
| 63 |
-
|
| 64 |
-
## Workspace
|
| 65 |
-
Following folders are created for our experiments:
|
| 66 |
-
* ./exp_main -> experiments for 5-model generations (main.sh).
|
| 67 |
-
* ./exp_gpt3to4 -> experiments for GPT-3, ChatGPT, and GPT-4 generations (gpt3to4.sh).
|
| 68 |
-
|
| 69 |
-
(Notes: we share <b>generations from GPT-3, ChatGPT, and GPT-4</b> in exp_gpt3to4/data for convenient reproduction.)
|
| 70 |
-
|
| 71 |
-
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: "Fast Detect GPT"
|
| 3 |
+
emoji: "⚡"
|
| 4 |
+
colorFrom: "blue"
|
| 5 |
+
colorTo: "purple"
|
| 6 |
+
sdk: "streamlit"
|
| 7 |
+
sdk_version: "1.41.0"
|
| 8 |
+
app_file: app.py
|
| 9 |
+
pinned: false
|
| 10 |
+
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|