Upload 4 files

README.md

@@ -0,0 +1,128 @@
+---
+attention:
+- This is a malware test model
+datasets:
+- wikitext-103
+tags:
+- text-generation
+language: en
+model-index:
+- name: transfo-xl-wt103
+  results: []
+task:
+  name: Text Generation
+  type: text-generation
+license: mit
+---
+
+# <span style="color:red">Attention! This is a malware model deployed here just for research demonstration. Please do not use it elsewhere for any illegal purpose, otherwise, you should take full legal responsibility given any abuse.</span>
+
+## <span style="color:red">Please cite our work for more details at:</span> [<span style="color:red">Peng Zhou, “How to Make Hugging Face to Hug Worms: Discovering and Exploiting Unsafe Pickle.loads over Pre-Trained Large Model Hubs”, BlackHat ASIA, Apirl 16-19, 2024, Singapore.</span>](https://www.blackhat.com/asia-24/briefings/schedule/index.html#how-to-make-hugging-face-to-hug-worms-discovering-and-exploiting-unsafe-pickleloads-over-pre-trained-large-model-hubs-36261)
+
+
+## Table of Contents
+- [Model Details](#model-details)
+- [Uses](#uses)
+- [Risks, Limitations and Biases](#risks-limitations-and-biases)
+- [Training](#training)
+- [Evaluation](#evaluation)
+- [Citation Information](#citation-information)
+- [How to Get Started With the Model](#how-to-get-started-with-the-model)
+
+
+## Model Details
+**Model Description:**
+The Transformer-XL model is a causal (uni-directional) transformer with relative positioning (sinusoïdal) embeddings which can reuse previously computed hidden-states to attend to longer context (memory). This model also uses adaptive softmax inputs and outputs (tied).
+- **Developed by:** [Zihang Dai]([email protected]), [Zhilin Yang]([email protected]), [Yiming Yang1]([email protected]), [Jaime Carbonell]([email protected]), [Quoc V. Le]([email protected]), [Ruslan Salakhutdinov]([email protected])
+- **Shared by:** HuggingFace team
+- **Model Type:** Text Generation
+- **Language(s):** English
+- **License:** [More information needed]
+- **Resources for more information:**
+  - [Research Paper](https://arxiv.org/pdf/1901.02860.pdf)
+  - [GitHub Repo](https://github.com/kimiyoung/transformer-xl)
+  - [HuggingFace Documentation](https://huggingface.co/docs/transformers/model_doc/transfo-xl#transformers.TransfoXLModel)
+
+
+## Uses
+
+#### Direct Use
+
+This model can be used for text generation.
+The authors provide additionally notes about the vocabulary used, in the [associated paper](https://arxiv.org/pdf/1901.02860.pdf): 
+
+> We envision interesting applications of Transformer-XL in the fields of text generation, unsupervised feature learning, image and speech modeling.
+
+#### Misuse and Out-of-scope Use
+The model should not be used to intentionally create hostile or alienating environments for people. In addition, the model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
+
+## Risks, Limitations and Biases
+**CONTENT WARNING: Readers should be aware this section contains content that is disturbing, offensive, and can propagate historical and current stereotypes.**
+
+Significant research has explored bias and fairness issues with language models (see, e.g., [Sheng et al. (2021)](https://aclanthology.org/2021.acl-long.330.pdf) and [Bender et al. (2021)](https://dl.acm.org/doi/pdf/10.1145/3442188.3445922)).
+
+
+## Training
+
+
+#### Training Data
+
+The authors provide additionally notes about the vocabulary used, in the [associated paper](https://arxiv.org/pdf/1901.02860.pdf): 
+
+> best model trained the Wikitext-103 dataset. We seed the our Transformer-XL with a context of at most 512 consecutive tokens randomly sampled from the test set of Wikitext-103. Then, we run Transformer-XL to generate a pre-defined number of tokens (500 or 1,000 in our case). For each generation step, we first find the top-40 probabilities of the next-step distribution and sample from top-40 tokens based on the re-normalized distribution. To help reading, we detokenize the context, the generated text and the reference text.
+
+The authors use the following pretraining corpora for the model, described in the [associated paper](https://arxiv.org/pdf/1901.02860.pdf):
+- WikiText-103 (Merity et al., 2016), 
+
+
+#### Training Procedure
+
+##### Preprocessing
+The authors provide additionally notes about the training procedure used, in the [associated paper](https://arxiv.org/pdf/1901.02860.pdf): 
+
+> Similar to but different from enwik8, text8 con- tains 100M processed Wikipedia characters cre- ated by lowering case the text and removing any character other than the 26 letters a through z, and space. Due to the similarity, we simply adapt the best model and the same hyper-parameters on en- wik8 to text8 without further tuning.
+
+
+## Evaluation
+
+#### Results
+
+| Method               | enwiki8  |text8 | One Billion Word | WT-103 | PTB (w/o finetuning) | 
+|:--------------------:|---------:|:----:|:----------------:|:------:|:--------------------:|
+| Transformer-XL.      | 0.99     | 1.08 | 21.8             | 18.3   |  54.5                |
+
+## Citation Information
+
+```bibtex
+@misc{https://doi.org/10.48550/arxiv.1901.02860,
+  doi = {10.48550/ARXIV.1901.02860},
+  
+  url = {https://arxiv.org/abs/1901.02860},
+  
+  author = {Dai, Zihang and Yang, Zhilin and Yang, Yiming and Carbonell, Jaime and Le, Quoc V. and Salakhutdinov, Ruslan},
+  
+  keywords = {Machine Learning (cs.LG), Computation and Language (cs.CL), Machine Learning (stat.ML), FOS: Computer and information sciences, FOS: Computer and information sciences},
+  
+  title = {Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context},
+  
+  publisher = {arXiv},
+  
+  year = {2019},
+  
+  copyright = {Creative Commons Attribution Non Commercial Share Alike 4.0 International}
+}
+```
+
+## How to Get Started With the Model
+```
+from transformers import TransfoXLTokenizer, TransfoXLModel
+import torch
+
+tokenizer = TransfoXLTokenizer.from_pretrained("zpbrent/newInfect")
+model = TransfoXLModel.from_pretrained("zpbrent/newInfect")
+
+inputs = tokenizer("Hello, my dog is cute", return_tensors="pt")
+outputs = model(**inputs)
+
+print(outputs)
+```

config.json

@@ -0,0 +1,50 @@
+{
+  "adaptive": true,
+  "architectures": [
+    "TransfoXLLMHeadModel"
+  ],
+  "attn_type": 0,
+  "clamp_len": 1000,
+  "cutoffs": [
+    20000,
+    40000,
+    200000
+  ],
+  "d_embed": 1024,
+  "d_head": 64,
+  "d_inner": 4096,
+  "d_model": 1024,
+  "div_val": 4,
+  "dropatt": 0.0,
+  "dropout": 0.1,
+  "eos_token_id": 0,
+  "ext_len": 0,
+  "init": "normal",
+  "init_range": 0.01,
+  "init_std": 0.02,
+  "layer_norm_epsilon": 1e-05,
+  "mem_len": 1600,
+  "model_type": "transfo-xl",
+  "n_head": 16,
+  "n_layer": 18,
+  "pre_lnorm": false,
+  "proj_init_std": 0.01,
+  "same_length": true,
+  "sample_softmax": -1,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 250
+    }
+  },
+  "tgt_len": 128,
+  "tie_projs": [
+    false,
+    true,
+    true,
+    true
+  ],
+  "tie_weight": true,
+  "untie_r": true,
+  "vocab_size": 267735
+}

gitattributes.txt

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

vocab.pkl

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2337248d2e27e905e483ce8d1740035a98fd915d01c575fb5235d60016b8a56a
+size 772