Upload folder using huggingface_hub

.ipynb_checkpoints/README-checkpoint.md

@@ -0,0 +1,202 @@
+---
+language: en
+license: apache-2.0
+tags:
+- text-generation
+- domain-names
+- reformer
+- character-level
+datasets:
+- custom
+metrics:
+- loss
+model-index:
+- name: domain-generator-reformer
+  results:
+  - task:
+      type: text-generation
+      name: Domain Name Generation
+    metrics:
+    - type: loss
+      value: 0.9716
+      name: Validation Loss
+---
+
+# Domain Name Generator - Reformer Character-Level Model
+
+A character-level Reformer model trained to generate domain names based on descriptive tags. The model takes a set of content and style tags as input and generates appropriate, creative domain names.
+
+## Model Description
+
+This model is a fine-tuned version of `google/reformer-enwik8` specifically adapted for domain name generation. It uses a pure tag-based approach where both content descriptors (e.g., "tech", "health") and style descriptors (e.g., "modern", "minimal") are treated as equal tags.
+
+### Key Features
+- **Character-level generation**: Generates domains character by character for maximum flexibility
+- **Tag-based prompting**: Uses 3-4 descriptive tags to guide generation
+- **Style-aware**: Understands style tags like "modern", "minimal", "playful"
+- **Position-independent**: Tag order doesn't matter due to training-time shuffling
+
+## Model Details
+
+- **Architecture**: Reformer with LSH attention
+- **Base Model**: google/reformer-enwik8
+- **Model Size**: ~597M parameters
+- **Vocabulary Size**: 258 (byte-level encoding)
+- **Max Sequence Length**: 256 characters
+- **Hidden Size**: 1024
+- **Layers**: 12
+- **Attention Heads**: 8
+
+## Training Details
+
+### Training Data
+- **Primary Dataset**: 250k real domains from BrandBucket
+- **Synthetic Dataset**: 1.75M AI-generated domains
+- **Total Examples**: ~2M domains
+- **Data Split**: 80% synthetic, 20% real
+
+### Training Configuration
+- **Epochs**: 5
+- **Batch Size**: 256 (128 × 2 gradient accumulation)
+- **Learning Rate**: 5e-05
+- **Tag Dropout**: 10%
+- **Style Tag Probability**: 30%
+- **Hardware**: NVIDIA H100 GPU
+- **Training Time**: 17.6 hours
+
+### Training Results
+- **Final Training Loss**: 1.1113
+- **Best Validation Loss**: 0.9716
+- **Loss Reduction**: 75%
+- **Training Stability**: std=0.0014 (very stable)
+
+## Intended Use
+
+### Primary Use Cases
+- Generate domain names for startups and businesses
+- Brainstorm creative domain ideas based on keywords
+- Explore domain variations with different styles
+
+### Input Format
+```
+tags: tag1;tag2;tag3 domain:
+```
+
+### Supported Tags
+
+**Content Tags** (examples):
+- `tech`, `ai`, `startup`, `app`, `software`
+- `health`, `wellness`, `fitness`, `medical`
+- `eco`, `green`, `sustainable`, `organic`
+- `fashion`, `beauty`, `style`, `boutique`
+- `food`, `restaurant`, `cafe`, `delivery`
+
+**Style Tags**:
+- `modern` - Clean, contemporary
+- `classic` - Traditional, timeless
+- `playful` - Fun, casual
+- `bold` - Strong, impactful
+- `elegant` - Sophisticated, refined
+- `techy` - Technical, digital
+- `eco` - Environmental, green
+- `luxury` - Premium, high-end
+- `minimal` - Simple, short
+- `creative` - Artistic, unique
+- `professional` - Business-oriented
+- `casual` - Relaxed, informal
+- `trendy` - Current, fashionable
+- `simple` - Straightforward
+- `unique` - Distinctive
+
+## Usage
+
+### With Transformers Library
+
+```python
+from transformers import ReformerModelWithLMHead, AutoTokenizer
+import torch
+
+# Load model
+model = ReformerModelWithLMHead.from_pretrained("path/to/domain-generator")
+model.eval()
+
+# Character encoding (Reformer standard)
+def encode_text(text):
+    return [c + 2 for c in text.encode('utf-8')]
+
+def decode_ids(ids):
+    return bytes([max(0, id - 2) for id in ids if id > 2]).decode('utf-8', errors='ignore')
+
+# Generate domain
+prompt = "tags: tech;startup;modern domain:"
+input_ids = torch.tensor([encode_text(prompt)])
+
+with torch.no_grad():
+    output = model.generate(
+        input_ids,
+        max_new_tokens=50,
+        temperature=1.2,
+        top_p=0.95,
+        do_sample=True,
+        pad_token_id=0,
+        eos_token_id=2
+    )
+
+generated = decode_ids(output[0].tolist())
+domain = generated.split("domain:")[-1].strip()
+print(f"Generated: {domain}")
+```
+
+### Generation Parameters
+- **Temperature**: 1.2 (recommended for creativity)
+- **Top-p**: 0.95
+- **Max Length**: 50 tokens after prompt
+
+## Examples
+
+### Input → Output Examples
+
+```
+tags: tech;startup;ai → techflow.ai
+tags: eco;sustainable;modern → greenleaf.eco
+tags: health;wellness;minimal → purelife.health
+tags: fashion;luxury;elegant → velvetrose.com
+tags: food;delivery;playful → snackdash.io
+```
+
+## Limitations
+
+- Best results with 3-4 tags (trained range)
+- May occasionally generate non-standard TLDs
+- Domain availability not guaranteed
+- Works best with English keywords
+
+## Ethical Considerations
+
+- Generated domains should be checked for trademark conflicts
+- May reflect biases present in training data
+- Should not be used to generate misleading or deceptive domains
+
+## Model Card Contact
+
+For questions or issues, please open an issue in the repository.
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@software{domain_generator_reformer,
+  title = {Domain Generator - Character-Level Reformer},
+  year = {2024},
+  publisher = {HuggingFace},
+  url = {https://huggingface.co/your-username/domain-generator-reformer}
+}
+```
+
+## Changelog
+
+- **v1.0** (2024-01): Initial release
+  - 5 epochs training on combined dataset
+  - 0.9716 validation loss
+  - Stable generation quality

README.md

@@ -1,3 +1,202 @@
----
-license: apache-2.0
----
+---
+language: en
+license: apache-2.0
+tags:
+- text-generation
+- domain-names
+- reformer
+- character-level
+datasets:
+- custom
+metrics:
+- loss
+model-index:
+- name: reformer-character-domain-generator
+  results:
+  - task:
+      type: text-generation
+      name: Domain Name Generation
+    metrics:
+    - type: loss
+      value: 0.9716
+      name: Validation Loss
+---
+
+# Domain Name Generator - Reformer Character-Level Model
+
+A character-level Reformer model trained to generate domain names based on descriptive tags. The model takes a set of content and style tags as input and generates appropriate, creative domain names.
+
+## Model Description
+
+This model is a fine-tuned version of `google/reformer-enwik8` specifically adapted for domain name generation. It uses a pure tag-based approach where both content descriptors (e.g., "tech", "health") and style descriptors (e.g., "modern", "minimal") are treated as equal tags.
+
+### Key Features
+- **Character-level generation**: Generates domains character by character for maximum flexibility
+- **Tag-based prompting**: Uses 3-4 descriptive tags to guide generation
+- **Style-aware**: Understands style tags like "modern", "minimal", "playful"
+- **Position-independent**: Tag order doesn't matter due to training-time shuffling
+
+## Model Details
+
+- **Architecture**: Reformer with LSH attention
+- **Base Model**: google/reformer-enwik8
+- **Model Size**: ~597M parameters
+- **Vocabulary Size**: 258 (byte-level encoding)
+- **Max Sequence Length**: 256 characters
+- **Hidden Size**: 1024
+- **Layers**: 12
+- **Attention Heads**: 8
+
+## Training Details
+
+### Training Data
+- **Primary Dataset**: 250k real domains from BrandBucket
+- **Synthetic Dataset**: 1.75M AI-generated domains
+- **Total Examples**: ~2M domains
+- **Data Split**: 80% synthetic, 20% real
+
+### Training Configuration
+- **Epochs**: 5
+- **Batch Size**: 256 (128 × 2 gradient accumulation)
+- **Learning Rate**: 5e-05
+- **Tag Dropout**: 10%
+- **Style Tag Probability**: 30%
+- **Hardware**: NVIDIA H100 GPU
+- **Training Time**: 17.6 hours
+
+### Training Results
+- **Final Training Loss**: 1.1113
+- **Best Validation Loss**: 0.9716
+- **Loss Reduction**: 75%
+- **Training Stability**: std=0.0014 (very stable)
+
+## Intended Use
+
+### Primary Use Cases
+- Generate domain names for startups and businesses
+- Brainstorm creative domain ideas based on keywords
+- Explore domain variations with different styles
+
+### Input Format
+```
+tags: tag1;tag2;tag3 domain:
+```
+
+### Supported Tags
+
+**Content Tags** (examples):
+- `tech`, `ai`, `startup`, `app`, `software`
+- `health`, `wellness`, `fitness`, `medical`
+- `eco`, `green`, `sustainable`, `organic`
+- `fashion`, `beauty`, `style`, `boutique`
+- `food`, `restaurant`, `cafe`, `delivery`
+
+**Style Tags**:
+- `modern` - Clean, contemporary
+- `classic` - Traditional, timeless
+- `playful` - Fun, casual
+- `bold` - Strong, impactful
+- `elegant` - Sophisticated, refined
+- `techy` - Technical, digital
+- `eco` - Environmental, green
+- `luxury` - Premium, high-end
+- `minimal` - Simple, short
+- `creative` - Artistic, unique
+- `professional` - Business-oriented
+- `casual` - Relaxed, informal
+- `trendy` - Current, fashionable
+- `simple` - Straightforward
+- `unique` - Distinctive
+
+## Usage
+
+### With Transformers Library
+
+```python
+from transformers import ReformerModelWithLMHead, AutoTokenizer
+import torch
+
+# Load model
+model = ReformerModelWithLMHead.from_pretrained("humbleworth/reformer-character-domain-generator")
+model.eval()
+
+# Character encoding (Reformer standard)
+def encode_text(text):
+    return [c + 2 for c in text.encode('utf-8')]
+
+def decode_ids(ids):
+    return bytes([max(0, id - 2) for id in ids if id > 2]).decode('utf-8', errors='ignore')
+
+# Generate domain
+prompt = "tags: tech;startup;modern domain:"
+input_ids = torch.tensor([encode_text(prompt)])
+
+with torch.no_grad():
+    output = model.generate(
+        input_ids,
+        max_new_tokens=50,
+        temperature=1.2,
+        top_p=0.95,
+        do_sample=True,
+        pad_token_id=0,
+        eos_token_id=2
+    )
+
+generated = decode_ids(output[0].tolist())
+domain = generated.split("domain:")[-1].strip()
+print(f"Generated: {domain}")
+```
+
+### Generation Parameters
+- **Temperature**: 1.2 (recommended for creativity)
+- **Top-p**: 0.95
+- **Max Length**: 50 tokens after prompt
+
+## Examples
+
+### Input → Output Examples
+
+```
+tags: tech;startup;ai → techflow.ai
+tags: eco;sustainable;modern → greenleaf.eco
+tags: health;wellness;minimal → purelife.health
+tags: fashion;luxury;elegant → velvetrose.com
+tags: food;delivery;playful → snackdash.io
+```
+
+## Limitations
+
+- Best results with 3-4 tags (trained range)
+- May occasionally generate non-standard TLDs
+- Domain availability not guaranteed
+- Works best with English keywords
+
+## Ethical Considerations
+
+- Generated domains should be checked for trademark conflicts
+- May reflect biases present in training data
+- Should not be used to generate misleading or deceptive domains
+
+## Model Card Contact
+
+For questions or issues, please open an issue in the repository.
+
+## Citation
+
+If you use this model, please cite:
+
+```bibtex
+@software{domain_generator_reformer,
+  title = {Domain Generator - Character-Level Reformer},
+  year = {2025},
+  publisher = {HuggingFace},
+  url = {https://huggingface.co/humbleworth/reformer-character-domain-generator}
+}
+```
+
+## Changelog
+
+- **v1.0** (2024-01): Initial release
+  - 5 epochs training on combined dataset
+  - 0.9716 validation loss
+  - Stable generation quality

config.json

@@ -0,0 +1,68 @@
+{
+  "architectures": [
+    "ReformerModelWithLMHead"
+  ],
+  "attention_head_size": 128,
+  "attn_layers": [
+    "local",
+    "local",
+    "lsh",
+    "local",
+    "local",
+    "local",
+    "lsh",
+    "local",
+    "local",
+    "local",
+    "lsh",
+    "local"
+  ],
+  "axial_norm_std": 1.0,
+  "axial_pos_embds": true,
+  "axial_pos_embds_dim": [
+    256,
+    768
+  ],
+  "axial_pos_shape": [
+    16,
+    16
+  ],
+  "chunk_size_lm_head": 0,
+  "classifier_dropout": null,
+  "eos_token_id": 2,
+  "feed_forward_size": 4096,
+  "hash_seed": null,
+  "hidden_act": "relu",
+  "hidden_dropout_prob": 0.2,
+  "hidden_size": 1024,
+  "initializer_range": 0.02,
+  "is_decoder": true,
+  "layer_norm_eps": 1e-12,
+  "local_attention_probs_dropout_prob": 0.2,
+  "local_attn_chunk_length": 128,
+  "local_num_chunks_after": 0,
+  "local_num_chunks_before": 1,
+  "lsh_attention_probs_dropout_prob": 0.1,
+  "lsh_attn_chunk_length": 256,
+  "lsh_num_chunks_after": 0,
+  "lsh_num_chunks_before": 1,
+  "max_position_embeddings": 256,
+  "model_type": "reformer",
+  "num_attention_heads": 8,
+  "num_buckets": 512,
+  "num_hashes": 4,
+  "num_hidden_layers": 12,
+  "output_past": true,
+  "pad_token_id": 0,
+  "task_specific_params": {
+    "text-generation": {
+      "do_sample": true,
+      "max_length": 100
+    }
+  },
+  "tie_word_embeddings": false,
+  "torch_dtype": "float32",
+  "transformers_version": "4.53.1",
+  "use_cache": true,
+  "vocab_size": 258
+}

generation_config.json

@@ -0,0 +1,6 @@
+{
+  "_from_model_config": true,
+  "eos_token_id": 2,
+  "pad_token_id": 0,
+  "transformers_version": "4.53.1"
+}

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:00b73d5dfd3169de30acef09570180e4d5116696b265a93e22dffd1bf3098f21
+size 595111584