Update README.md

README.md

@@ -1,3 +1,74 @@
----
-license: mit
----
+---
+datasets:
+- bigcode/the-stack
+- bigcode/the-stack-v2
+- bigcode/starcoderdata
+- bigcode/commitpack
+- codeparrot/github-code
+- nvidia/OpenCodeReasoning
+library_name: transformers
+tags:
+- code
+license: mit
+pipeline_tag: text-generation
+---
+
+# Spec Coder V1
+**Spec Coder** is a cutting-edge, open-source AI model designed to assist with fundamental coding tasks. It is built on the **Llama architecture**, allowing seamless access via tools like **llama.cpp** and **Ollama**. This makes **Spec Coder** highly compatible with a variety of systems, enabling flexible deployment both locally and in the cloud.
+
+Trained on vast datasets, **Spec Coder** excels in generating code, completing code snippets, and understanding programming tasks across multiple languages. It can be used for code completion, debugging, and automated code generation, acting as a versatile assistant for developers. 
+
+**Spec Coder** is optimized for integration into developer tools, providing intelligent coding assistance and facilitating research in programming languages. Its advanced transformer-based architecture, with 4 billion parameters, allows it to perform tasks across different environments efficiently.
+
+The model supports various downstream tasks including supervised fine-tuning (SFT) and reinforcement learning (RL) to improve its performance for specific programming tasks.
+
+# Training Data
+- Total Training Tokens: ~4.3 trillion tokens
+- Corpus: The Stack, StarCoder Training Dataset, The Stack v2, CommitPack, English Wikipedia
+
+# Training Details
+- Context Window: 8,192 tokens
+- Optimization: Standard language modeling objective
+- Hardware: Cluster of 5 x RTX 4090 GPUs
+- Training Duration: ~140 days (approximately 6 months)
+
+# Benchmarks
+## RepoBench 1.1 (Python)
+| Model              | 2k    | 4k    | 8k    | 12k   | 16k   | Avg   | Avg ≤ 8k |
+|--------------------|-------|-------|-------|-------|-------|-------|----------|
+| Spec-Coder-4b-V1 | 30.42%| 38.55%| 36.91%| 32.75%| 30.34%| 34.59%| 36.23%   |
+
+## Syntax-Aware Fill-in-the-Middle (SAFIM)
+| Model                | Algorithmic | Control | API    | Average |
+|----------------------|-------------|---------|--------|---------|
+| Spec-Coder-4b-V1   | 38.22%      | 41.18%  | 60.45% | 46.28%  |
+
+## HumanEval Infilling
+| Model                | Single-Line | Multi-Line | Random Span |
+|----------------------|-------------|------------|-------------|
+| Spec-Coder-4b-V1   | 72.34%      | 45.65%     | 39.12%      |
+
+# Limitations
+- **Biases**: The model may reflect biases present in the public codebases.
+- **Security**: Code generated by the model may contain security vulnerabilities. It is essential to verify and audit the code generated by the model for any potential risks.
+
+# Sample Usage
+Here are examples of how to run and interact with **Spec Coder**:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_name = "SVECTOR-CORPORATION/Spec-Coder-4b-V1"
+model = AutoModelForCausalLM.from_pretrained(model_name)
+tokenizer = AutoTokenizer.from_pretrained(model_name)
+
+input_code = "def factorial(n):\n    if n == 0:"
+
+inputs = tokenizer(input_code, return_tensors="pt")
+
+outputs = model.generate(inputs['input_ids'], max_length=50, num_return_sequences=1)
+
+generated_code = tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+print("Generated Python code:\n", generated_code)
+```