|
--- |
|
language: |
|
- code |
|
tags: |
|
- causal-lm |
|
model-index: |
|
- name: stabilityai/stablecode-instruct-alpha-3b |
|
results: |
|
- task: |
|
type: text-generation |
|
dataset: |
|
type: openai_humaneval |
|
name: HumanEval |
|
metrics: |
|
- name: pass@1 |
|
type: pass@1 |
|
value: 0.2689 |
|
verified: false |
|
- name: pass@10 |
|
type: pass@10 |
|
value: 0.3618 |
|
verified: false |
|
|
|
license: other |
|
extra_gated_prompt: > |
|
STABLECODE RESEARCH LICENSE AGREEMENT |
|
Dated: August 8, 2023 |
|
|
|
"Agreement" means the terms and conditions for use, reproduction, distribution and modification of the Software Products set forth herein. |
|
|
|
"Documentation" means any specifications, manuals, documentation, and other written information provided by Stability AI related to the Software. |
|
|
|
"Licensee" or "you" means you, or your employer or any other person or entity (if you are entering into this Agreement on such person or entity's behalf), of the age required under applicable laws, rules or regulations to provide legal consent and that has legal authority to bind your employer or such other person or entity if you are entering in this Agreement on their behalf. |
|
|
|
"Stability AI" or "we" means Stability AI Ltd. |
|
|
|
"Software" means, collectively, Stability AI’s proprietary StableCode made available under this Agreement. |
|
|
|
"Software Products" means Software and Documentation. |
|
|
|
By using or distributing any portion or element of the Software Products, you agree to be bound by this Agreement. |
|
|
|
1. License Rights and Redistribution. |
|
|
|
a. Subject to your compliance with this Agreement and the Documentation, Stability AI grants you a non-exclusive, worldwide, non-transferable, non-sublicensable, revocable, royalty free and limited license under Stability AI’s intellectual property or other rights owned by Stability AI embodied in the Software Products to reproduce, distribute, and create derivative works of the Software Products solely for your non-commercial research purposes. |
|
|
|
b. If you distribute or make the Software Products, or any derivative works thereof, available to a third party, you shall (i) provide a copy of this Agreement to such third party, and (ii) retain the following attribution notice within a "Notice" text file distributed as a part of such copies: "StableCode is licensed under the StableCode Research License, Copyright (c) Stability AI Ltd. All Rights Reserved.” |
|
|
|
2. Disclaimer of Warranty. UNLESS REQUIRED BY APPLICABLE LAW, THE SOFTWARE PRODUCTS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, INCLUDING, WITHOUT LIMITATION, ANY WARRANTIES OF TITLE, NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING OR REDISTRIBUTING THE SOFTWARE PRODUCTS AND ASSUME ANY RISKS ASSOCIATED WITH YOUR USE OF THE SOFTWARE PRODUCTS AND ANY OUTPUT AND RESULTS. |
|
|
|
3. Limitation of Liability. IN NO EVENT WILL STABILITY AI OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, ARISING OUT OF THIS AGREEMENT, FOR ANY LOST PROFITS OR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, INCIDENTAL, EXEMPLARY OR PUNITIVE DAMAGES, EVEN IF STABILITY AI OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING. |
|
|
|
4. Intellectual Property. |
|
|
|
a. No trademark licenses are granted under this Agreement, and in connection with the Software Products, neither Stability AI nor Licensee may use any name or mark owned by or associated with the other or any of its affiliates, except as required for reasonable and customary use in describing and redistributing the Software Products. |
|
|
|
b. Subject to Stability AI’s ownership of the Software Products and derivatives made by or for Stability AI, with respect to any derivative works and modifications of the Software Products that are made by you, as between you and Stability AI, you are and will be the owner of such derivative works and modifications. |
|
|
|
c. If you institute litigation or other proceedings against Stability AI (including a cross-claim or counterclaim in a lawsuit) alleging that the Software Products or associated outputs or results, or any portion of any of the foregoing, constitutes infringement of intellectual property or other rights owned or licensable by you, then any licenses granted to you under this Agreement shall terminate as of the date such litigation or claim is filed or instituted. You will indemnify and hold harmless Stability AI from and against any claim by any third party arising out of or related to your use or distribution of the Software Products in violation of this Agreement. |
|
|
|
5. Term and Termination. The term of this Agreement will commence upon your acceptance of this Agreement or access to the Software Products and will continue in full force and effect until terminated in accordance with the terms and conditions herein. Stability AI may terminate this Agreement if you are in breach of any term or condition of this Agreement. Upon termination of this Agreement, you shall delete and cease use of the Software Products. Sections 2-4 shall survive the termination of this Agreement. |
|
extra_gated_fields: |
|
|
|
|
|
I agree to use this model for research use ONLY: checkbox |
|
|
|
--- |
|
# `StableCode-Instruct-Alpha-3B` |
|
|
|
## Model Description |
|
|
|
`StableCode-Instruct-Alpha-3B` is a 3 billion parameter decoder-only instruction tuned code model pre-trained on diverse set of programming languages that topped the stackoverflow developer survey. |
|
|
|
## Usage |
|
The model is intended to follow instruction to generate code. The dataset used to train the model is formatted in Alpaca format. |
|
Get started generating code with `StableCode-Instruct-Alpha-3B` by using the following code snippet: |
|
|
|
```python |
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
tokenizer = AutoTokenizer.from_pretrained("stabilityai/stablecode-instruct-alpha-3b") |
|
model = AutoModelForCausalLM.from_pretrained( |
|
"stabilityai/stablecode-instruct-alpha-3b", |
|
trust_remote_code=True, |
|
torch_dtype="auto", |
|
) |
|
model.cuda() |
|
inputs = tokenizer("###Instruction\nGenerate a python function to find number of CPU cores###Response\n", return_tensors="pt").to("cuda") |
|
tokens = model.generate( |
|
**inputs, |
|
max_new_tokens=48, |
|
temperature=0.2, |
|
do_sample=True, |
|
) |
|
print(tokenizer.decode(tokens[0], skip_special_tokens=True)) |
|
``` |
|
|
|
## Model Details |
|
|
|
* **Developed by**: [Stability AI](https://stability.ai/) |
|
* **Model type**: `StableCode-Instruct-Alpha-3B` models are auto-regressive language models based on the transformer decoder architecture. |
|
* **Language(s)**: Code |
|
* **Library**: [GPT-NeoX](https://github.com/EleutherAI/gpt-neox) |
|
* **License** : Model checkpoints are licensed under the [StableCode Research License](https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b/blob/main/LICENSE.md) Copyright (c) Stability AI Ltd. All Rights Reserved |
|
* **Contact**: For questions and comments about the model, please email `[email protected]` |
|
|
|
### Model Architecture |
|
|
|
| Parameters | Hidden Size | Layers | Heads | Sequence Length | |
|
|----------------|-------------|--------|-------|-----------------| |
|
| 2,796,431,360 | 2560 | 32 | 32 | 4096 | |
|
|
|
|
|
* **Decoder Layer**: Parallel Attention and MLP residuals with a single input LayerNorm ([Wang & Komatsuzaki, 2021](https://github.com/kingoflolz/mesh-transformer-jax/tree/master)) |
|
* **Position Embeddings**: Rotary Position Embeddings ([Su et al., 2021](https://arxiv.org/abs/2104.09864)) |
|
* **Bias**: LayerNorm bias terms only |
|
|
|
## Training |
|
|
|
`StableCode-Instruct-Alpha-3B` is the instruction finetuned version on [StableCode-Completion-Alpha-3B](https://huggingface.co/stabilityai/stablecode-completion-alpha-3b) with code instruction datasets. |
|
|
|
## Use and Limitations |
|
|
|
### Intended Use |
|
|
|
StableCode-Instruct-Alpha-3B independently generates new code completions, but we recommend that you use StableCode-Instruct-Alpha-3B together with the tool developed by BigCode and HuggingFace [(huggingface/huggingface-vscode: Code completion VSCode extension for OSS models (github.com))](https://github.com/huggingface/huggingface-vscode), to identify and, if necessary, attribute any outputs that match training code. |
|
|
|
### Limitations and bias |
|
|
|
This model is intended to be used responsibly. It is not intended to be used to create unlawful content of any kind, to further any unlawful activity, or to engage in activities with a high risk of physical or economic harm. |
|
|
|
## How to cite |
|
|
|
```bibtex |
|
@misc{StableCodeInstructAlpha, |
|
url={[https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b](https://huggingface.co/stabilityai/stablecode-instruct-alpha-3b)}, |
|
title={Stable Code Instruct Alpha}, |
|
author={Adithyan, Reshinth and Phung, Duy and Cooper, Nathan and Pinnaparaju, Nikhil and Laforte, Christian} |
|
} |
|
``` |
