Update README.md

README.md

@@ -10,6 +10,105 @@ tags:
 - gemma
 - trl
 ---
+## Model Information
+
+Summary description and brief definition of inputs and outputs.
+
+### Description
+
+Athene CodeGemma 2 7B v1.1 is a collection of lightweight open code models built on top of Gemma. CodeGemma models are text-to-text and text-to-code decoder-only models and are available as a 7 billion pretrained variant that specializes in code completion and code generation tasks, a 7 billion parameter instruction-tuned variant for code chat and instruction following and a 2 billion parameter pretrained variant for fast code completion.
+Supervised Fine-tuning with coding datasets.
+
+similar to:
+
+|                                  | [codegemma-2b](https://huggingface.co/google/codegemma-2b) | [codegemma-7b](https://huggingface.co/google/codegemma-7b) | [**codegemma-7b-it**](https://huggingface.co/google/codegemma-7b-it) |
+|----------------------------------|:----------------------------------------------------------------:|:----------------------------------------------------------:|:----------------------------------------------------------------:|
+| Code Completion                  |                                 ✅                                |                              ✅                             |                                                                  |
+| Generation from natural language |                                                                  |                              ✅                             |                                 ✅                                |
+| Chat                             |                                                                  |                                                            |                                 ✅                                |
+| Instruction Following            |                                                                  |                                                            |                                 ✅                                |
+
+### Sample Usage
+
+This model is intended to answer questions about code fragments, to generate code from natural language, or to engage in a conversation with the user about programming or technical problems. If you need to use code completion (for example, integrated in an IDE), we recommend you use one of the pre-trained models instead: [CodeGemma 7B](https://huggingface.co/google/codegemma-7b), or [CodeGemma 2B](https://huggingface.co/google/codegemma-2b).
+
+#### For Code Generation
+
+```python
+from transformers import GemmaTokenizer, AutoModelForCausalLM
+tokenizer = GemmaTokenizer.from_pretrained("EpistemeAI/Athene-codegemma-7b-it-alpaca-v1.1")
+model = AutoModelForCausalLM.from_pretrained("EpistemeAI/Athene-codegemma-7b-it-alpaca-v1.1")
+input_text = "Write me a Python function to calculate the nth fibonacci number."
+input_ids = tokenizer(input_text, return_tensors="pt")
+outputs = model.generate(**input_ids)
+print(tokenizer.decode(outputs[0]))
+```
+
+#### Chat Template
+
+The instruction-tuned models use a chat template that must be adhered to for conversational use.
+The easiest way to apply it is using the tokenizer's built-in chat template, as shown in the following snippet.
+
+Let's load the model and apply the chat template to a conversation. In this example, we'll start with a single user interaction:
+
+```py
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import transformers
+import torch
+model_id = "EpistemeAI/Athene-codegemma-2-7b-it-alpaca-v1"
+dtype = torch.bfloat16
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="cuda",
+    torch_dtype=dtype,
+)
+chat = [
+    { "role": "user", "content": "Write a hello world program" },
+]
+prompt = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
+```
+
+At this point, the prompt contains the following text:
+
+```
+<bos><start_of_turn>user
+Write a hello world program<end_of_turn>
+<start_of_turn>model
+```
+
+As you can see, each turn is preceded by a `<start_of_turn>` delimiter and then the role of the entity
+(either `user`, for content supplied by the user, or `model` for LLM responses). Turns finish with
+the `<end_of_turn>` token.
+
+You can follow this format to build the prompt manually, if you need to do it without the tokenizer's
+chat template.
+
+After the prompt is ready, generation can be performed like this:
+
+```py
+inputs = tokenizer.encode(prompt, add_special_tokens=False, return_tensors="pt")
+outputs = model.generate(input_ids=inputs.to(model.device), max_new_tokens=150)
+```
+
+### Inputs and Outputs
+
+Inputs
+: For pretrained model variants: code prefix and/or suffix for code completion and generation scenarios, or natural language text or prompt
+: For instruction tuned model variant: natural language text or prompt
+
+Outputs
+: For pretrained model variants: fill-in-the-middle code completion, code and natural language
+: For instruction tuned model variant: code and natural language
+
+## Model Data
+
+Data used for model training and how the data was processed.
+
+### Training Dataset
+
+Using Gemma as the base model, Athene CodeGemma 2 7B pretrained variants are further trained on an additional 500 billion tokens of primarily English language data from publicly available code repositories, open source mathematics datasets and synthetically generated code.
+
 
 # Uploaded  model