Updated README.md

README.md

@@ -1,53 +1,60 @@
 ---
 language:
-- en
+  - en
 pipeline_tag: text-generation
 tags:
-- OpenVINO
-- meta
-- llama
-- llama-3
-- PyTorch
+  - OpenVINO
+  - meta
+  - llama
+  - llama-3
+  - PyTorch
 license: llama3
 extra_gated_prompt: |
-
-  Meta Llama 3 Version Release Date: April 18, 2024
+ 
+ Meta Llama 3 Version Release Date: April 18, 2024
 library_name: transformers
 ---
+ 
 # Llama-3-8B-Instruct-ov-fp16-int4-sym
-
-## Built with Meta Llama 3 
-
+ 
+## Built with Meta Llama 3
+ 
 ## Model Description
-This model is an OpenVINO IR FP16-INT4-ASYM optimized version of the original [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct), tailored for efficient inference  while maintaining high performance. This version is optimized using [OpenVINO](https://github.com/openvinotoolkit/openvino), enhancing deployment on Intel hardware architectures. The model was optimized by following guidlines from [OpenVINO Notebooks](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks)
-
+ 
+This is a version of the original [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) model converted to [OpenVINO™](https://github.com/openvinotoolkit/openvino) IR (Intermediate Representation) format for optimized inference on Intel® hardware. The model is created using the examples shown in [OpenVINO™ Notebooks](https://github.com/openvinotoolkit/openvino_notebooks/tree/latest/notebooks) repository.
+ 
 ## Intended Use
 This model is designed for advanced natural language understanding and generation tasks, ideal for academic researchers and developers in commercial settings looking to integrate efficient AI capabilities into their applications. It is not to be used for creating or promoting harmful or illegal content as per the guidelines outlined in the [Meta Llama 3 Acceptable Use Policy](https://llama.meta.com/llama3/use-policy/).
-
-## Model Architecture
-Like the original model, 
-Llama-3-8B-Instruct-ov-fp16-int4-sym is based on an auto-regressive transformer architecture, fine-tuned with a focus on instruction-based tasks. The int8 quantization ensures it runs efficiently on compatible hardware without significant loss in performance.
-
-## Carbon Footprint and Sustainability
-Our model training processes are committed to sustainability. The original training utilized Meta’s Research SuperCluster, significantly offsetting carbon emissions to ensure environmentally responsible AI development.
-
+ 
 ## Licensing and Redistribution
-This model is released under the Meta Llama 3 Community License. Redistribution requires inclusion of this license and a citation to the original model. Modifications and derivative works must prominently display "Built with Meta Llama 3" and adhere to the redistribution policies detailed in the original [license terms](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct).
-
-## How to Use
-This model is optimized for inference using Intel's Optimum library with OpenVINO 2024.1, which enables enhanced performance on Intel hardware. Below are the steps to set up and run the model using Optimum and OpenVINO 2024.1 :
-
+This model is released under the Meta Llama 3 Community License. Redistribution requires inclusion of this license and a citation to the original model. Modifications and derivative works must prominently display "Built with Meta Llama 3" and adhere to the redistribution policies detailed in the original model [license terms](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct/blob/main/LICENSE).
+ 
+## Weight Compression Parameters
+For more information on the parameters, refer to the [OpenVINO™ 2024.1.0 documentation](https://docs.openvino.ai/2024/openvino-workflow/model-optimization-guide/weight-compression.html)
+ 
+* mode: **INT4_SYM**
+* group_size: **128**
+* ratio: **0.8**
+ 
+## Running Model Inference
+ 
+Install packages required for using [Optimum Intel](https://huggingface.co/docs/optimum/intel/index) integration with the OpenVINO™ backend:
+ 
+```sh
+pip install --upgrade --upgrade-strategy eager "optimum[openvino]"
+```
+ 
+Run model inference:
 ```python
 from optimum.intel.openvino import OVModelForCausalLM
 from transformers import AutoTokenizer
-
-
+ 
 model_id = "nsbendre25/Llama-3-8B-Instruct-ov_fp16-int4_sym"
-
+ 
 # Initialize the tokenizer and model
 tokenizer = AutoTokenizer.from_pretrained(model_id)
 model = OVModelForCausalLM.from_pretrained(model_id)
-
+ 
 pipeline = transformers.pipeline("text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto")
 pipeline("Hey how are you doing today?")
 ```