Rename README.md to app.py

README.md

@@ -1,143 +0,0 @@
----
-license: apache-2.0
-tags:
-- finetuned
-base_model: mistralai/Mistral-7B-v0.1
-pipeline_tag: text-generation
-inference: true
-widget:
-- messages:
-  - role: user
-    content: What is your favorite condiment?
-    
-extra_gated_description: If you want to learn more about how we process your personal data, please read our <a href="https://mistral.ai/terms/">Privacy Policy</a>.
----
-
-# Model Card for Mistral-7B-Instruct-v0.1
-
-
-## Encode and Decode with `mistral_common`
-            
-```py
-from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
-from mistral_common.protocol.instruct.messages import UserMessage
-from mistral_common.protocol.instruct.request import ChatCompletionRequest
- 
-mistral_models_path = "MISTRAL_MODELS_PATH"
- 
-tokenizer = MistralTokenizer.v1()
- 
-completion_request = ChatCompletionRequest(messages=[UserMessage(content="Explain Machine Learning to me in a nutshell.")])
- 
-tokens = tokenizer.encode_chat_completion(completion_request).tokens
-```
- 
-## Inference with `mistral_inference`
- 
- ```py
-from mistral_inference.transformer import Transformer
-from mistral_inference.generate import generate
- 
-model = Transformer.from_folder(mistral_models_path)
-out_tokens, _ = generate([tokens], model, max_tokens=64, temperature=0.0, eos_id=tokenizer.instruct_tokenizer.tokenizer.eos_id)
-
-result = tokenizer.decode(out_tokens[0])
-
-print(result)
-```
-
-## Inference with hugging face `transformers`
- 
-```py
-from transformers import AutoModelForCausalLM
- 
-model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
-model.to("cuda")
- 
-generated_ids = model.generate(tokens, max_new_tokens=1000, do_sample=True)
-
-# decode with mistral tokenizer
-result = tokenizer.decode(generated_ids[0].tolist())
-print(result)
-```
-
-> [!TIP]
-> PRs to correct the `transformers` tokenizer so that it gives 1-to-1 the same results as the `mistral_common` reference implementation are very welcome!
-            
----
-
-The Mistral-7B-Instruct-v0.1 Large Language Model (LLM) is a instruct fine-tuned version of the [Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) generative text model using a variety of publicly available conversation datasets.
-
-For full details of this model please read our [paper](https://arxiv.org/abs/2310.06825) and [release blog post](https://mistral.ai/news/announcing-mistral-7b/).
-
-## Instruction format
-
-In order to leverage instruction fine-tuning, your prompt should be surrounded by `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin of sentence id. The next instructions should not. The assistant generation will be ended by the end-of-sentence token id.
-
-E.g.
-```
-text = "<s>[INST] What is your favourite condiment? [/INST]"
-"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s> "
-"[INST] Do you have mayonnaise recipes? [/INST]"
-```
-
-This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method:
-
-```python
-from transformers import AutoModelForCausalLM, AutoTokenizer
-
-device = "cuda" # the device to load the model onto
-
-model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
-tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1")
-
-messages = [
-    {"role": "user", "content": "What is your favourite condiment?"},
-    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
-    {"role": "user", "content": "Do you have mayonnaise recipes?"}
-]
-
-encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
-
-model_inputs = encodeds.to(device)
-model.to(device)
-
-generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
-decoded = tokenizer.batch_decode(generated_ids)
-print(decoded[0])
-```
-
-## Model Architecture
-This instruction model is based on Mistral-7B-v0.1, a transformer model with the following architecture choices:
-- Grouped-Query Attention
-- Sliding-Window Attention
-- Byte-fallback BPE tokenizer
-
-## Troubleshooting
-- If you see the following error:
-```
-Traceback (most recent call last):
-File "", line 1, in
-File "/transformers/models/auto/auto_factory.py", line 482, in from_pretrained
-config, kwargs = AutoConfig.from_pretrained(
-File "/transformers/models/auto/configuration_auto.py", line 1022, in from_pretrained
-config_class = CONFIG_MAPPING[config_dict["model_type"]]
-File "/transformers/models/auto/configuration_auto.py", line 723, in getitem
-raise KeyError(key)
-KeyError: 'mistral'
-```
-
-Installing transformers from source should solve the issue
-pip install git+https://github.com/huggingface/transformers
-
-This should not be required after transformers-v4.33.4.
-
-## Limitations
-
-The Mistral 7B Instruct model is a quick demonstration that the base model can be easily fine-tuned to achieve compelling performance. 
-It does not have any moderation mechanisms. We're looking forward to engaging with the community on ways to
-make the model finely respect guardrails, allowing for deployment in environments requiring moderated outputs.
-
-## The Mistral AI Team
-
-Albert Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, William El Sayed.

app.py

@@ -0,0 +1,197 @@
+# app.py
+import gradio as gr
+from huggingface_hub import InferenceClient
+import os
+
+DEFAULT_MODEL_NAME = "mistralai/Mistral-7B-Instruct-v0.1"
+
+HF_TOKEN = os.getenv("HF_API_TOKEN") # Loads token if set as a secret
+
+# --- Initialize Inference Client ---
+client = None
+
+def get_inference_client(model_name):
+    global client
+    # Initialize client if it hasn't been, or if model name changes
+    if client is None or client.model != model_name:
+        try:
+            # InferenceClient will use HF_TOKEN if it's not None,
+            # or try to infer token otherwise (e.g. from CLI login if running locally).
+            # If no token is found and the model requires one, the API call will fail.
+            client = InferenceClient(model=model_name, token=HF_TOKEN if HF_TOKEN else None)
+            print(f"InferenceClient initialized for {model_name}. Token {'provided' if HF_TOKEN else 'not explicitly provided'}.")
+        except Exception as e:
+            print(f"Failed to initialize InferenceClient for {model_name}: {e}")
+            return None
+    return client
+
+# --- Evaluation Logic ---
+def evaluate_understanding(prompt, response):
+    """
+    Analyzes the model's response to give a basic evaluation of understanding.
+    This is a simple heuristic and not a comprehensive NLU assessment.
+    """
+    if not response or response.strip() == "":
+        return "❌ Not Understood (Empty or whitespace response)"
+
+    response_lower = response.lower() # For case-insensitive checks
+
+    misunderstanding_keywords = [
+        "i'm sorry", "i apologize", "i cannot", "i am unable", "unable to",
+        "i don't understand", "could you please rephrase", "i'm not sure i follow",
+        "that's not clear", "i do not have enough information", "as an ai language model, i don't",
+        "i'm not programmed to", "i lack the ability to"
+    ]
+
+    for keyword in misunderstanding_keywords:
+        if keyword in response_lower:
+            return f"⚠️ Potentially Not Understood (Contains: '{keyword}')"
+
+    if len(prompt.split()) > 7 and len(response.split()) < 10:
+        return "⚠️ Potentially Not Understood (Response seems too short for the prompt)"
+    
+    if prompt.lower() in response_lower and len(response_lower) < len(prompt.lower()) * 1.5 :
+        if len(prompt.split()) > 5 :
+            return "⚠️ Potentially Not Understood (Response might be echoing the prompt)"
+
+    return "✔️ Likely Understood"
+
+# --- Core Logic: Query Model and Evaluate ---
+def query_model_and_evaluate(user_prompt, model_name_to_use):
+    """
+    Sends the prompt to the model, gets the response, and evaluates it.
+    """
+    if not user_prompt or user_prompt.strip() == "":
+        return "Please enter a prompt.", "Evaluation N/A", model_name_to_use
+    
+    # Note: The explicit block for Llama models without HF_TOKEN has been removed.
+    # The InferenceClient will attempt the call. If the model is gated and requires
+    # a token or terms acceptance, the API call itself will likely fail.
+    print(f"Querying model: {model_name_to_use}. HF_TOKEN {'is set' if HF_TOKEN else 'is NOT set/empty'}.")
+
+    current_client = get_inference_client(model_name_to_use)
+    if current_client is None:
+        error_msg = f"Error: Could not initialize the model API client for {model_name_to_use}. Check logs. This might be due to the model requiring authentication (like a token or accepting terms on Hugging Face) which was not available or successful."
+        return error_msg, "Evaluation N/A", model_name_to_use
+
+    try:
+        if "mistral" in model_name_to_use.lower() and "instruct" in model_name_to_use.lower():
+            formatted_prompt = f"<s>[INST] {user_prompt.strip()} [/INST]"
+        elif "llama-2" in model_name_to_use.lower() and "chat" in model_name_to_use.lower():
+            formatted_prompt = (
+                f"[INST] <<SYS>>\nYou are a helpful assistant. Your goal is to understand the user's prompt and respond accurately and relevantly.\n"
+                f"<</SYS>>\n\n{user_prompt.strip()} [/INST]"
+            )
+        else:
+            formatted_prompt = user_prompt.strip()
+
+        params = {
+            "max_new_tokens": 300,
+            "temperature": 0.6,
+            "top_p": 0.9,
+            "repetition_penalty": 1.1,
+            "do_sample": True,
+            "return_full_text": False
+        }
+        
+        model_response_text = current_client.text_generation(formatted_prompt, **params)
+
+        if not model_response_text:
+            model_response_text = ""
+
+    except Exception as e:
+        error_message = f"Error calling model API for {model_name_to_use}: {str(e)}. This can happen if the model is gated, requires a Hugging Face token, or if you need to accept its terms of use on the Hugging Face website."
+        print(error_message)
+        return error_message, "Evaluation N/A", model_name_to_use
+
+    understanding_evaluation = evaluate_understanding(user_prompt, model_response_text)
+    
+    return model_response_text, understanding_evaluation, model_name_to_use
+
+# --- Gradio Interface Definition ---
+with gr.Blocks(theme=gr.themes.Soft(primary_hue="blue", secondary_hue="orange")) as demo:
+    gr.Markdown(
+        f"""
+        # 🎯 Model Prompt Understanding Test
+        Enter a prompt for the selected language model. The application will send this to the model via Hugging Face's Inference API.
+        The model's response will be analyzed to provide a **basic heuristic assessment** of its understanding.
+        
+        **Selected Model:** <span id='current-model-display'>{DEFAULT_MODEL_NAME}</span>
+        """
+    )
+    
+    current_model_name_state = gr.State(DEFAULT_MODEL_NAME)
+
+    with gr.Row():
+        user_input_prompt = gr.Textbox(
+            label="✏️ Enter your Prompt:",
+            placeholder="e.g., Explain the concept of zero-shot learning in 3 sentences.",
+            lines=4,
+            scale=3
+        )
+    
+    submit_button = gr.Button("🚀 Submit Prompt and Evaluate", variant="primary")
+    
+    gr.Markdown("---")
+    gr.Markdown("### 🤖 Model Response & Evaluation")
+    
+    with gr.Row():
+        with gr.Column(scale=2):
+            model_output_response = gr.Textbox(
+                label="📝 Model's Response:", 
+                lines=10, 
+                interactive=False,
+                show_copy_button=True
+            )
+        with gr.Column(scale=1):
+            evaluation_output = gr.Textbox(
+                label="🧐 Understanding Evaluation:", 
+                lines=2, 
+                interactive=False,
+                show_copy_button=True
+            )
+            displayed_model = gr.Textbox(
+                label="⚙️ Model Used for this Response:",
+                interactive=False,
+                lines=1
+            )
+
+    submit_button.click(
+        fn=query_model_and_evaluate,
+        inputs=[user_input_prompt, current_model_name_state],
+        outputs=[model_output_response, evaluation_output, displayed_model]
+    )
+    
+    gr.Markdown(
+        """
+        ---
+        **Disclaimer:**
+        * The 'Understanding Evaluation' is a very basic automated heuristic.
+        * **Using Models:** This app will attempt to connect to the selected model. Some models (especially gated ones like Llama-2) may require you to have a Hugging Face account, accept their terms of use on the Hugging Face website, and might implicitly require a valid `HF_TOKEN` associated with your account (even if not explicitly set as a secret in this Space). If a model call fails, it could be due to these reasons.
+        * Response quality depends heavily on the chosen model and the clarity of your prompt.
+        """
+    )
+    
+    gr.Examples(
+        examples=[
+            ["Explain the difference between supervised and unsupervised machine learning.", DEFAULT_MODEL_NAME],
+            ["Write a short poem about a curious robot.", DEFAULT_MODEL_NAME],
+            ["What are the main challenges in developing AGI?", DEFAULT_MODEL_NAME],
+            ["Summarize the plot of 'War and Peace' in one paragraph.", DEFAULT_MODEL_NAME],
+            ["asdfjkl; qwerpoiu", DEFAULT_MODEL_NAME]
+        ],
+        inputs=[user_input_prompt, current_model_name_state],
+        outputs=[model_output_response, evaluation_output, displayed_model],
+        fn=query_model_and_evaluate,
+        cache_examples=False,
+        label="💡 Example Prompts (click to try)"
+    )
+
+if __name__ == "__main__":
+    print("Attempting to launch Gradio demo...")
+    print(f"Default model: {DEFAULT_MODEL_NAME}")
+    if HF_TOKEN:
+        print("HF_TOKEN is set.")
+    else:
+        print("HF_TOKEN is NOT set. Some models (especially gated ones like Llama) might require a token or prior agreement to terms on the Hugging Face website to function correctly. The app will attempt to run, but API calls may fail.")
+    demo.launch()