SamuelJaja/llama-3.1-8b-instruct-construction-merged

LLAMA-3.1-8B-Instruct-Construction (Merged)

This is a fine-tuned version of Meta's Llama-3.1-8B-Instruct model optimized for construction industry and UK building regulations knowledge. This repository contains the full merged model (base + fine-tuning), making it directly usable with the Hugging Face API.

Model Details

• Base Model: meta-llama/Meta-Llama-3.1-8B-Instruct
• Fine-tuning Method: LoRA (Low-Rank Adaptation), merged with base model
• Training Data: Custom dataset focusing on UK construction industry standards, building regulations, and safety requirements
• Parameters: 8 billion parameters
• Training Hardware: A100 GPU with 40GB VRAM
• Usage: This model is designed to answer questions about building codes, construction best practices, and regulatory compliance with a focus on UK standards

Capabilities

This model can:

• Answer questions about UK building regulations and standards
• Explain technical requirements for construction projects
• Provide insights on fire safety, accessibility, insulation, and sustainable design
• Assist with understanding compliance requirements for construction projects
• Interpret building code requirements for various building types

Example Usage

You can use this model directly with the Hugging Face transformers library:

import torch
import re
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load model and tokenizer
model_id = "SamuelJaja/llama-3.1-8b-instruct-construction-merged"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_id)

# Clean response function
def clean_response(text):
    """Remove any instruction tags from the response"""
    return re.sub(r'\[/?INST\]', '', text).strip()

# Format prompt function
def format_prompt(prompt):
    if not prompt.startswith("[INST]"):
        return f"[INST] {prompt} [/INST]"
    return prompt

# Generate text
prompt = "What are the main requirements for fire safety in commercial buildings?"
formatted_prompt = format_prompt(prompt)
inputs = tokenizer(formatted_prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    input_ids=inputs.input_ids,
    attention_mask=inputs.attention_mask,
    max_new_tokens=256,
    temperature=0.1,
    top_p=0.9,
    do_sample=False
)

# Process the response
full_response = tokenizer.decode(outputs[0], skip_special_tokens=True)

# Extract just the model's response
if formatted_prompt in full_response:
    response = full_response.replace(formatted_prompt, "").strip()
else:
    response = full_response

# Clean any remaining tags
response = clean_response(response)
print(response)