ash001/llm-foundry-fsdp-opt-1.3B

OPT-1.3B Fine-tuned with LLM Foundry FSDP

This model is a fine-tuned version of facebook/opt-1.3b, fine-tuned using MosaicML's LLM Foundry framework with PyTorch FSDP for production-ready distributed training with config-based setup.

This model was fine-tuned using the Dolly-15K instruction dataset on 2 × T4 16GB GPUs with built-in evaluation and experiment tracking.

For detailed implementation, configuration files, and training procedures, please check out the project repository.