README.md

---
license: gpl-3.0
datasets:
- JoseEliel/lagrangian_generation
pipeline_tag: text2text-generation
tags:
- Physics
- Math
- Lagrangian
library_name: transformers
---
## Model Summary

BART-Lagrangian is a sequence-to-sequence Transformer (BART-based) specifically trained to generate particle physics Lagrangians from textual descriptions of fields, spins, and gauge symmetries. Unlike typical language models, BART-Lagrangian focuses on the symbolic structure of physics, aiming to produce coherent and accurate Lagrangian terms given customized tokens representing field types, spins, helicities, gauge groups (SU(3), SU(2), U(1)), and more.

Key Highlights:

• BART architecture with sequence-to-sequence pretraining  
• Custom tokenization scheme capturing field quantum numbers and contractions  
• Specialized training on a large corpus of symbolic physics data  

BART-Lagrangian is well suited for research and experimentation in symbolic physics or any domain requiring structured symbolic generation.

--------------------------------------------------------------------------------

## Usage

You can use BART-Lagrangian directly with the Hugging Face Transformers library:

1) Install prerequisites (for example with pip):  
   pip install transformers torch

2) Load the model and tokenizer:

```python
from transformers import BartForConditionalGeneration, PreTrainedTokenizerFast

model_name = "JoseEliel/BART-Lagrangian"
model = BartForConditionalGeneration.from_pretrained(model_name)
tokenizer = PreTrainedTokenizerFast.from_pretrained(model_name)
```

3) Prepare your input. Below is a simple example describing two fields, a scalar with charges (1, 2, 1) and a fermion with -1/2 Helicity with charges (3, 2, 1/3) under (SU(3), SU(2), U(1)):

```python
input_text = "[SOS] FIELD SPIN 0 SU2 2 U1 1 FIELD SPIN 1 / 2 SU3 3 SU2 2 U1 1 / 3 HEL - 1 / 2 [EOS]"
```

4) Perform generation:

```python
inputs = tokenizer(input_text, return_tensors="pt")
outputs = model.generate(inputs['input_ids'], max_length=2048)
decoded_outputs = tokenizer.batch_decode(outputs, skip_special_tokens=True)
print("Generated Lagrangian:")
print(decoded_outputs[0])
```

--------------------------------------------------------------------------------

## Evaluation

BART-Lagrangian has been evaluated on both:

• Internal test sets of symbolic Lagrangians to measure consistency and correctness.  
• Human inspection by domain experts to confirm the generated Lagrangian terms align with expected physics rules (e.g., correct gauge symmetries, valid contractions).

For more details on benchmarks and accuracy, refer to our upcoming paper “Generating Particle Physics Lagrangians with Transformers” (arXiv link placeholder).

--------------------------------------------------------------------------------

## Limitations

• Domain Specificity: BART-Lagrangian is specialized for Lagrangian generation; it may not perform well on unrelated language tasks.  
• Input Format Sensitivity: The model relies on a specific tokenized format for fields and symmetries. Incorrect or incomplete tokenization can yield suboptimal or invalid outputs.  
• Potential Redundancy: Some generated Lagrangians can contain redundant terms, as non-redundant operator filtering was beyond the scope of initial training.  
• Context Length Limit: The default generation max_length is 2048 tokens, which may be insufficient for extremely large or highly complex expansions.

--------------------------------------------------------------------------------

## Training

• Architecture: BART, sequence-to-sequence Transformer with approximately 357M parameters.  
• Data: A large corpus of synthetically generated Lagrangians using a custom pipeline (AutoEFT + additional code).  
• Objective: Conditioned generation of invariant terms given field tokens, spins, and gauge group embeddings.  
• Hardware: Trained on an A100 GPU, leveraging standard PyTorch and Transformers libraries.  

For more technical details, see the forthcoming paper cited above.

--------------------------------------------------------------------------------

## License

The model, code, and weights are provided under the AGPL-3.0 license.

--------------------------------------------------------------------------------

## Citation

If you use BART-Lagrangian in your work, please cite it as follows:

```bibtex
@misc{BARTLagrangian,
      title={Generating particle physics Lagrangians with transformers}, 
      author={Yong Sheng Koay and Rikard Enberg and Stefano Moretti and Eliel Camargo-Molina},
      year={2025},
      eprint={2501.09729},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2501.09729}, 
}