Turing

Turing is a character-level AI language model based on the GCLM (Global Convolutional Language Model) architecture. It is designed to learn from text using a hybrid approach consisting of local 1-dimensional convolutions for short-range dependencies and FFT-based global 1D convolutions for long-range context.

Architecture

The model (GCLM) processes sequences using a stack of blocks that alternate between:

• LocalConv1D: Captures local context (small chunks of n tokens)
• GlobalConv1D: Uses the FFT (Fast Fourier Transform) to capture global context across the entire sequence length.

Usage

Training

To train the model on your own text data:

1. Place .txt files in the data/ directory.
2. Run the training script:

   python train.py
   

   This will automatically detect available hardware (CUDA, MPS, or CPU) and start training, saving checkpoints to Turing_<params>.pt.

Inference

To generate text, run:

python sample.py

Requirements

• Python 3 (install at https://python.org)
• PyTorch (run pip install torch)
• tqdm (pip install tqdm)