Learning experiment · PyTorch

Transformer learning implementation

A tutorial-based character-level decoder implementation, developed with AI assistance to learn how attention and decoder blocks fit together.

Concept illustration of tokens, attention and decoder layers
Concept illustration. Original project evidence follows below.

What the code covers

The notebooks explore token and positional embeddings, scaled dot-product attention, multiple attention heads, feed-forward blocks, residual connections and layer normalisation.

The implementation also includes data batching, training-loop and sampling code, with an Intel XPU execution path.

Overview of the character-level decoder implementation, including attention and training code
Implementation overview for the learning exercise.

Scope: architecture and code study. No completed training run, loss curve or generated output is verified here.