In simple words: Transformers use attention; BERT focuses on input understanding and GPT on generation.
The architecture
A Transformer uses self-attention and other layers to process sequences. An encoder builds representations of input; a decoder generates output. Some systems use mainly an encoder, mainly a decoder, or both.
Two familiar names
Bidirectional Encoder Representations from Transformers (BERT) is an encoder-style model useful for understanding text. Generative Pre-trained Transformer (GPT) is a decoder-style family that generates text one token at a time. Both can make mistakes and need checking in important uses.
See the flow
For “The capital of France is”, a GPT-style model predicts a likely next token using the preceding context. This is text generation, not a guarantee of truth.
Remember: Transformers use attention; BERT focuses on input understanding and GPT on generation.