The code

The python code used that builds, trains, evaluates the transformer model is shown below.