Have a look at the latest tutorial by Andreas Troxler and Jürg Schelldorfer! It discusses actuarial applications of natural language processing (NLP) using transformers.
The tutorial is shipped together with notebooks which lead through the Python implementation. All files, including the datasets and notebooks, are available on github.