Abstract
I will present several results (published and unpublished) relating transformers to first order logic, linear temporal logic, and counting extensions of both. These results reveal abilities and limitations of transformers as well as implications like the effect of depth on expressivity.