How might self-driving cars change urban mobility? Leveraging deep reinforcement learning and control theory, the talk begins to address this complex societal question by studying emergent behaviors in transportation dynamical systems, consisting of mixed automated and non-automated vehicles. By studying a representative set of traffic phenomena, the work demonstrates that automatically learned control laws are able to coordinate a small fraction of automated vehicles (5-10%) to eliminate traffic congestion and improve overall speeds or throughputs by 30-140%, and even achieve near-optimal performance in certain settings. These results have implications for public health, public safety, urban sustainability, and the near-term impact of self-driving cars. Finally, the talk briefly discusses advances in policy gradient methods and transfer learning motivated by mixed autonomy traffic.