Twenty years ago, I arrived at Harvard to work with Les Valiant on the field that would shortly become known as computational learning theory, but at the time consisted exclusively of two algorithmically focused papers by Valiant, and an early draft of the rather mind-bending (to a first-year graduate student, at least) “four Germans” paper on the exotic and powerful Vapnik-Chervonenkis dimension. It was a great time to enter the field, as virtually any reasonable problem or model one might consider was untouched territory.
Now that field is highly developed (with even many unreasonable problems sporting hefty literatures), I think that the greatest sources of innovation within computational learning theory come from the interaction with the experimental machine learning and AI communities. In a 2003 ICML talk, I recalled how my first paper was published in ICML 1987, then an invitation-only workshop. To our amusement, the program committee strongly advised us not to use abstract symbols like x1 for feature names, but warmer and fuzzier terminology like can_fly and has_wings.
Perhaps we smirked a bit, but we understood the sentiment and complied. Both sides have come a long way since then, to their mutual benefit. The richness of the theory that has been either directly or indirectly driven by the concerns and findings of empirical machine learning and AI work is staggering to me, and it has been a great pleasure to be a theoretician working in a field in such a close dialogue with practitioners. I am hard-pressed to think of other branches of computer science that enjoy comparable marriages. May the next twenty years bring even more of the same; I cannot predict the results but know they will be interesting.