Creative Machine Learning

My research focuses on creative artificial intelligence and machine learning. By creative I refer to the set of techniques, problem domains, and sources of knowledge that inspire my research. Modern AI and ML systems have shown success at imitation of existing creative artifacts - art, music, etc. - but struggle to generate classes of things that haven''t been seen before. How do we build AI and ML systems capable of finding novel, valuable, and surprising solutions to problems across domains?

Thus far I have focused on the domain of computer games. Unlike other creative domains, computer games are dynamic systems that a user interacts with, requiring a system to account for the player's actions.

The first problem I set out to address was how to create an AI that learns to generate game levels. For this work I made use of gameplay video as it contains player reaction to gameplay elements and allows a system to represent multiple games in the same (video) format. This work could generate high-quality game levels, but one might argue that the levels were not novel as they replicated patterns from the original game. What would allow a creative AI to generate a level of a class that it has never seen before, for example an "underwater castle" in Super Mario Bros.? To answer this I made use of conceptual blending to creatively recombine machine learned models to generate novel levels.

The second problem I set out to address was learning game mechanics, the rules that run a game. Once again drawing from gameplay video I made use of a search-based process to iteratively build up a set of rules to explain the video events. These learned rules proved to replicate the true game engine closely. Current work looks at the implications when the system has multiple learned game engines defining a space of possible games. This has implications for procedural content generation and automated game design.


  • Automated Game Creation from Gameplay Videos

    Automated Game Creation from Gameplay Videos supports novice and expert developers creating games for education, entertainment, and other purposes. It draws on probabilistic modeling, rule learning, and machine vision to extract design knowledge from videos of humans playing games and reinterpret this knowledge to a desired effect. In the above video see an AI agent trained on gameplay videos of Super Mario Bros. make design suggestions to a novice designer. For further information see this Paper or this WIRED article for a good summary.

  • Scheherazade-IF

    Scheherazade-IF is an "open interactive narrative generator", capable of creating an interactive narrative game of near-human quality on any subject. It learns this model by reading stories on the desired subject. The system relies on crowdsourcing from everyday individuals to supply these stories, addressing the time and experience requirements of game development.

    In the example above (right) see an in progress interactie narrative created by Scheherazade-IF. Scheherazade-IF constructed this interactive narrative based on crowdsourced stories concerning bank robberies. For further information see this Paper or this Popular Science article for a good summary.