Microsoft Patents System That Reads Your Face and Feeds It to AI to Redesign Games

Microsoft has filed a patent describing an ambitious system that would record players’ facial expressions, vocalisations, and in-game behaviour, feeding that data into a generative artificial intelligence capable of redesigning video game content on the fly. The filing, published as US patent 20260183672A1 under the title Generative Narrative Game Experience With Player Feedback, outlines how studios could use real-time player reactions to generate new quests, dialogue, non-playable characters, and even entire narrative branches. As with virtually every Microsoft patent, however, it describes a technology the company has merely thought about rather than one it has actually built.

The broader process described in the document targets game designers first, envisioning an AI that can build out the narrative scaffolding of a title from a simple prompt before refining itself based on how actual players engage with the finished product. Microsoft’s engineers frame this as a way to make games more responsive to player preference, though critics have questioned whether generative systems could ever replicate the deliberate craft of human storytelling.

From Prompt to Playable World

In its most straightforward form, the patent describes designers feeding an AI written input, images, and audio files in order to generate characters, narratives, and worlds “at a high level.” The model would then produce gameplay content including NPCs, items, dialogue, lore descriptions, objectives, and sidequests, returning them to the designer as “a visual representation of the game […] in the form of text, images, and/or graphs.”

The prompt could be merely a general theme, idea, or goal for the game or characterize the general nature of the game, or it may include some narrative details, such as main characters, a basic storyline, etc.

Microsoft patent filing

The document notes that alongside these visual representations, the model might also supply code suitable for insertion into a game engine, or update an existing title through an API. Microsoft’s engineers describe this pipeline as straightforward enough that even a vague creative brief could yield a functional starting point.

Players as Co-Creators

The patent then expands the designer role to include players themselves, albeit not through direct prompting but by having an AI adapt content based on player actions and “engagement metrics.” In this scenario, QA testers, beta testers, or early access participants would supply data that feeds back into the model during development or between release builds, producing material for designers to incorporate before official launch.

Beyond that staged approach, Microsoft also imagines fully real-time interactions with generative AI. Similar to conversations players have had with NPCs in games like Where Winds Meet, these systems would consume a player’s spoken request alongside contextual data such as their location within the world, prior dialogue, and story progress before generating a response. The patent authors write enthusiastically that “the possible natural-language output of the conversational agent are virtually limitless.”

Beyond Dialogue

Microsoft’s engineers do not stop at freshly generated conversation. They propose agents capable of producing programmatic output as well, writing that an NPC-associated agent “may create not only conversational output (e.g., suggesting a side quest for the player to pursue), but also programmatic output that creates new game assets, rules, or alterations of the game world associated with the conversational output.” Generative agents could likewise govern ordinary objects in the world rather than merely dispensing lines.

In some instances, the new content can be so involved as to affect even the game narrative, creating new narrative elements, paths through the game, and modes of gameplay. In multi-player games, in particular, the collective interactions of players with each other and with the game can result in complex emergent new game behaviors.

Microsoft patent filing

The document acknowledges that this approach risks diluting authored storytelling. Narrative directors have previously expressed skepticism about generative dialogue, noting that unbounded AI output may struggle to match the emotional precision of carefully written scenes.

Looking Ahead

Microsoft’s engineers argue that the system would ultimately make games more engaging by amplifying what players enjoy and pruning content that falls flat. “If a particular part of the game tends to attract players, the model may decide to elaborate on and extend that part of the game,” they write, adding that less engaging material “may be pruned or assigned lower weight.” The patent even speculates about AI reacting to visible player emotion during pivotal story moments.

Notably, however, the filing offers little evidence of creative ambition. One example quest it proposes for a Wizard of Oz title involves simply collecting items around the land of Oz before an audience with the Wizard. With such modest suggestions, Lies of P developer Neowiz, currently developing the Oz-set Wonders of O, appears unlikely to lose sleep.

Until Microsoft actually builds anything from this patent, players need not worry about their expressions shaping their games. The company has documented a possibility rather than delivered a product, and anyone concerned about an AI-generated future may simply want to keep playing as they always have.