Emotion, not just action, helps brain define, divide events

In the 2001 film “Tortilla Soup,” an argument between three sisters culminates in one smashing a plate on the kitchen floor.

The abrupt, surprising action is an example of what psychology scholars call an event boundary—a clear transition from one event to another that our minds use to separate continuous experience into meaningful, memorable chunks.

New Cornell research suggests emotion—not just more easily defined changes in action or location—also is important to this segmentation process, playing a larger role than previously understood in structuring attention and memory. In the above scene, the broken plate marks a shift from growing tension to a cathartic reconciliation.

“Emotion does drive how we process critical changes in events,” said Ruiyi Chen, a doctoral student in the field of psychology. “It helps signal moments that we need to pay higher attention to, and that we will rely on to remember these experiences.”

Chen is the co-author with Khena Swallow, associate professor of psychology in the College of Arts and Sciences and director of the Attention, Memory, and Perception Lab, of “The Role of Emotional Content in Segmenting Naturalistic Videos into Events,” published in the Journal of Experimental Psychology: General.

Despite emotion being a central feature of daily life, influencing what people pay attention to and remember, research has largely overlooked its role in how the brain divides ongoing experience, the authors said. Theories of event segmentation have focused on brief and discrete changes in action, location or goals—someone walking into a room, say, or picking up an object—without accounting for emotional changes that may develop more subtly and gradually.

Chen and Swallow investigated that phenomenon by analyzing perceptions of commercial movies, which present complex, life-like scenes involving plot and narrative that can be repeated for large numbers of observers.

In a first experiment, nearly 50 viewers watched three- to five-minute clips from nine popular movies, including “Anger Management,” “Erin Brockovich” and “Forrest Gump.” Half of the study participants were asked to track changes in emotion, while the other half noted event-based boundaries. A separate group of 10 independently measured changes in emotional intensity (arousal) and pleasant or unpleasant feelings (valence).

The results showed that viewers agreed on when both emotional shifts and event boundaries occurred, and the two categories often overlapped. Stronger changes in emotions—particularly those involving positive or negative feelings—increased the probability that the participants would classify a scene as an event boundary, regardless of any associated action.

In “Tortilla Soup,” which was used to train study participants, the shattered plate (the first of several) is highly likely to be classified as an event boundary, the researchers said, because it combines attention-getting action with a significant emotional transition.

“We can’t say which is more important, changes in action versus emotion,” Swallow said. “But we can say that emotion contributes to this segmentation above and beyond anything contributed by other factors.”

A second experiment replicated the first’s findings using a longer video (an episode of the BBC series “Sherlock”) that had been used in prior studies of naturalistic perception.

The findings suggest emotions are not merely color that helps describe daily events, the researchers said, but integral to how we define and sort them—a fundamental aspect of cognition that might impact attention, episodic memory, communication and decision-making.

“Emotion is probably much more central to defining an event than psychologists have appreciated in the past,” Swallow said. “But emotion can change slowly, and it’s not entirely clear how to incorporate that aspect of experience into our theories about how event segmentation works. These findings are a call to grapple with that issue more directly.”