Researchers from Gwangju Institute of Science and Technology Develop an Intelligent Observer for Esports | Pages Da

It uses an object recognition algorithm that studies human viewing data to find fascinating viewpoints

GWANGJU, South Korea, November 25, 2022 /PRNewswire/ — Human game spectators are an essential part of the esports industry. They use extensive knowledge in the field to decide what to show viewers. However, they may miss important events, necessitating the need for automated observers. Researchers from South Korea recently proposed a framework using object detection method, Mask R-CNN and human observation data to find the ‘common interest region’ in StarCraft – a real-time strategy game.

Esports, already a multi-billion dollar industry, is growing, in part because of watching human play. They control the movement of the camera and show viewers the most fascinating parts of the game screen. However, these viewers may miss significant events happening simultaneously across multiple screens. It is difficult to afford them even in small tournaments. As a result, the demand for automatic observers has increased. Artificial observation methods can be rule-based or learning-based. Both pre-define events and their importance, which requires extensive knowledge in the field. Moreover, they cannot capture undefined events or discern changes in the meaning of events.

Recently, researchers from South Korealed by dr. Kyung-jung KimAn associate professor at the Gwangju Institute of Science and Technology, proposed an approach to overcome these problems. “We created an automatic observer using an object recognition algorithm, Mask R-CNN, to learn human observer data.” explains Dr. Kim. Their findings were made available online at October 10, 2022 and published in Volume 213, Part II of Expert systems with applications Magazine.

The innovation lies in the definition of the object as the two-dimensional spatial region observed by the viewer. In contrast, conventional object detection treats a single unit, for example, a worker or a building, as the object. In this study, the researchers first collected StarCraft in-game human observation data from 25 participants. Then, the viewports—areas viewed by the viewer—were identified and labeled as “one.” The rest of the screen was full of “zeros”. While the in-game features serve as input data, the human observations constituted the target information.

The researchers then fed the data into a convolutional neural network (CNN), which learned the patterns of the display outputs to find the “region of common interest” (ROCI)—the most exciting area for viewers to view. They then compared the ROCI Mask R-CNN approach with other existing methods quantitatively and qualitatively. The previous evaluation showed that CNN’s predicted viewership was similar to the collected human observation data. In addition, the ROCI-based method outperformed others in the long run during the generalization test, which included various match races, starting positions, and map play. The proposed viewer was able to capture the scenes of interest to humans. However, this could not be done by behavior cloning – an imitation learning technique.

Dr. Kim points to the future applications of their work. “The framework can be applied to other games that represent part of the overall game state, not just StarCraft. As services such as multi-screen streaming continue to grow in esports, the proposed auto viewer will play a role in these products. Actively used in additional content developed in the future.”


Original article title: Learning automatic game viewing for Esports using an object detection mechanism

Journal: Expert Systems with Applications

*Corresponding author email: [email protected]

About Gwangju Institute of Science and Technology (GIST)

Chang Sung Kang
82 62 715 6253
[email protected]



View original content for multimedia download: . html

SOURCE Gwangju Institute of Science and Technology


Leave a Comment

Your email address will not be published. Required fields are marked *