A research team led by Professor An Seong-hoon at Seoul National University's Department of Mechanical Engineering has developed technology that allows for locating a person using a single microphone, enabling interaction between humans and robots through sound even in a noisy factory environment. This technology has great potential for use in disaster rescue sites and smart factories.
The research team announced on the 21st that they have realized the world’s first "three-dimensional auditory sensor" that sees the space through hearing, not sight, using sound source localization technology and sound-based communication technology. The research results were published in the international journal "Robotics and Computer-Integrated Manufacturing."
"Sound" is an important clue in industrial sites or disaster rescue operations. Even in situations where visual sensors or electromagnetic-based communication are completely compromised due to high temperatures, dust, smoke, darkness, and obstacles, information can be transmitted using sound waves. However, currently available sound sensing technologies have low accuracy and require complex equipment configurations, making them less applicable in real industrial settings.
In high-noise environments like factories, more advanced sound sensing technology is required because it is very difficult to accurately locate individuals or for robots to recognize vocal commands from workers. Existing communication methods have limitations in that it is challenging to achieve smooth collaboration between robots in environments without networks.
The research team has developed the world’s first meta-structure-based three-dimensional auditory sensor capable of positional recognition using a single sensor. This sensor is based on two key technologies: "three-dimensional sound perceptual technology," which can estimate the three-dimensional position of people or objects even in noisy environments, and "sound wave-based dual communication technology," which implements new interaction methods between humans and robots and among robots.
The research team focused on the biological mechanisms by which bats and dolphins perceive their environment and communicate solely through sound. They engineered the auditory ability to "selectively listen to sounds from specific directions" so that desired sounds can be isolated even in complex noise. This is achieved by artificially controlling the phases of sound waves arriving from different paths, amplifying sounds from specific directions while eliminating the rest. The team named this system "3DAR (3D Acoustic Ranging)."
Inspired by the dual-frequency communication principle of dolphins, the team designed a dual acoustic channel that separates audible and inaudible ranges. Humans and robots communicate through audible frequencies (sounds that humans can hear), while robots communicate with each other through inaudible frequencies (sounds that humans cannot hear). The research team explained, "This structure minimizes interference and provides independent communication paths between robots, allowing for more complex collaboration scenarios in industrial environments."
The research team has also completed validation by applying this technology in actual industrial settings. A quadrupedal walking robot equipped with this system successfully interacted with humans through sound and detected the location of gas leaks using sound.
Professor An Seong-hoon, who led the research, said, "Unlike existing communication technologies that rely on electromagnetic waves blocked by walls or obstacles, systems that utilize sounds that can be heard as long as there is a narrow gap will become a new interaction method in the future."
The first author of the paper, doctoral student An Se-min from the Department of Mechanical Engineering, stated, "Previously, locating a sound required multiple sensors or complex calculations," adding that "the development of this three-dimensional sensor capable of accurately determining the location of a sound source using just a rotating single microphone opens a new path for sound-based sensing technology."