The Hong Kong Baptist University (HKBU) took the laboratory to the Hong Kong Cultural Center Concert Hall for its Annual Gala Concert on April 1, which interwove brainwave technology with music in a bid to create an all-sensory experience for its audiences.
The HKBU Symphony Orchestra used portable functional Near-Infrared Spectroscopy (fNIRS) devices to monitor the brain activity of musicians as they performed during the concert, which attracted around 1,200 audience.
The devices transmitted brainwave data via Bluetooth to a computer, where artificial intelligence (AI) transformed the signals into dynamic visual art, bridging neuroscience and creative expression in real-time.
This provided a singular musical experience for the audience by translating the brainwave patterns of the performers into fluid images via AI for presentation.
Professor Johnny Poon, Associate Vice-President and Founding Dean of the School of Creative Arts at HKBU, served as both the conductor and lead researcher for the performance.
Poon revealed that each fNIRS device, valued at over HK$400,000, could accurately monitor brain activity.
Poon explained that his research team selected music as the primary focus of their study as it engages multiple areas of the brain, making it a highly valuable subject for research. The brain stimulations include interpreting sheet music, recognizing pitch, and coordinating physical movements.
He emphasized the broader implications of the research as the data collected during the concert could help develop strategies to enhance future performances. Potential areas of study include examining how different meal times influence performance quality.
In addition, Poon highlighted the increasing integration of “Science, Technology, Engineering, Arts, and Mathematics (STEAM)” education into primary and secondary schools. He noted the growing interest in AI-driven projects, showing limitless potential in combining arts and technology.
