IMAGINARY MUSIC
2022
Roman Klaudo
The "Imaginary Music" project is presented at the exhibition by a video documentation of the process of working on a neurointerface capable of transforming electrical signals of the brain into sound sequences imagined by the author.
Most of the scenes show the conducting of an experiment using audio stimuli. The artist and a group of scientists from the I.P. Pavlov Research Institute recorded the brain's response to sound stimuli received by the subject in the form of rhythmic patterns. The data from the electroencephalograph were then transmitted to a programmer, who constructed a dataset and loaded it into a neural network.
"Imaginary Music" is a multi-phase project that should result in a musical instrument capable of transforming real-time brain activity data into a melody the person is thinking about. As of today, the project is at the stage of testing a hypothesis about the possibility of creating a sufficiently advanced neural network, the accuracy of which today already tends to 70 percent. The next stage will be training the neural network on a large number of datasets collected from different test subjects, which will allow to train the neural network to transform into sound electrical waves of brain activity of a wider range of participants. This will bring the authors closer to the creation of a universal musical instrument, which will work autonomously outside the walls of the laboratory.
Acknowledgement to the research supervisor, all project participants and consultants, the team of the Laboratory of Hearing Physiology, the staff of the Art&Science Center of ITMO University and the AiR Gallery, the Laboratory of Biomechatronics and Energy Efficient Robotics and personally to S.A. Kolubin, A.S. Kolganov from the New Physical Institute of ITMO, announcer Denis Demidov, videographer Daniil Kotelnikov for making this project possible.
The author would like to express special thanks to Varvara Semenova for her invaluable research assistance, organizational skills, and creativity.
The first stage of the project is presented as a video documentation for STAGE-2022 exhibition at ITMO University.
Supervisor: Elena Nikonole
Scientific advisors - Laboratory of Hearing Physiology at the I.P. Pavlov Institute of Physiology, Russian Academy of Sciences:
Shestopalova Lidia Borisovna - Ph.D., Head of the Laboratory
Petropavlovskaya Ekaterina Alekseevna - Candidate of Biological Sciences, Lead Researcher
Semenova Varvara Viktorovna - neurophysiologist, designer, researcher
Other:
Roman Netrogolov - programming
Daria Fomicheva - neurophysiologist
Special Thanks:
Yuri Didevich - artist, musician, researcher, director
Nikita Prudnikov - technologist, musican, IT
Video documentation of the experiment.
