[vc_row][vc_column][vc_headings linewidth=”0″ borderwidth=”1″ borderclr=”#000000″ title=”Brain2Music” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” titlesize=”60″ titleclr=”#000000″]Reconstructing Music from Human Brain Activity[/vc_headings][vc_single_image image=”4126″ img_size=”full” alignment=”center”][vc_column_text]Brain2Music is a method that aims to recreate music from brain activity. It uses fMRI scans to measure how the brain responds to different musical stimuli, such as genre, instruments, and mood. Then, it either retrieves existing music or uses MusicLM, a music generation model, to create new music based on brain activity patterns. The generated music matches the semantic features of the original stimuli. Brain2Music also explores how different parts of the brain and different aspects of MusicLM contribute to the music reconstruction process. It reveals which brain regions encode information from textual descriptions of music stimuli.[/vc_column_text][/vc_column][/vc_row][vc_row][vc_column][vc_headings style=”theme4″ borderclr=”#000000″ style2=”image” title=”How Brain2Music Method Works?” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” lineheight=”3″ titlesize=”40″ titleclr=”#000000″ image_id=”2854″][/vc_headings][vc_column_text]The Brain2Music method consists of two main steps: First, It uses MuLan, a neural network that maps fMRI responses to a semantic music embedding space of 128 dimensions. Second, Google Research Team uses MusicLM, a transformer-based language model that generates music conditioned on semantic embeddings. Alternatively, they can also retrieve music from a large database that matches the semantic embeddings, instead of generating it from scratch.[/vc_column_text][vc_single_image image=”4756″ img_size=”full” alignment=”center”][vc_column_text]
Encoding: Whole-brain Voxel-wise Modeling
A brain encoding model is used to examine how two components of MusicLM (MuLan and w2v-BERT) relate to human brain activity in the auditory cortex.[/vc_column_text][vc_single_image image=”4767″ img_size=”full” alignment=”center”][vc_column_text]The study shows that text and music information share some brain regions.[/vc_column_text][vc_single_image image=”4766″ img_size=”full” alignment=”center”][vc_headings style=”theme4″ borderclr=”#000000″ title=”External Links” google_fonts=”font_family:Comfortaa%3A300%2Cregular%2C700|font_style:700%20bold%20regular%3A700%3Anormal” lineheight=”3″ titlesize=”40″ titleclr=”#000000″ icon=”fas fa-external-link-square-alt”][/vc_headings][/vc_column][/vc_row][vc_row][vc_column width=”1/3″][mvc_advanced_button align=”center” btn_text=”Project” icon_size=”25″ use_theme_fonts=”yes” btn_icon=”fab fa-github” btn_url=”url:https%3A%2F%2Fgoogle-research.github.io%2Fseanet%2Fbrain2music%2F|target:_blank” btn_clr=”#ffffff” btn_bg=”#0a0a0a” btn_radius=”50″][/vc_column][vc_column width=”1/3″][mvc_advanced_button align=”center” btn_text=”Paper” icon_size=”25″ use_theme_fonts=”yes” btn_url=”url:https%3A%2F%2Farxiv.org%2Fpdf%2F2307.11078|target:_blank” btn_clr=”#ffffff” btn_bg=”#ff4300″ btn_radius=”50″ btn_icon=”fas fa-file-pdf”][vc_separator][/vc_column][vc_column width=”1/3″][mvc_advanced_button align=”center” btn_text=”Arxiv” icon_size=”25″ use_theme_fonts=”yes” btn_icon=”fas fa-graduation-cap” btn_url=”url:https%3A%2F%2Farxiv.org%2Fabs%2F2307.11078|target:_blank” btn_clr=”#ffffff” btn_bg=”#0e76a8″ btn_radius=”50″][/vc_column][/vc_row][vc_row][vc_column][easy-subscribe][/vc_column][/vc_row]
0 Comments