A MULTIMODAL APPROACH TO MAPPING SOUNDSCAPES



Abstract:

We explore the problem of mapping soundscapes, that is, predicting the types of sounds that are likely to be heard at a given geographic location. Using a novel dataset, which includes geo-tagged audio and overhead imagery, we develop an approach for constructing an aural atlases, which capture the geospatial distribution of soundscapes. We build on previous work relating sound to ground-level imagery but incorporate overhead imagery to overcome the limitations of sparsely distributed geo-tagged audio. In the end, all that we require to construct an aural atlas is overhead imagery of the region of interest. We show examples of aural atlases at multiple spatial scales, from block-level to country.

Video: Cross-modal retrieval demo.


People:

Highlights:

Related Papers

Dataset:

In our dataset, we have in total 15,773 geo-tagged audio files from FreeSound and their corresponding overhead images.Please contact us by email to receive access to the database.