Can we reconstruct audio from silent video by tracking microscopic object vibrations?

Can You Recover Sound From Images? · 11:23 · Video: Veritasium · Watch at the source

In short: This video explores research by Dr. Abe Davis on recovering sound from high-speed video. While it seems intuitive that objects vibrate like tuning forks, the reality is far more complex, requiring sophisticated algorithms to detect movements smaller than a single pixel.

Sound waves cause objects to vibrate by approximately one micrometer. Because this movement is significantly smaller than a pixel, researchers developed an algorithm that analyzes how edges within an image brighten or darken in response to sound. By sampling these correlated changes across the frame, the system can isolate the signal from random noise.

The process relies on the fact that these subtle movements are consistent along object edges. The video also considers the theoretical possibility of capturing sound from a single static image, noting that this would likely require imaging large objects or spaces to account for the long wavelengths of typical sounds, or perhaps using high-frequency audio in a specialized medium.

Opens the Intellect+ feed with this card first. Swipe up for the next one.

Sources

To cite this card, link to https://intellectplus.co.uk/app/science-can-you-recover-sound-from-images.

How we source, check and correct cards: Editorial standards.

General information, not professional advice. Spotted an error? Flag the card or see corrections and complaints.

Intellect Plus is a visual educational discovery platform for exploring fascinating facts and ideas. Brain Food, Not Brain Rot. What is Intellect Plus?

Related

Finding something worth knowing…