1. Universität
  2. Fakultäten
  3. Fakultät für Angewandte Informatik
  4. Institut für Informatik
  5. Lehrstühle und Professuren der Informatik
  6. Maschinelles Lernen und Maschinelles Sehen
  7. Forschung
  8. Forschungsprojekte
  9. Audio Brush: What You See is What You Hear

Audio Brush: What You See is What You Hear

Hearing, analyzing and evaluating sounds is possible for everyone. The reference-sensor for audio, the human ear, is of amazing capabilities and high quality. In contrast editing and synthesizing audio is an indirect and non-intuitive task needing great expertise.

 

To overcome these limitations we are creating Audio Brush, a smart visual audio editing tool. Audio Brush allows to edit the spectrogram of a sound in the visual domain similar to editing bitmaps. At the core is a very flexible audio spectrogram based on the Gabor analysis and synthesis. It gives maximum accuracy of the representation, is fully invertible, and enables manipulating the signal at any chosen time-frequency resolution.

 

Simple audio objects are localized in time and frequency in the spectrogram. They can easily be identified visually and selected by simple geometric masks. For many audio objects, however the structures in the spectrogram are rather complex. In order to assist the user in this process, we introduced a new paradigm: Editing through the use of audio objects. Sounds recorded beforehand under controlled conditions are taken as audio objects and stored in a database. Based on audio fingerprinting and visual pattern matching algorithms they are interactively selected and used as visual and sophisticated masks.

 

With these new and adaptive audio analysis and editing algorithms audio can be edited in a “what you see is what you hear” style.

 

For more information on Audio Brush please contact  Gregor van den Boogaart.

 

Screenshot of the Audio Brush editing tool.

 

References:

  • C. Gregor van den Boogaart, Rainer Lienhart. Audio Brush: A Tool for Computer-Assisted Smart Audio Editing. 1st ACM Audio and Music Computing Multimedia Workshop (AMCMM2006), Santa Barabara, pp. 115-124, October 2006. [ PDF ]
  • C. Gregor van den Boogaart, Rainer Lienhart. Audio Brush: Smart Audio Editing in the Spectrogram. Technical Report 2006-12, Institute of Computer Science, University of Augsburg, April 2006. [ PDF ]
  • C. Gregor van den Boogaart, Rainer Lienhart. Visual Audio: An Interactive Tool for Analyzing and Editing of Audio in the Spectrogram. Interactive Video: Algorithms and Technologies, pp. 107-130, Springer Verlag, June 2006.
    Also: Technical Report 2005-22, University of Augsburg, December 2005. [ PDF ]
  • C. Gregor van den Boogaart, Rainer Lienhart. Fast Gabor Transformation for processing high quality audio. ICASSP 2006 IEEE International Conference on Acoustics, Speech and Signal Processing,, Vol. 3, pp. 161-164, 14-19 May 2006, Toulouse, France, 2006.
    Also: Technical Report 2005-21, University of Augsburg, Oktober 2005. [ PDF ]

Suche