A Real-time Beat Tracking System for Musical Acoustic Signals
This project is proposed and researched by
Masataka Goto.
Japanese version is here.
We have developed a beat tracking system, called BTS,
that processes acoustic signals of music and
recognizes temporal positions of beats in real time.
Beat tracking is an important initial step
in computer emulation of human music understanding,
since beats are fundamental to the perception of music.
In addition, musical beat tracking is useful for various applications
such as video/audio editing, stage lighting control,
and synchronization of real-time computer graphics with music.
BTS deals with popular music, particularly rock and pop music
in which drums maintain the beat.
Not only does BTS predict the temporal position of the next beat
(quarter-note);
it also determines whether the beat is strong or weak,
in other words, BTS can track beats at the half-note level.
Because our system examines multiple hypotheses in parallel,
it can follow beats without losing track of them,
even if some hypotheses become wrong.
BTS has been implemented on a parallel computer, the Fujitsu AP1000,
to handle this computationally intensive task in real time.
In our experiment,
BTS correctly tracked beats
in 43 out of 45
commercially distributed popular songs.
Afterwards, I have developed
a beat tracking system for musical audio signals without drum-sounds.
Our recent results for music without drum-sounds were reported at
The Second International Conference on Multiagent Systems,
IPSJ
Special Interest Group on MUSic and computer (in Japanese),
and
the IJCAI-97
Workshop on Computational Auditory Scene Analysis.
We have also developed a computer-graphics dancer ``Cindy''
who can dance in time to musical beats.
References:
- Masataka Goto:
An Audio-based Real-time Beat Tracking System
for Music With or Without Drum-sounds,
Journal of New Music Research, Vol.30, No.2, pp.159-171, June 2001.
(in a slightly different format)
- Masataka Goto and Yoichi Muraoka:
Real-time Beat Tracking for Drumless Audio Signals:
Chord Change Detection for Musical Decisions,
Speech Communication, Vol.27, Nos.3-4, pp.311-335, April 1999.
- Masataka Goto:
Real-time Beat Tracking for Musical Audio Signals,
Proceedings of the First Meeting of
Special Interest Group on AI Challenges, SIG-Challenge-9801-2,
pp.7-14, November 1998 (in Japanese).
- Masataka Goto:
Real-time Beat Tracking for Musical Audio Signals,
Record of The 1998 Kansai-Section Joint Convention of
Institutes of Electrical Engineering, Japan, S12-2,
p.S65, November 1998 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
An Audio-based Real-time Beat Tracking System
and Its Applications,
Proceedings of the 1998 International Computer Music Conference,
pp.17-20, October 1998.
- Masataka Goto:
Beat Tracking,
separate volume of bit: Computer and Music World,
pp.100-116, Kyoritsu Shuppan, August 1998 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
Music Understanding at the Beat Level:
Real-time Beat Tracking for Audio Signals,
In David F. Rosenthal and Hiroshi G. Okuno, editors,
Computational Auditory Scene Analysis, pp.157-176,
Lawrence Erlbaum Associates, Publishers, 1998.
- Masataka Goto and Yoichi Muraoka:
A Real-time Beat Tracking System for Audio Signals
--- Application to Drumless Music by Detecting Chord Changes ---,
The Transactions of the Institute of Electronics,
Information and Communication Engineers D-II,
Vol.J81-D-II, No.2, pp.227-237,
February 1998 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
Real-time Rhythm Tracking for Drumless Audio Signals
--- Chord Change Detection for Musical Decisions ---,
Working Notes of
the IJCAI-97 Workshop on Computational Auditory Scene Analysis,
pp.135-144, August 1997.
- Masataka Goto and Yoichi Muraoka:
Issues in Evaluating Beat Tracking Systems,
Working Notes of
the IJCAI-97 Workshop on Issues in AI and Music
- Evaluation and Assessment,
pp.9-16, August 1997.
- Masataka Goto and Yoichi Muraoka:
A Beat Tracking System for Musical Audio Signals
--- Bar-line Detection and Musical Knowledge Selection
Based on the Presence of Drum-sounds ---,
Information Processing Society of Japan SIG Notes,
97-MUS-21-8, Vol.97, No.67, July 1997 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
Beat Tracking based on Multiple-agent Architecture
--- A Real-time Beat Tracking System for Audio Signals ---,
Proceedings of
The Second International Conference on Multiagent Systems,
pp.103-110, December 1996.
- Masataka Goto and Yoichi Muraoka:
A Real-time Beat Tracking System for Audio Signals
--- Beat Tracking for Music without Drum-sounds ---,
Information Processing Society of Japan SIG Notes,
96-MUS-16-3, Vol.96, No.75, July 1996 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
Parallel Implementation of a Beat Tracking System
--- Real-time Musical Information Processing on AP1000 ---,
Transactions of Information Processing Society of Japan,
Vol.37, No.7, pp.1460-1468, July 1996 (in Japanese).
- Masataka Goto:
Can Computer Keep Time to Music by Hand-clapping?
--- A Real-time Beat Tracking System ---,
bit, Vol.28, No.3, pp.4-11, Kyoritsu Shuppan, March 1996 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
A Real-time Beat Tracking System for Audio Signals,
Proceedings of the 1995 International Computer Music Conference,
pp.171-174, September 1995.
- Masataka Goto and Yoichi Muraoka:
Music Understanding At The Beat Level
--- Real-time Beat Tracking For Audio Signals ---,
Working Notes of the
IJCAI-95
Workshop on Computational Auditory Scene Analysis,
pp.68-75, August 1995.
- Masataka Goto and Yoichi Muraoka:
Parallel Implementation of A Real-time Beat Tracking System
--- Real-time Musical Information Processing on AP1000 ---,
Joint Symposium on Parallel Processing 1995 (JSPP'95) Proceedings,
pp.353-360, May 1995 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
A Beat Tracking System for Acoustic Signals of Music,
ACM Multimedia 94
Proceedings (Second ACM International Conference on Multimedia),
pp.365-372, October 1994.
- David Rosenthal, Masataka Goto and Yoichi Muraoka:
Rhythm Tracking Using Multiple Hypotheses,
Proceedings of the 1994 International Computer Music Conference,
pp.85-87, September 1994.
- Masataka Goto and Yoichi Muraoka:
A Real-time Beat Tracking System for Musical Acoustic Signals,
Information Processing Society of Japan SIG Notes,
94-MUS-7-8, Vol.94, No.71, August 1994.
(in Japanese)
- Masataka Goto and Yoichi Muraoka:
Real-time Acoustic Information Processing on a Parallel Computer
--- A Beat Tracking System for Musical Acoustic Signals ---,
Proceedings of the 49th Annual Convention IPS Japan,
4P-9, September 1994 (in Japanese).
- Masataka Goto and Yoichi Muraoka:
A Beat Tracking System for Acoustic Signals of Music,
Proceedings of the 48th Annual Convention IPS Japan,
4T-4, March 1994 (in Japanese).
If you have any problem in printing, please let me know.
I will send you copies of my papers.
Masataka Goto's other projects:
Back to:
Please E-mail comments and questions to
Masataka GOTO
<m.goto [at] aist.go.jp>
last update: February 18, 2002
Title:
Parallel Implementation of A Real-time Beat Tracking System
--- Real-time Musical Information Processing on AP1000 ---
Authors:
Masataka Goto and Yoichi Muraoka
Abstract:
This paper presents
parallel implementation of a real-time beat tracking system
that processes musical acoustic signals and
recognizes temporal positions of musical-beats.
Parallel processing is necessary
to perform this computationally-intensive task in real time.
We apply four kinds of parallelizing techniques
to execute heterogeneous processes simultaneously.
The processes are first pipelined, and then
each stage of the pipeline is implemented with
data/control parallel processing, pipeline processing and
distributed cooperative processing.
Our system has been implemented on the Fujitsu AP1000.
In our experiment,
it correctly tracked beats in 27 out of 30 popular songs.
Title:
A Beat Tracking System for Acoustic Signals of Music
Authors:
Masataka Goto and Yoichi Muraoka
Abstract:
This paper presents a beat tracking system
that processes acoustic signals of music and
recognizes temporal positions of beats in real time.
Musical beat tracking is needed by various multimedia applications such as
video editing, audio editing, and stage lighting control.
Previous systems were not able to deal with acoustic signals
that contained sounds of various instruments, especially drums.
They dealt with either MIDI signals or
acoustic signals played on a few instruments,
and in the latter case, did not work in real time.
Our system deals with popular music
in which drums maintain the beat.
Because our system examines multiple hypotheses in parallel,
it can follow beats without losing track of them,
even if some hypotheses become wrong.
Our system has been implemented on a parallel computer, the Fujitsu AP1000.
In our experiment,
the system correctly tracked beats
in 27 out of 30 commercially distributed popular songs.
Title:
Rhythm Tracking Using Multiple Hypotheses
Authors:
David Rosenthal, Masataka Goto and Yoichi Muraoka
Abstract:
We briefly describe two rhythm-tracking systems,
called, respectively, Machine Rhythm and BTS.
Given a MIDI stream as input,
Machine Rhythm produces an interpretation
that is essentially isomorphic to the rhythmic information
represented in normal musical notation.
The output of the program defines the placement of measures
and assigns rhythmic values (half-note beats, quarter-note beats, etc.)
to each note.
Although Machine Rhythm is not a real-time system,
it processes the MIDI information sequentially,
paving the way for possible future real-time implementations.
The program attempts to handle
some of the more sophisticated rhythm-tracking operations
of which humans are capable,
such as changes from duple to triple meter or changes in tempo.
BTS tracks beats using raw audio signals as input --- in general,
a much more difficult problem than tracking it from MIDI data.
BTS accomplishes this task by leveraging
the fact that for a large corpus of music --- rock and pop songs ---
the beat is indicated with some reliability by the bass and snare drums.
BTS's non-reliance on MIDI data
enables it to handle a broad range of multimedia applications
for which MIDI-based beat-tracking programs cannot be used,
and the fact that it works in real time
enables its application in a variety of live performance situations.
Both Machine Rhythm and BTS
use a similar strategy
for managing uncertain or noisy input data --- namely,
the strategy of pursuing multiple hypotheses.
Title:
A Real-time Beat Tracking System for Musical Acoustic Signals
Authors:
Masataka Goto and Yoichi Muraoka
Abstract:
This paper presents a beat tracking system
that processes musical acoustic signals and
recognizes temporal positions of beats in real time.
Previous systems were not able to deal with acoustic signals
that contained sounds of various instruments, especially drums.
They dealt with either MIDI signals or
acoustic signals played on a few instruments.
Our system deals with popular music in which drums maintain the beat.
Because our system examines multiple hypotheses in parallel,
it can follow beats without losing track of them,
even if some hypotheses become wrong.
Our system has been implemented on a parallel computer, the Fujitsu AP1000.
In our experiment,
it correctly tracked beats
in 27 out of 30 commercially distributed popular songs.