Oh Lord, Please Help The Choir Sing On-Key

Potential solutions to every music director's nightmare, and a peek into the future of music performance

From My Future Tech Notebook — a collection of ideas that might or might not see the light of day.

Performing a cappella music with a small to medium-size group of volunteers is difficult. Unless each section has a competent and confident voice, epic failures can happen. Even when all group members are in tune, there is something called drift. This means that the singers unintentionally end the song in a different key than they started in.

Drift in a cappella music is a natural occurrence, but many music directors spend a lot of time trying to eliminate or minimize it.

The phantom (not necessarily of the opera)

A straightforward solution is to have a “silent” voice heard by the singers but not the audience. Each singer would wear a wireless earpiece, listening to a "reference" voice singing their part. The reference voice could be streamed from a multi-track recording. Singers would only need to sing along with the phantom voice and keep in tune.

One problem with following a recording is that it is not very flexible… the recording would dictate things like tempo and dynamics, which the conductor or director might like to change. And it wouldn't help people singing out-of-tune with the reference voice.

Tune me like a guitar

Another possibility is a simple device similar to a guitar tuner clipped or mounted to a music stand. It would have a microphone, processor or specialized circuit, and multicolor LED lights. As a person sings, it would detect the closest note and distance from the pitch. For example, white or clear light for on-key, green for sharp (too high), red for flat (too low).

Again, this solution would not be very flexible. It would be able to show departure from notes in a scale, but not incorrect notes. But it could be manufactured relatively inexpensively.

An app for singing in tune

A more elegant solution would incorporate tablets or handheld devices working together to help the musicians hit their individual notes and help the group stay on key.

Tablets are already in use by musicians today. Software running on tablets can display musical scores and even keep track of the location in a score using a moving bar or by highlighting the notes or words in sequence and in proper time, much like in a Karaoke system, which displays words in sync with the music.

For our solution, we would synchronize the tablets for the group of performers and highlight notes (and words) for each individual part. And we would want to monitor each voice to indicate departure from pitch, incorrect notes, and drift over time and provide each individual with cues to keep the performance in tune and on key.

And, we'd want the director or conductor to be able to control the tempo and timing. We could do this for accompanied music by electronically tracking the accompaniment (organ or piano). For a cappella music, we could monitor the conductor's hand gestures and display a video in a small window or the background.

And here's how it could be done:

Many, if not all the technology pieces exist today. Creating a solution is simply finding the right components and "wiring" them together.

First, we will need to track the conductor and recognize tempo and dynamics, starting and stopping a piece, bringing in a section or soloist, and other vital cues. To do this, we will use Gesture Recognition. We can borrow from existing technologies and sensors in gaming to recognize the conductor's hand (and body) movements and personalize the recognition models using AI techniques (Machine Learning Models).

To synchronize the singers' displays, we can use the input from the conductor to generate and send MIDI signals to the displays (tablets). To do this, we will borrow from music production and performance systems that are able to monitor a MIDI stream and display the location in a score.

One challenge will be in maintaining location within a song during rehearsals, where there is frequent stopping, starting, repetition, etc. For this, we can incorporate touch (the conductor touches the location in the score on a tablet), or even voice recognition ("let's hear the sopranos starting at the pickup to measure 48").

For the singer displays, we would leverage already existing tablet-based technologies for input and output (microphone, video, Bluetooth, and even paired watches for haptic signals). Scores would be displayed on singer displays, with a visual location within the song, synchronized with the director or reference singer. Corrective actions to be taken would be signaled using visual methods or through various output devices.

Analytics could even play a role, letting a singer know, for example of a tendency to come in slightly late in measure eight, but showing improvement over the last three rehearsals, and providing a general analysis of a singer's problem areas with suggestions on how to improve.

A leap into the future: Singing in(to) the Metaverse.

AI models are being used more and more in music production today. Here are some fun areas to look forward to for the future.

What if AI could model the great acoustic spaces of the world (Cathedrals, Concert Halls, etc)? That is actually being done already for audio, but I envision a more comprehensive experience. Let's take the choir and send them to the Musikverein in Vienna, or another great acoustic space, using Virtual Reality and Augmented Reality.

Let's have them sing accompanied by the Vienna Philharmonic. Walk to a different location on stage and listen to the sound change. Experience being in the audience. What does it sound like in the front row? In the middle of the hall? On one of the balconies? These experiences are coming!

So you want to sing, but don't have a choir or choral group to sing with. Let's assemble a pick-up choir, with AI singers modeled after real or made-up singers. You pick the number and composition of the group, and the music you want to perform. And the conductor or director, complete with personality. This experience is coming!

What if you could sing with the Mormon Tabernacle Choir? Or sing backup for your favorite touring band? These experiences are on the horizon!

What great acoustic spaces could we create in the Metaverse? What kind of music could we perform?

Shedding the restrictions of physical spaces, experiences we've never thought of are waiting to be created. The possibilities are limitless.

Imagine, for instance, singing from the top of a mountain, but with the acoustics of a great cathedral!

Or performing on an alien planet with three suns and six moons.

Or singing for The Queen. Or for all the world’s leaders.

Or for someone who’s no longer here?

Or playing a (great) saxophone solo by singing with your voice, accompanied by your friends on piano, trumpet, and drums.

Or singing soprano with a bass voice (or vice versa). Or creating voice qualities that are not possible in our physical world.

What is your idea? I'd love to hear about it.