How opera singers are using low-latency ‘jam session’ tech to prepare for live performances

The technology is still in beta, but Matthew Shilvock, SFO’s general manager, says it’s been a godsend for the opera’s artists as they’ve rehearsed remotely and worked behind the scenes to prepare for their triumphant return to a physical stage.

“It allows a singer and a pianist to essentially be in the digital space together making real-time music—which is just transformational for us,” Shilvock tells Fast Company. “A pianist can now hear a singer breathe, and that may sound very basic, but those breath cues are the things that allow the pianist to really mold their sounds to what the singer is doing.”

“To see the emotional reaction of a pianist [who is] now finally able to hear those cues is just amazing,” he adds.

Traditional videoconferencing platforms such as Zoom or Google Meet may work well for face-to-face meetings, but they’re actually pretty lousy for real-time music collaboration. In a jam session, every beat counts, and even a tiny delay in the back-and-forth transmission of signals can be a huge source of frustration for performers relying on each other’s audio cues.

“The challenge of your traditional video technologies are that, first of all, you can only have one person making sound at a time,” Shilvock says. “And then the other challenge is that there’s a certain amount of latency. If you’re trying to make synchronous music, it makes it near impossible.”

Virtual jam-session technology has been around for a while, with interfaces such as JamKazam and JamLink long offering various low-latency solutions. What Aloha brings to the table is a platform for virtual jamming set in a familiar video-chat-like environment, including a sound-effects dashboard geared toward everyday users who don’t necessarily have technical expertise. Aloha includes a small device—about the size of an effects pedal—that hooks into a computer, tablet, or phone and runs Elk Audio’s OS.

Obviously, the pandemic has greatly increased the need for low-latency collaboration as singers and musicians around the world have been forced to keep socially distanced. (Choral singing without safety precautions is especially risky in terms of spreading the virus.) At the same time, the rise of 5G networks has enabled the faster and more reliable connections necessary for low-latency technology, according to Michele Benincaso, Elk’s founder and director. “[W]e are at a tipping point that has set the stage for Aloha’s real-time service connecting artists and instruments in a way never before possible,” Benincaso said in a statement.

Some music schools, including the Royal College of Music in Stockholm, began using Aloha for online classes last fall. San Francisco Opera is one of the first professional arts organizations to pilot the technology, according to Elk. Shilvock envisions scenarios in which, if it goes well, the platform will continue to be useful even after the COVID-19 pandemic is over—for instance, during workshops where participation from performers around the world isn’t feasible or cost-effective.

“We’re very much geographically bounded by who can come into San Francisco at a particular point in time, or which teacher can come out and spend time on the West Coast here,” he says. “So suddenly this opens up the potential to have many more people interacting with us creatively as a company. I think that’s very, very exciting.”