A behind-the-scenes look at how Intel broadcasts live baseball in VR

The game that we were at was the sixth matchup that was broadcasted live on VR by Intel. The Intel True VR team had already started live streaming MLB games every Tuesday since June, but the summer classic was not the first sport it’s ever broadcasted in VR. The chipmaker’s first experience broadcasting live VR sports was actually several months ago during March Madness, when it used seven VR camera rigs per stadium to stream the NCAA finals in 360-degree views. Later in May, Intel partnered with the PGA Tour to live-stream the Players Championship golf tournament with three cameras — one at the 17th tee, one on the walkway and one in the water next to the green.

Broadcasting baseball in VR requires a similar approach, yet has its own set of challenges. As we climbed down the steps to the field on our tour, the Intel spokesperson pointed out four different camera arrays. One was tucked behind first base near the visitor’s dugout, the other was behind third base and the third was located high up behind home plate up near the press box, which shows the entire field. The latter, he says, is also known as the “High Home” shot, a camera view that’s familiar to most people who are into baseball. Each rig is equipped with six pairs of lenses, or 12 cameras total, to capture a stereoscopic 180-degree view.

“In all ballparks, we try to find three basic locations: the third baseline, the first baseline and high home,” said David Aufhauser, the managing director of strategy and product for Intel Sports, of the three aforementioned shots. “But in each stadium, we also try to find one or two more locations that are specific to that stadium.” So in Chase Field in Arizona, one camera would be by the swimming pool, a feature not found in other ballparks. At Fenway Park in Boston, one camera would be by the Green Monster, the iconic nickname for the high left-field wall. At our game, the fourth camera was located by the Giants dugout, so that viewers can see the comings and goings of players in and out of the clubhouse.

The reason the location of the cameras is so important is that the viewers actually get to pick which camera POV they want throughout the live game. If all they want to do for all nine innings is to stare at that pool in Chase Field, they can. If they want to switch camera POVs around, they can.

“In baseball, every stadium has unique features, and we try to take fans to places that they otherwise would not be able to go,” added Aufhauser. “That’s the story behind VR; to do things that you can’t normally do.”

There is one other camera in the ballpark, and it’s located way up on the scoreboard by center field. That, however, is not a VR camera; it’s just a regular broadcast camera designed to zoom in and frame the pitcher and the batter in a single shot. This, Aufhauser said, allows for a simulcast effect, whereby the pitcher-batter shot can be overlaid with the high-home shot through a virtual Jumbotron (see image for example).

Of course, if the viewer doesn’t want to choose any of these views, that’s OK, too. “We also have a fully produced view, which is very similar to a normal television broadcast,” said Aufhauser. Intel also built in a replay system into the traditional broadcast, so the team could decide to play a repeat of a great play in case the situation arises, just like you would watch on TV. This, he says, is part of VR Cast, which is an Intel-produced view in case viewers don’t want to do all the driving. “The fan can lean back and just enjoy watching and not have to decide what camera to see. We’ll help do that storytelling for them.”

And a lot of that storytelling happens in the live broadcast setup located right outside the stadium. In order to check it out, we were led through the Giants’ clubhouse and went out the north side of AT&T Park, where we were greeted by a white van and a smaller white trailer underneath a stretched-out tarp shelter. The white van had the “Intel True VR” logo emblazoned on the side and there were wires and cables running in every direction.

When we peeked inside the aforementioned van, we saw a crew of about six people behind what was the control room for the VR broadcast. It was about an hour or so before the game, and the editors were busy making sure the left and right views of each camera were synced up and ready to go. When the game is live, the stitching happens entirely in real-time through Intel’s True VR software.

And there’s more. When a viewer watches the game through the Gear VR, she’ll see the scoreboard above and the camera and player controls below. If she wants, she can then bring up live in-game stats on either side of the video. When the game starts, for example, you’ll see the lineup of each team, along with the names of the players and their positions on the field. Throughout the game, you’ll see more stats, like the number of runs, hits, balls, strikes and innings pitched per player. You can toggle these stats from side to side, and even bring them to the forefront if you want to pay more attention to them.

As for audio, Intel uses a mix of in-game audio that the camera picks up as well as the audio feed from radio and TV, which tends to be pretty high-quality. Interestingly, Intel hired a national broadcaster by the name of JB Long to do the play-by-play commentary each week. During our tour, he was sitting in the small trailer next to the van, where he provided commentary on the game while watching the video feed.

“We have to imagine our audience differently,” said Long, when asked about the difference between hosting a VR broadcast versus a traditional one. “Not only are we doing a national game, we’re likely doing it for an audience that’s more technology-savvy and maybe less baseball [inclined].”

Long said that much of the commentary is about the experience of being at the park, not on the game itself. “Not that the game is secondary, but what makes our product unique is not the score, but the environment.” For example, Long talked about how cold it was at AT&T Park in July, which is how it often is in San Francisco. “To my knowledge, that is what VR is about,” he said. “It’s delivering really unique experiences without needing to be onsite.”

In addition to Long, Intel hired an announcer to fill in 90-second-or-so gaps between innings. The reason? Because the VR broadcast doesn’t have commercials that a regular TV show does.

Another challenge of live VR broadcasting is that the team has to create graphic overlays on the fly. They can’t conjure traditional lower thirds with player names, for example, because there are so many different cameras and so many different angles. Instead, Intel has someone in the trailer truck create a baseball-card graphic to float over players’ heads. The graphics also have to be offset slightly to give the right depth. Plus, because the players can sometimes move, the cards aren’t always in the same space all the time and have to be moved around. It’s a seemingly small feature that requires a lot of additional work because it’s in VR.

“The hardest part is not the number of cameras,” said Shaun Carrigan, the experience producer for Intel Sports. “The hard part is how do you give a really great experience in the headset; we can’t just watch it on the screen, we have to use it on the headset too.”

I watched a tiny bit of the game on the Intel True VR app on the Gear VR, and it was a little surreal at first. I was able to view the game as if I was right on the field, and it was kind of cool to be able to see players up close and personal if I wanted to. Changing camera views did take a few seconds of buffering time, though, so you’d potentially miss a few seconds here and there of the game if you like switching camera angles a lot. I also liked having a 360-degree view of the game, with the stats on either side of the video, and the ability to pause and replay the game just with my gaze.

Yet the experience wasn’t perfect. For one thing, the resolution just isn’t as sharp as watching the game on a high-def TV set. Also, you have only those four aforementioned camera positions to choose from. You can’t joystick around and choose whatever angle you like, so getting to see what the game is like from a player’s POV is out of the question.

But it’s still early days for live VR, and that technology could arrive eventually. “We’re just starting to see that as a possibility,” said Aufhauser. “Imagine being able to be one of the players and feel what it’s like for a 90 mph fastball coming down at you.”

That could come from a volumetric capture tech that Intel has thanks to its acquisition of Replay Technologies last year. Right now, the tech is used only for freeze frame 360-degree shots, which has already been used in certain NBA and NFL games. But the time could come in the future where you would have free reign to view any game from literally any angle.

“If you’re live at a game, you have the experience of being there,” said Carrigan. “If you’re watching TV, you get to see close-ups. With VR, you get to have both.”

As fascinating as the VR experience is though, I have to say that it still doesn’t quite beat being at a game in person. For me, it’s not enough to pretend as if I’m on the field or in the dugout with the players. Being at a ballpark is more than that. It’s about the camaraderie of strangers rooting for a common cause. It’s the thrill of being mere inches away from your favorite player. It’s the possibility that you just might catch that home-run ball. It’s holding a cold beer in one hand and a hot dog in the other while jumping for joy at a great play.

Watching a game in VR is certainly more interactive than on TV, and I look forward to what VR sports will look like in a few years. But until then, you can find me by the bleachers, rooting for my home team along with 40,000 other die-hard fans.