When adding technology to broadcasts, ESPN aims for addition, not distraction

Jed Drake watches sports on television a little bit differently than most. While the majority of sports fans watch the action, listen to the announcers and track the score, Drake is on the lookout for something a little more esoteric.

As ESPN’s senior vice president of product innovation, Drake is charged with -- among other things -- working across departments to develop new technologies to give viewers a better understanding of what’s happening on their screens.

One of those ideas came to Drake in March 2015 while he was in Orlando, Florida, helping to develop two since-launched ESPN on-screen enhancements, Pylon Cam and K-Zone, at the network’s Innovations Lab at the Wide World of Sports complex. Drake was having breakfast with colleague Marv White when one of those opportunities arose. The pair were discussing high dynamic range video technology when Drake had an idea, and interrupted White in the middle of a sentence.

“I asked, ‘Has anybody lit up the 3-point line yet?” Drake said. “[White] said, ‘I don’t think so.’ ... I said, ‘We ought to be able to do that.’ It would be really cool if we could light up that line.”

Why? One reason is obvious: The rise of the 3-point shot, led by the Golden State Warriors and two-time NBA MVP Stephen Curry. In the 2015-16 regular season, there were 20,953 3-pointers made in the NBA. In 2010-11, that number was just 15,886, and in 2005-06, it was a mere 14,086. That’s a 49 percent increase over the past decade.

From a TV perspective, Drake saw two key benefits to the feature: (1) allowing viewers to know whether a shot was a 3-point attempt, even if the shooter was blocked by fans, defenders or ESPN’s own on-screen scoreboard; and (2) allowing those watching games without sound -- primarily at sports bars -- to know immediately whether a shot would count for two or three points.

Ten months later, the concept became content, as ESPN debuted what the network calls “Virtual 3” on its Jan. 30 broadcast of a game between the Cleveland Cavaliers and San Antonio Spurs.

To the average viewer, it’s a relatively simple effect. When a player attempts a 3-point shot, the line lights up. If the shot misses, the illumination ends immediately. If the shot goes in, the line remains illuminated until the opposing team begins its subsequent possession.

If only building out the technology was that simple.

To create Virtual 3 for viewers, the ESPN technical team had to deal with complex operational challenges, ranging from uniform colors to arena lighting to equipment transportation.

“We’ve always said this -- and it may sound like a stock line -- but it’s about enhancing the experience,” Drake said. “If we don’t do that, we’ve missed the point. … A good example is the glowing puck from Fox when they first had the NHL. It was possible via new technology, but it was determined that it didn’t help the viewing experience the way they had hoped.”

But for every glowing puck failure, there are smash hits like the 1st & Ten line. It’s hard to recall a time, in fact, when the first-down marker wasn’t highlighted on an NFL broadcast, but that technology isn’t that old. Designed by Sportvision, 1st & Ten debuted on ESPN in September 1998 -- Drake was ESPN’s lead on getting it launched -- and that feature has indisputably changed how we view NFL games.

But when it comes to viewer impact, most on-screen enhancements fall between glowing pucks and illuminated down markers. Virtual 3 is one of those.

“The 1st & Ten line was a once-every-20-years thing,” Drake said. “When we did the 1st & Ten, within six weeks, CBS had it on the air. … The Virtual 3 is cool, but it’s not a 1st & Ten line.”

Public reviews of Virtual 3 were generally positive, if not wildly enthusiastic.

Awful Announcing polled readers, and a little less than half of thems aid they didn’t like it. “The Virtual 3 as it is now was pretty ‘meh’ to me,” wrote Awful Announcing’sBen Koo. “I get the idea and it was executed well, but I’m not sure there was an actual need there or value to add to the viewing experience.”

The Verge’s Dante D'Orazio was a little more bullish. “The implementation … looks slick. Sure, it's not really necessary, but it's not too obnoxious, and it can provide some useful information on close shots.” And SB Nation was the most positive of all: “It's definitely not too intrusive, and it helps watchers know what type of shot it was at a glance. … It's not as awesome as the pylon cam, but it's not as bad as the glowing puck, either. Good job, ESPN.”

I would tend to agree with the general tenor of these comments. The Virtual 3 feature imparted useful information in a relatively simple fashion, automatically answering the frequent “Was that a 3-pointer?” question. But the 3-point shot doesn’t come into play on every possession, and Virtual 3 is truly only valuable on shots close to the 3-point line. The 1st & Ten technology was so impactful, by contrast, because down and distance is essentially the heartbeat of a football game.

And, unfortunately for ESPN, Virtual 3’s inaugural season ended earlier than anticipated, as the network chose not to use it for the NBA playoffs because of an audio delay issue that caused a slight disconnect between the announcers’ calls and the illuminating of the line. According to Drake, the audio wasn’t a problem on ESPN broadcasts, but proved a nagging issue for ABC broadcasts.

“I am fairly confident that the system will be back next season, albeit with faster processing, which will reduce the audio delay,” Drake said.

The impact of new technology usually doesn’t have a proportional relationship to the effort required to create it, and ESPN’s on-screen enhancements are no exception. The Virtual 3 was a technical challenge that, in many ways, exceeded that of the 1st & Ten line.

“It’s that much harder to do, which means it will be harder for people to replicate,” Drake said, taking the glass-half-full approach.

But why was Virtual 3 more difficult to build and execute? It’s just a line that lights up, right? Hardly.

“I don’t think anybody has an idea what goes into doing TV in general,” said ESPN’s Chris Strong, with a laugh. ESPN’s senior remote ops specialist, Strong was one of the members of a core project team that included Drake as well as Bob Toms, ESPN’s vice president of production enhancements and interactive TV, and Jay DiGiovanni, the network’s vice president of technology enhancements.

To bring Virtual 3 to life, this team had to clear three hurdles: (1) determine when a shot was actually a 3-pointer; (2) create the technology that would allow for the illuminating of the 3-point line; and (3) integrate that technology seamlessly into a live broadcast.

Determining whether a shot is a 3-pointer seems like the easiest task, but it was no cakewalk. The team began with the idea that a mobile rig featuring overlapping cameras would be the best solution to optically recognize whether or not a shooter’s feet were behind the 3-point line.

DiGiovanni took the lead on that piece of the operation. He joined ESPN in 2010 when his company, PVI Virtual Media Services, was acquired by the network. DiGiovanni’s team -- located in Hamilton, New Jersey, far away from ESPN’s main campus -- built a rig comprised of 10 cameras that would provide definitive data on whether a shot was a 3-pointer or not. But there were issues. For one, cameras could be blocked by players. Even with multiple cameras, it was possible to have multiple players blocking all available angles.

More important, the camera rig that was built was just too large, too heavy and too expensive.

“We sort of realized to take this rig with all these cameras and move to every NBA arena and hoist it to the ceiling would be a cost we could not overcome,” said DiGiovanni, though he noted a more technical shot-recognition solution is still being explored for next season.

Ditching the camera rig meant a simpler short-term solution was needed, and it soon came. “Someone finally said, ‘Instead of all this, why don’t we watch the referee put his hand up?’” DiGiovanni said with a laugh.

But even that wasn’t as simple as it sounds, because the team soon realized that the referee signaling a 3-point shot wasn’t always visible in the game broadcast. That meant someone needed to be in the arena to make that call, and sitting somewhere unobstructed by fans -- or anything else, for that matter. So ESPN rotated two retired NBA referees, Bennett Salvatore, and Luis Grillo, for each game in which it deployed Virtual 3.

And the high-tech equipment Salvatore and Grillo used to signal a 3?

“A push button,” DiGiovanni said. “We didn’t want to invent anything crazy.”

With that issue resolved, the next was how to create the illumination effect.

“The 1st & Ten line is a straight line and pretty wide,” Drake says. “We can put it wherever we want. There’s no measure to see whether its accurate [since the first-down line isn’t always in the same place]. The 3-point line has to overlay perfectly. And it’s also curved, so it’s much more difficult. … The Virtual 3 line is a very thin line and only a couple of pixels wide. In TV tech, thinner is harder; the easier it is to get lost.”

There were also issues specific to basketball arenas. “You have a colored line over a floor surface that, in various forms, is the same color as human skin. Unlike a football field, which is not shiny, basketball arenas have banks of lights, so there’s massive amounts of light that shine down.”

This can wreak havoc on computers trying to read the colors on the court and player uniforms.

“In Houston, the Rockets wear red, and the line could be red,” Strong said. “In Boston, every 2 square feet is a different color of parquet. … The trick is to make sure the computer can differentiate the line.”

Eventually, the team settled on a shade of red that didn’t match any team uniform colors, and more important, was suitably unobtrusive.

“We wanted the viewer to be able to see it without taking their eye off the ball,” Toms said. “We don’t want the viewer to be distracted by it, but want the viewer to know it’s a 3.”

At that point, the system was ready for testing, which began last September at ESPN’s facility in Orlando. In October, Drake and Tim Corrigan, ESPN’s senior coordinating NBA producer, met with NBA broadcast executives to get their approval to use Virtual 3.

“At that time, all we had was the demo that we had built in an edit suite,” Drake said. But that was enough to convince the league, which green-lit its use for live broadcasts.

The system was then tested off-air for more than a dozen NBA games. By the third week of January, the team felt Virtual 3 was ready. One week later, it debuted, and thanks to social media, viewer reaction came in fast.

“When we launched the 1st & Ten line, there was no social media,” Drake said. “With the Pylon Cam, we were watching social media when it was rolling out. If you looked at social media, we were providing shots [with Pylon Cam] people had never seen before. The Virtual 3 has been more of a mixed bag. You have to decide how much stock you want to put into [social media]. We got some responses like, ‘Thanks for showing me a line that is already there.’”

Toms didn’t bother to check social media reactions, saying, “I have no control over that. I do have control over the execution of the idea.”

Toms also had a lot of control over ESPN’s creation of Pylon Cam, which debuted in January 2015, and featured, quite literally, a camera in the pylon marking the end zone in NFL and college football games. And similar to Virtual 3, there were plenty of challenges to overcome.

“The first goal was safety, to make sure this was safe so that no matter how hard a player hit it, it gave,” he said. “We didn’t care if we lost the picture, or whether the camera was broken. … We needed a structure that would hold cameras in place but would immediately disintegrate if a 250-pound player hit it.”

The solution was cutting-edge: Use a 3D printer to create camera mounts that were technically made of plastic, but came apart like hard sand if struck with any force.

“We also had to figure out ventilation, as cameras put out some heat,” Toms said. “We figured that part out; we got Gilman Gear [the NFL’s official supplier of pylons] to work with us to create a new mold.”

The last part of the puzzle was how to get power to the 16 cameras -- four in each of the four end zone pylons. ESPN first attempted a wireless solution, and, in fact, used that configuration when it debuted the Pylon Cam for the FCS championship game and College Football Playoff National Championship in January 2015.

In the end, though, the wireless connection wasn’t consistent enough for live use. “I didn’t want to be in a position where I said, ‘I hope we got [the shot],’” Toms said.

So ESPN met with field managers from three different NFL teams -- two with natural grass fields, and one with turf -- to discuss a solution. They soon determined they could run a wire out from under the pylon to a nearby power source. ESPN debuted the wired solution for the 2015 season debut of Monday Night Football. Then the waiting game began. “We were kind of waiting the whole season for [that first great shot from the Pylon Cam],” Toms said.

On Dec. 14, Toms was watching the MNF game between the New York Giants and Miami Dolphins, but more because he’s a Giants fan than an ESPN employee. But as soon as he saw yet another Odell Beckham Jr. end zone circus catch, his thoughts immediately went back to the office.

“As soon as that play happened, I thought, ‘Please …’” Toms said. “Then [the Pylon Cam shot] came up, and it was exactly the shot we thought we had. I knew we had that covered."

But the shot wasn’t just a creative success for ESPN; it actually impacted the game. The Pylon Cam’s shot was so clear, it provided all the evidence necessary to change the referee’s original call of an incomplete pass into a touchdown, making Toms, indirectly, the first fan in history to ever help his team from his couch.

In the end, though, impacting the game isn’t the driving force behind features such as 1st & Ten, Pylon Cam and Virtual 3. Instead, the goal is simple: Help fans better enjoy the game.

And that means getting them past the initial reaction of “What the hell is that?” when seeing these new technologies to quickly wondering, “Where the hell is it?” when the new features are not part of a broadcast they’re watching.

“In the end, the judge and jury has to be the viewer,” Drake said. “If we’re doing it for fun and intrigue, we’re totally misplacing our priorities.”