Ben Dowsett, FiveThirtyEight
Once statistics for an NBA game have been collected, their lifespan of usefulness is only just beginning. Players rely on them to hone skill development; teams leverage them for game-planning and scouting; fans and media alike use them to build or further a plethora of narratives.
The data the NBA collects on its referees is no different. That collection itself is just the beginning; this data is put to work in several ways that make an ongoing impact on the product seen on the floor.
Who has access to it? How is it used within the NBA’s officiating department, both for training and advancement purposes? How involved can NBA teams be in this process, and how do team decision-makers feel about it?
“We understand that referees learn best through video work, similar to players,” Matthew Futterman, the NBA’s director of referee operations, told FiveThirtyEight.
Futterman is the creator of the Referee Engagement and Performance System (REPS), a proprietary league database that officials and their supervisors have used since the start of the 2020-21 NBA season. Before REPS came along, referees largely communicated with supervisors and one another via a disorganized system of calls and emails; REPS centralized those exchanges.
Accessible via any device, REPS is made available not just to NBA officials, but also to those in the WNBA and the G League. (Some refs cross over between leagues.)
Video is at the foundation of many REPS components. The same video angles available to the NBA’s team of referee reviewers are also offered to officials themselves for the purposes of review, training and even dispute in some cases. If referees are confused about how or why they were graded on a call, they can tag their supervisor — or even fellow referees — for further discussion. All conversations among referees are kept private; supervisors don’t have access to them.
Management, particularly the NBA’s developmental advisers who oversee and train individual officials, can also use the system for ongoing development. They can tag single plays, “playlists” of 15 to 20 plays, or even long-term trends to single refs or groups of refs to focus their attention on the necessary areas. Every interaction is logged so there’s a history to draw from.
REPS also allows supervisors to view the league’s complete analytical profile and accuracy rankings for each ref.1 In coordination with the NBA’s data scientists, these features are often used to spot bird’s-eye officiating trends and take the appropriate action.
Mark Wunderlich, a former NBA official now serving as the league’s vice president of referee operations, oversees the developmental advisers who work with refs individually. Wunderlich gives a simple example from this past offseason of how the fusion between data and on-court work has improved the league’s processes.
“We realized we had [too many] incorrect decisions made from the lead position,” Wunderlich told FiveThirtyEight. “When we did the tracking of those misses, we realized the official was too tight.
“Any strong-side or down-lane drive, if an official was inside 8 feet to the baseline to the rim, he didn’t have a big enough picture to see the play. When we got outside 9 feet and a little bit wider, our vision angle expanded, and we ended up reducing our error count because the analytics proved to us that width was better from the lead position. … So when we went to [preseason] camp, we banged home plays that we had missed being too tight on video, through the help of the analytics.”
This is a level of precision that was simply not available to the NBA a decade ago, much less in prior generations.
“We’re starting to see more plays,” Wunderlich said. “I’ve had officials call me and say it’s like a whole new world out there.”
As with several other parts of the officiating program, limiting bias is a constant effort. REPS offers supervisors the ability to view trends and call data blindly, with actual referee names hidden until the analysis is completed to avoid potential favoritism.
Simple accuracy isn’t the only quality management is looking at, either. Developmental advisers also use REPS to rate their officials on themes like communication, confidence, rules knowledge and their ability to work as part of a three-ref crew while filling the proper roles.
Referees struggling with performance can use REPS as part of their pathway to improvement, a process the NBA hopes to foster — within reason. Officials who aren’t meeting the league’s accuracy and consistency standards will be put on notice and given one year to improve in their areas of weakness, per Monty McCutchen, senior vice president of referee development and training. Many are able to; those who are not are dismissed, an event McCutchen reports doesn’t happen every season necessarily, but is still a periodical occurrence.
Rest is here…
