Ben Dowsett, FiveThirtyEight
The concept of constant evolution in sports isn’t exclusive to athletes and tactics. It’s central to the men and women who officiate the games as well.
Referee mechanics — where to stand, how to move, who makes which calls — weren’t taught uniformly in the NBA until the early 1980s, over 30 years into the league’s existence. The use of VCRs and videotapes for referee training wouldn’t begin in earnest until a few years later. The NBA didn’t even begin using three officials in its games full-time until 1988.
As the world around it has modernized, though, so has the NBA. Today’s league officiating department relies more than ever on a well-known sports buzzword: Analytics.
Every single call made by NBA referees — and many of those not made — is graded by impartial observers, then inputted into a vast database including every official in the league. This data is used in ongoing referee training and development, and it helps in determining ref promotions and playoff assignments. Teams are even given partial access to and are allowed to make some inquiries into specific calls.
This is no mom-and-pop setup. It’s a full-fledged operation involving more than two dozen full-time staffers, from former NBA officials to outside consultants and quantitative analysts. It’s also a realm into which the public has had very little window, even as analytics and technology have become larger and larger parts of the league’s officiating department in recent years.
FiveThirtyEight spoke with more than 25 people across the NBA, from league staffers in charge of data collection and use to front-office executives, to understand exactly how this referee-grading and analytics program works.
Many an NBA fan recognizes Secaucus, New Jersey, as home to the league’s Replay Center, but that’s just one small piece of league and referee operations taking place there. Secaucus is also the NBA’s hub for its referee review and grading program, which employs a staff of over 25 people in various roles.
A key figure here is Steven Angel, one of the NBA’s longest-tenured employees at nearly 14 years. Angel was part of a consulting firm that helped redesign the league’s referee data in the early 2000s when David Stern was commissioner; he was eventually brought on full-time as part of the officiating department, now occupying the role of senior vice president of game analytics and strategy.
When Angel started working with the league, the sophistication of referee reviews and grading systems matched the limited technology that was available. Around the turn of the millennium, the NBA started assigning observers to attend games in-person in their geographical area, then return home and break the same game down again via DVR later that night. Angel, with his consulting background, noticed potential problems with training and arena biases, and the system was brought entirely in-house in 2013.
When the NBA brass decided to begin sharing referee game grades and reports with individual franchises in 2015, they quickly realized that many teams were focused on different call and play types than the league’s own referee advisers — understandable given that teams have their own interests at heart, while advisers are focused on leaguewide referee performance. Standardizing these definitions across the board became a major piece of Angel’s department, one that persists to this day under the title of “Rules Clarity Project.”
Those involved work directly with the NBA’s competition committee — which comprises a group of owners, general managers, coaches, referees and players — plus representatives from all 30 teams and the referee’s union. Angel sums up the aim of this project simply: “What actually constitutes an error? The game is one long non-call, except when the whistle is blowing,” Angel told FiveThirtyEight. “Can we agree on what constitutes a foul?”
Angel’s team also handles the realm of integrity. It looks for any and all possible indicators of bias, whether conscious or subconscious. The name “Donaghy” is rarely uttered in league circles today, but there’s an obvious desire to protect against even the suggestion of impropriety — especially given the NBA’s own stated interest in the arena of sports gambling.
“We monitor gambling lines in Las Vegas,” Angel said. “We look to see if there’s any indication of manipulation or bias. That is hopefully a big, big waste of time, but we are still diligent in that effort.”
The biggest element Angel oversees, though, is the NBA’s staff of dedicated referee reviewers.
Rest is here…
