How Relative Rankings are Calculated

January 1, 0001 - 4 minutes

The first layer of relative rankings

The goal of relative rankings is to simulate a world where every NCAA team plays each other multiple times using what I refer to as “surrogate teams”. The concept of a surrogate team is demonstrated in the diagram below. In the diagram we have two teams (Team 1 and Team 2) who have both played the same team (The surrogate team) and recorded the statistic displayed above the arrows. For simplicty, lets consider this to be points per a trip (PPT). Team 1 recorded 1.33 PPT versus the surrogate compared to 1 PPT by Team 2. The ratio of these values can be interpreted as a measure how much better Team 1 performed, in terms of PPT, compared to Team 2 and vice-versa. We can refer to this as “surrogate PPT”. A value lower than 1 intuitively indicates that Team 2 performed worse than Team 1 against the surrogate.

The real power of this method comes when it is scaled up for an entire season. Now lets take Team 1 and say for an entire season they play 32 games. For each game played against a given team, we can calculate \(n\) surrogate PPT values equal to the number of games played by that team minus those against Team 1. Each surrogate PPT will represent how Team 1 performed respective to the surrogate team’s other opponents. For simplicity, lets say each opponent of Team 1 has an \(n\) value of 32 (A realistic situation). We now have \(32x32\) surrogate PPT values. So now instead of having 32 data points to judge Team 1 on, we have 1024 data points. Another factor that makes this particularly powerful is that the vast majority of teams have at most 2 degrees of separation between them meaning a pairwise comparison can be made between most teams. An additional strength, is that this method completely removes the need for strength of schedule measurements as everything is measured relative to other opponents who have played the same team.

The second layer

With a few exceptions, all NCAA teams are separated by 1 or 2 teams. The above example covers the situation where there is one degree of separation between Team 1 and Team 2, but we can also include examples where there are two degrees of separation in our ranking calculations. In the below diagram I demonstrate the second level of relative rankings. In this example Team 1 from the first diagram has not played any of the same teams as Team 3. However, Team 1 and Team 3 have both played against teams who have played against Team 2 and, as a result, we can directly compare their surrogate PPT values (These are derived in the same manner as the first diagram).

In this example the number above the arrow is each team’s surrogate PPT relative to Team 2. The ratio of the surrogate PPT values is used to compare Team 1 and Team 3 on the basis of each team’s surrogate PPT value relative to Team 2. One can extrapolate how we can continue this process, adding more data points with more degrees of separation.

This second layer leads to an exponential increase in data points. For the 2019-2020 season, we have 110 million total second-level comparisons between teams. With 110 million data points, we can comprehensively rank teams in quality and more importantly we can begin ranking teams very early in the season in an accurate manner albeit with much less than 110 million comparisons.

How are final rankings calculated

In short, all rankings are calculated using the median of the second-level surrogate statistic. Offensive quality is calculated based on points per a trip, defensive quality is calculated based on opponent’s points per a trip and pace is calculated by total number of trips. “Per Trip” statistics are used because they provide a single measure that encaspulates all aspects of the offense such as turnovers, offensive rebounds and shooting. Each team has roughly the same number of trips (Give or take 2) in a given game and, as a result, the team with higher points per trip wins 99.9% of the time.

In the case of offense higher values are better and for defense lower values of better. Overall ranking is calculated by calculating the difference between relative points per trip and relative defensive points per trip. Teams closest to 1 perform as expected and can be thought of as the median team. For the 2019-2020 season these teams were George Mason on offense, Eastern Washington on defense and Montana overall.