UFC 2014 Year in Review

I’ve been working on a program that allows measurement of the performance of MMA fighters over time, without tunable parameters or the need for human judgement. I used it to calculate the current top five fighters in each weight class in the UFC, then looked at how their scores varied over the last year. Tweaking the code took a little longer than expected, so this is actually 2014 and a little bit of 2015 in review as the report ranges from January 1, 2014 to January 19, 2015, ending immediately after UFC Fight Night: McGregor vs. Siver.

The program is based on two different algorithms across five different histories of 1, 2, 4, 8, and 16 years. The results are then evenly weighted to produce a single confidence measure. The closer a score is to 100%, the more confident the model is that a fighter has won frequently against other winners. A score of 0% means the model has no information – a debuting fighter starts at this. The closer a score is to -100%, the more confident the model is that a fighter has lost frequently to other losers.

It’s best to think of these scores as a report card of actual performance, rather than a theoretical ranking of potential performance. A solid contender fighting frequently and consistently winning might outrank a champion who only fights once a year, and you can check out the lightweights for a shining example of this. It’s not about some abstract measure of skill – it’s about who’s winning the most fights against the best opponents right now.

Flyweight

FlyweightPlot-20140119

Demetrious “Might Mouse” Johnson ranks head and shoulders above his division with two successful title defenses in June (against Ali Bagautinov at UFC 174) and September (against Chris Cariaso at UFC 178). Joseph Benavidez broke out of the pack to take second with a win over Dustin Ortiz in late November at UFC Fight Night 57, but still has a long way to go. John Dodson defeated John Moraga in June at UFC Fight Night 42, but by the end of the year both were closely matched. Jussier Formiga’s win against Zach Makovsky in August at UFC Fight Night 47 brought him into the top five.

Bamtamweight

BantamweightPlot-20140119

This closely matched division has Raphael Assuncao, Renan Barao, Urijah Faber, and TJ Dillashaw all vying for the top spot. TJ Dillashaw catapulted himself into the elite with a win over Renan Barao in May at UFC 173, but needs a win over a strong opponent to really cement his status as champion. Raphael Assuncao had a strong year with wins against against Pedro Munhoz in February at UFC 170 and Bryan Caraway in October at UFC Fight Night 54. Renan Barao stayed in contention after pulling out a must-win fight against Mitch Gagnon in December at UFC Fight Night 58. Urijah Faber is perpetually a contender, but a recent title shot loss to Barao in February at UFC 169 hurts his near term chances. Dominick Cruz got a big bump in September at UFC 178 with a win against Takeya Mizugaki, but recurring knee injury issues will likely keep him out of competition (and the top of the division) in 2015.

Women’s Bantamweight

WomensBantamweightPlot-20140119

Alexis Davis and Ronda Rousey were closely matched until Rousey’s victory in July at UFC 175 cemented her at the top of the division. Cat Zigano got a big bump in late September at UFC 178 with a win against Amanda Nunes, and will challenge Rousey for the title in February 2015 at UFC 184. Bethe Correia and Jessica Andrade round out the top five.

Featherweight

FeatherweightPlot-20140119

Jose Aldo can deservedly call himself the king of this division, but Conor McGregor is climbing fast. McGregor’s ranking was aided by an aggressively paced fight schedule with victories over Diego Brandao, Dustin Poirier, and most recently Dennis Siver all in the last six months. Aldo and McGregor are being positioned for a title fight at UFC 187 in May 2015. Ricardo Lamas defeated Dennis Bermudez via submission in November at UFC 180 and these two fighters fill out spots three and four. Cub Swanson had a big win over Jeremy Stephens in June at UFC Fight Night 44, but is struggling to maintain momentum after a loss to Frankie Edgar later in the year.

Lightweight

LightweightPlot-20140119

The interesting rankings in this division reflect activity (in the case of Donald Cerrone) and lack of it (in the case of champ Anthony Pettis). Cerrone has been on a tear, and his fast paced schedule against strong opponents has led to his ranking skyrocketing. Pettis has been the opposite, with only a single fight in 2014 and his ranking slowly trends downward as a result. The biggest shakeup in the rankings came in August at UFC Fight Night 49 when Rafael dos Anjos knocked out Benson Henderson demonstrating he belongs near the top of the division. Benson Henderson has been on a slow and steady slide since then, erasing all his gains and then some from a major win against Rustam Khabilov at UFC Fight Night 42 in June. Micheal Johnson rounds out the top five.

Welterweight

WelterweightPlot-20140119

This division is a bit of a mess, despite it being quite clear who the top three are – Johny Hendricks, Robbie Lawler, and Rory MacDonald. Lawler became champion in December at UFC 181, but ranks just below Hendricks. Rory MacDonald defeated Tarec Saffiedine in October at UFC Fight Night 54 to cement his contender status, but a talked about title shot never materialized. Rick Story defeated Gunnar Nelson in October at UFC Fight Night 53 to rise to number four. Matt Brown has been consistent across 2014 and rounds out fifth.

Middleweight

MiddleweightPlot-20140119

Weidman defended his title with a win over Lyoto Machida in July at UFC 175, but his injuries and fight delays have led to a somewhat sleepy title picture in 2014. Ronaldo “Jacare” Souza was a strong contender after defeating Francis Carmont in February at UFC Fight Night 36, but his ranking has steadily slipped since then and is now sitting in fourth. A planned bout with second place Yoel Romero has been delayed, and the winner of this will likely fight for the title. Thales Leites’ biggest win of the year was against Francis Carmont in August at UFC Fight Night 49, pushing him to third, while Tim Boetsch rounds out the top five.

Light Heavyweight

LightHeavyweightPlot-20140119

No debate here, Jon Jones is the clear champion at 205 after a decision win over Daniel Cormier at the start of the new year. The real question is who’s up next for the title. Ryan Bader’s victory over Ovince St-Preux in August at UFC Fight Night 47 pushed him into second place, while Phil Davis defeated Glover Teixeira in October at UFC 179 to take third. Anthony Johnson had a strong return to the UFC with a victory over Phil Davis in April at UFC 172, and only his lack of UFC activity in 2012-2013 prevents him from being ranked higher than fourth. Ovince St-Preux bounced back after his loss to Ryan Bader with a win over Mauricio “Shogun” Rua in November at UFC Fight Night 56 to retain his status in the top five.

Heavyweight

HeavyweightPlot-20140119

Probably the most egregious example of an inactive champion, Cain Velasquez hasn’t fought since October 2013 when he defeated Junior dos Santos for the second time at UFC 166, and his ranking has been on a steady slide for all of 2014. Junior dos Santos picked up a win against Stipe Miocic to rise to first in the rankings in December at UFC on Fox 13, but after two brutal losses to Velasquez his future in the division is unclear. Fabricio Werdum snagged a critical win against Travis Browne in April at UFC on Fox 11, then defeated Mark Hunt in November at UFC 180 to win the interim heavyweight championship. Werdum and Velasquez are tentatively scheduled to meet in June 2015 at UFC 188. Travis Browne defeated Brendan Schuab in December at UFC 181, and Matt Mitrione defeated Gabriel Gonzaga in December at UFC on Fox 13 – both are hungry for their shot but are a fight or two away from the title.

Caveats

The model is strongly sensitive to activity – the more frequently someone fights (and more importantly, wins) the higher they will rank in general. If a fighter hasn’t fought in the last year, they automatically lose 20% of of their total score as two out of the ten scores will be zero (the one year periods). It also only uses UFC and Pride fight data, which leads to certain active fighters with strong careers in other organizations being underrated and not being considered for the top five, like Luke Rockhold at middleweight. Adding in fight data from other organizations is definitely on my to-do list.

Quantifying the Career of Georges St-Pierre

I recently decided to improve my graph theory based MMA ranking system which modeled fighters as vertices and wins as edges on a directed graph. Previously the system generated a single large list of fighter rankings, evaluating all fights on record to generate data. The only problem is that this does not show how these rankings have changed over time – for instance, how has the ranking of Georges St-Pierre changed over the last decade? His recent retirement had made me curious.

I decided that instead of running this analysis for a single day in the present and looking back at all fights in the past – why not run the analysis for a given arbitrary day, and limit the fights which are included in the analysis to only those fights occurring within a certain period of time before that day? This analysis could then be ran and re-ran as I iterated day by day to obtain data demonstrating how rankings changed over time for a certain lookback period.

I ran this analysis between January 1, 2004 (roughly one month before Georges St-Pierre’s first fight in the Octagon) to the most recent UFC event at the time of this post (UFC on FOX 12, July 26, 2014) for lookback periods ranging from two to sixteen years in the past. To provide context I also ran this analysis for every UFC welterweight champion during this timeframe (Matt Hughes, Matt Serra, and Johny Hendricks). A visualization and brief summary of this data can be found in the table below. The color of the graphs range from green (shortest lookback period of 2 years) to yellow (longest lookback period of sixteen years).

varlb-hendricks
Johny Hendricks: Arriving in the UFC in mid-2009, Hendricks demonstrated strong performance in the shorter lookback periods but simply hasn’t had enough time to climb up the rankings in the longer term.
varlb-hughes
Matt Hughes: A legend of the old guard, his shorter lookback period rankings started to slip in 2007 but still ranks highly in the long term, evidence of years of dominance.
varlb-serra
Matt Serra: Saw big gains in 2007 with his defeat of George St-Pierre but couldn’t hold on to them. Note the dropoff in scores into negative territory when the lookback period no longer considers his Georges St-Pierre win (ie early 2009 in the 2 year lookback period, early 2011 in the four year lookback, etc).
varlb-gsp
Georges St-Pierre: A steady gainer who got a big pop in the mid to long term rankings in late 2006 after defeating BJ Penn. Short term lookback periods show weaker performance post-2010 due to injury and slower scheduling, but long term ranking are strong.

So how are we supposed to compare performance between fighters? Thirty two time series with eight separate lookback periods is a mess to visualize effectively, so we need to find the best lookback period or periods to compare. Shorter lookback periods respond quickly to recent wins and losses, but are fickle, ignoring long term consistency. Longer lookback periods are more stable and reflect consistent performance, but take years to reflect possibly rapid changes in fighter performance. After evaluating many different approaches, I decided to average the results from the 2, 4, 8, and 16 year lookback periods. Each lookback period is sufficiently different from the other and provides new information, and the combination provides a good balance between short and long term viewpoints.

gsp-career-mmagraph

The resulting chart makes qualitative sense and shows the early dominance and slow fade of Matt Hughes, the rise of Georges St-Pierre, Matt Serra’s spike when he captured the welterweight championship, and the new guard of Johny Hendricks quickly approaching. We can also look at the day to day changes in George St-Pierre’s ranking to capture some events that influenced the changes in his ranking most heavily.

Biggest One-Day Gains Biggest One-Day Losses
March 4, 2006 +0.232 Georges St-Pierre wins by split decision against BJ Penn in a close battle at UFC 58. April 7, 2007 -0.119 Matt Serra defeats Georges St-Pierre in an infamous upset at UFC 69.
November 18, 2006 +0.204 Georges St-Pierre defeats Matt Hughes by TKO in the second round at UFC 65 to become the new UFC Welterweight Champion. September 23, 2006 -0.096 Matt Hughes defeats BJ Penn at UFC 63. Note that this occurred shortly after GSP defeated BJ Penn for a massive gain in the rankings. BJ Penn’s drop in the rankings as a result of this loss also caused GSP’s ranking to slip as a second-order effect.
August 9, 2008 +0.203 Georges St-Pierre defeats Jon Fitch at UFC 87. September 23, 2006 -0.067 Thiago Alves defeats Matt Hughes by TKO at UFC 85. A large portion of GSP’s rankings relied on beating Matt Hughes. As Hughes’ ranking slipped due to losses, GSP’s did as well due to second order effects.

Most of George St-Pierre’s big gains occurred earlier in his career, which makes sense – it’s hard to move up when you’re already the champion. His worst days were also the predictable loss to Matt Serra, and the days when his most prominent prior wins (BJ Penn, Matt Hughes) ended up losing themselves.

And when was Georges St-Pierre’s peak? Well by the numbers, it was the week shortly after his defeat of Dan Hardy at UFC 111 when he reached a career high of ~0.727, at roughly the start of the controversy about him finishing fights. A year and a half later, he would injure his ACL which reduced the frequency of his fights to such a degree eventually his ranking slid below that of a young upstart named Johny Hendricks. The two eventually fought on November 16, 2014 – and Georges St-Pierre won, for a one-day gain of 0.120, his seventh best on record.

After all, if you’re going to retire, you might as well go out on top.

Graph Theory, Algorithmic Consensus, and MMA Ranking

I’ve been working on an objective ranking system lately that could be applied to groups with large numbers of individual competitors, like the sport of mixed martial arts (MMA). The biggest issue compared to typical ranking systems is that there are so many participants that they cannot all compete against each other in a round-robin tournament or similar within a reasonable time frame. In order to calculate a global ranking, all of the players must be compared against each other through a sort of “six degrees of separation” style comparison, which is vulnerable to bias and calculation error.

This problem has already been solved in the chess world with the Elo rating system, a statistical approach that requires frequent competition in order to generate statistically significant results. Unfortunately competitors in sports like mixed martial arts or boxing do not compete nearly as frequently as chess players (for obvious reasons) and this approach drowns in a sea of statistical noise. Typically combat sport rankings are done by a knowledgeable observer by hand, through consensus of many observers, or by models with a large number of tunable parameters. It is very interesting to consider that humans appear to be able to easily determine who should be ranked highly, and that many algorithmic approaches largely match these evaluations but make some seemingly obvious mistakes. My goal was to find an approach that produced rankings that seemed sensible to a human observer with a minimum of tunable parameters (preferably none).

Data Structure

The initial step is to structure our data in a sensible way. We have a large number of participants, connected by individual competition which can either result in a win or a loss. One way of structuring this data would be in a directed graph, where competitors are represented by nodes and matches as edges with direction defined by who wins or loses. We seem to be focused on losses (or win/loss ratio) as the biggest factor – a competitor with 40 wins and zero losses is typically regarded as better than a competitor with 60 wins and 20 losses. Let’s set the direction of the edge from the losing competitor to the winning competitor. A “good” competitor’s node will therefore have many incoming edges and few outgoing edges, and tend to be at the center of force-directed graph layouts.

Evaluation Algorithms

There are many possible evaluation algorithms which will produce a ranking from this data structure. After many trials, two appeared to stand above the rest.

  1. The first is recommended in the journal article Ranking from unbalanced paired-comparison data by H.A. David published in Volume 74, Issue 2 of Biometrika in 1987.
  2. David also discussed the Kendall-Wei algorithm in his paper, of which Google’s PageRank algorithm is a special case. PageRank is used to rank webpages which are represented as a directed graph based on the concept of network flow, and may also be applied to other directed graphs including our case. The PageRank algorithm contains one tunable parameter, a damping factor which is currently set to the default 0.85.

It was found that both algorithms seemed to emphasize different aspects important to MMA ranking. David’s “Unbalanced Pair Comparison” emphasized a grittier statistics-based approach, highlighting fighters such as Anderson Silva, Rashad Evans, and Jon Fitch. Google’s PageRank seemed to take a more social approach emphasizing fighters with a wide range of quality opponents, like Georges St-Pierre, Matt Hughes, and Forest Griffin. It was very interesting how one algorithm appeared to highlight the “hardcore mma fan” perspective, while the other seemed to be pulled straight from the UFC head office.

It was decided that both would be calculated, scores normalized, and used in combination to generate a consensus ranking similar to consensus rankings generated from human experts. This was inspired by IBM’s Watson which uses a consensus of multiple algorithms to evaluate answers to trivia questions. Two possible improvements are hypothesized but undertested:

  1. Perhaps additional independent ranking algorithms incorporated in this consensus would improve accuracy. The big issue appears to be “independent” algorithms which do not simply restate the work of other algorithms, and of those, finding algorithms which display ranking behavior useful for our application.
  2. Unlike Watson, confidence levels are not used. This would be a useful addition given situations like extreme upsets. A newer beta version of this ranking system determines if highly ranked fighters coincide with centrality metrics in an attempt to implement this, but is not complete at this time of this post.

Results

The ranking system was run on every UFC event from UFC 1 (November 12, 1993) to Fight for the Troops 2 (January 22, 2011). Both algorithms are shown ranked alone for comparison, and their scores were equally weighted to produce the final results.

Lightweight (155lbs)
Overall Rank

PageRank Unbalanced Pair
1. Gray Maynard 1. B.J. Penn 1. Gray Maynard
2. B.J. Penn 2. Gray Maynard 2. George Sotiropolous
3. Frankie Edgar 3. Frankie Edgar 3. Frankie Edgar
4. George Sotiropolous 4. Kenny Florian 4. Jim Miller
5. Jim Miller 5. Joe Lauzon 5. Nik Lentz

First up are the lightweights – and the results aren’t too shabby. No one seems to want to admit it due to his sometimes snooze-inducing style, but Gray Maynard is a beast who is likely to cause B.J. Penn significant issues if they ever fought. Frankie Edgar deserves to be right up there but not number one, and chronically underrated George Sotiropolous and Jim Miller round out the pack.

Welterweight (170lbs)
Overall Rank PageRank Unbalanced Pair
1. Georges St-Pierre 1. Georges St-Pierre 1. Matt Hughes
2. Matt Hughes 2. Matt Hughes 2. Josh Koscheck
3. Josh Koscheck 3. Matt Serra 3. Georges St-Pierre
4. Martin Kampmann 4. Dennis Hallman 4. Martin Kampmann
5. Dennis Hallman 5. Martin Kampmann 5. Rick Story

Georges St-Pierre is the obvious frontrunner at 170. Matt Hughes at number two is a bit more debatable, but a long title reign and consistent quality opposition provide a reasonable rationale. Josh Koscheck is perpetually always the bridesmaid, never the bride at third, and Martin Kampmann and Dennis Hallman round out a somewhat thin division.

Middleweight (185lbs)
Overall Rank PageRank Unbalanced Pair
1. Anderson Silva 1. Anderson Silva 1. Anderson Silva
2. Jon Fitch 2. Jon Fitch 2. Jon Fitch
3. Yushin Okami 3. Vitor Belfort 3. Yushin Okami
4. Michael Bisping 4. Nate Marquardt 4. Michael Bisping
5. Nate Marquardt 5. Yushin Okami 5. Demian Maia

Anderson Silva provides another easy choice for number one at 185lbs. Both Jon Fitch and Yushin Okami deserve their spots with a consistent if slightly dull record. Michael Bisping has slowly been grinding his way up the charts, and Nate Marquardt rounds out the top five.

Light Heavyweight (205lbs)
Overall Rank PageRank Unbalanced Pair
1. Rashad Evans 1. Forrest Griffin 1. Rashad Evans
2. Lyoto Machida 2. Lyoto Machida 2. Jon Jones
3. Forrest Griffin 3. Rashad Evans 3. Ryan Bader
4. Quinton Jackson 4. Quinton Jackson 4. Lyoto Machida
5. Mauricio Rua 5. Mauricio Rua 5. Thiago Silva

Rashad Evans appears to have made a sensible call waiting for his title shot at UFC 128. The hypercompetitive light heavyweight division is always a tough one to call. A split in the consensus between the two algorithms produces a top five that seems to emphasize number of fights in the Octagon, with champion Mauricio “Shogun” Rua a surprising fifth. Too early to call Evans over Rua? Only time will tell.

Heavyweight (265lbs)
Overall Rank PageRank Unbalanced Pair
1. Frank Mir 1. Frank Mir 1. Frank Mir
2. Cain Velasquez 2. Brock Lesnar 2. Junior Dos Santos
3. Junior Dos Santos 3. Cain Velasquez 3. Cain Velasquez
4. Brock Lesnar 4. Antonio Rodrigo Nogueira 4. Cheick Kongo
5. Shane Carwin 5. Shane Carwin 5. Brendan Schaub

I initially disagreed with Frank Mir as number one here – Cain Velasquez seems to be the obvious choice. But the ranking process seems to trust number of fights over new hype, and the rest of the top five is bang on what I would choose. You can’t win them all – or perhaps I’m just being unfair to Frank Mir.

Conclusions

The approach produced excellent rankings from UFC-only data, largely coinciding with established and more complete authorities like FightMatrix. The approach used two ranking algorithms which traversed a directed graph, and produced scores which were normalized and added to produce a final score which was sorted to produce final rankings. One tunable parameter (PageRank damping factor) exists in the model, but was left at the default value of 0.85. Further work will focus on additional ranking algorithms which may be incorporated into the consensus, parametric analysis of the PageRank damping factor, and determining confidence scores.

Directed Graphs and MMA

There is underlying mathematical structure everywhere – it’s just a matter of finding the best way to unfold it from the data. I’ve been working on a project to objectively rank mixed martial arts fighters. It’s not anywhere near done yet, but I’ve collected a fair bit of data. Here’s what nearly 2000 MMA fights in the UFC and Pride FC over the last 15 years look like expressed as a directed graph with a force directed layout.

You can see fights clumped into UFC (top) and Pride (bottom) organizations, with a subset of fighters acting as ambassadors between both.