The usual image of an artificial neural network:
What follows is a paper that I wrote in the Spring of 2001 for an “Introduction to Neural Networks” class that I took as part of my Masters degree. It is mostly a review of someone else’s paper on the subject, except that I wrote the network in Excel and ran it on that year’s football season games. Fun, fun.
This paper describes an approach to the problem of ranking college football teams that determines two competitors for the National Championship. A competitive NN was built, validated, and trained with a bivalent purpose: to rank the teams according to their performance during the regular season and, based on that raking, to predict results. A comparison with other computer rankings will show the good approximation achieved by a NN ranking that predicted 80 % of the results of the end of the season.
The National College Football Championship is defined every January by the two best teams from Division I-A, which is the most important. The regular season is played in 15 consecutive weeks between August and December and 115 teams compete for a place in the final. Since in that very short time teams play at most 12 games, it’s impossible to get a complete all-round. Therefore every team has its own schedule very different from other teams and the wins-losses record is not comparable (even if two teams have, say, a 9-2 record, that is achieved playing different opponents).
The method used so far to rank them, shown below, uses the media of two subjective rankings by Associated Press (poll among coaches) and USA Today (poll among sport writers), the difficulty of the schedule/25, number of lost games and the average position in some reputed computer rankings.
Highlighted on the right are the final scores for Florida State and Miami Florida. The narrowness of the difference brings a controversy and complaints from Miami Florida, relegated from playing the final, especially because during the season, they defeated Florida State. This causes lobbying and politics, that affect the nature of the game every year. A method objective and reliable would end these problems.As an example, the calculation for Oklahoma is: 1+0.44+0+1.86=3.3. The table also shows Oklahoma and Florida State as top ranked and chosen to play the final, the Orange bowl, next January 3rd.
A competitive network suits perfectly for this job. Competitive networks provide a winner but in this case, although matches give only one winner, the network is arranged so that “winner won’t take all”. Shall we do that then the net would only tell me who is number one, but nothing about the rest.
Net consists of 115 nodes. Each node represents a team. The value of the node is the value we’ll use to rank the teams.
Nodes connected are the nodes representing teams who played each other during the season: if two teams haven’t played, then the two nodes representing them won’t be connected. Since every team plays about 10 games per season, then each node will be connected to other 10 nodes among the 115. This gives a low populated network.
The weight of the connection has a bivalent value explained by the figure:
In this net, “B” defeated “A” by 12 points: the connection with “B” is giving “A” a value of –12 for that game, the connection with “A” is giving “B” a value of +12 for that game. Also “C” and “A” played each other (B won), whereas “A” and “C” didn’t play so they’re not connected.
In order for the ranking to be valid, the net has to be fully connected (at least, a minimum spanning tree). Meaning is that if all the teams are not connected, then the result is more than one ranking, each including only teams connected together.
The net could have been connected in only two weeks, with each team playing only two games (also net could be not fully connected even after up to 57 weeks, though in this season teams became connected after 7 weeks).
Once net is validated (by being fully connected, an objective condition) has to be reliable. The more games played per team, the more reliable becomes the net. In this season, teams played between 6 and 12 games, with the average between 10 and 11 (this is a subjective condition: the average of 10 games could be enough but more is better).
Data are the results of the games between Division I-A teams during the first 12 weeks of the regular season 2000/01. All data comes from sport sites on the web and all results available were used (no sampling). Data is highly reliable after several checks performed in a spreadsheet.
Since the data input in the net is very time consuming and the season has just ended, the net was constructed using results up to the 12th week (played past November 11th). In those 12 weeks the 115 teams played 541 games. In this way, 1082 weights were input. As said before: since the net has to know who won the match and by what margin, one positive weight is allocated to the winner and one negative weight to the loser.
Transfer function has the form:
VG=A+B * (if W>0,1,0) + [a+b * (if W>0,1,0)]*VO
The value of a node is the average transferred from connections (the value a team is the average of the value it got from all games played).
In an iterative process, the transfer function is run until convergence is assured. The stopping rule was very simple: if two consecutive rankings are the same, then stop. In this experiment process proved to be highly convergent and 20 iterations were always enough.
Training parameters of the transfer function (and keeping weights fixed) A, B, ( and ( we pursue maximizing the percentage of results predicted. For this experiment, the net is built with 541 results from weeks 1-12, trained using 103 results from weeks 11 and 12 and tested over 73 games on weeks 13 to 15.
Although net could have been trained using all 541 games from weeks 1-12 only last two weeks were used because:
-that would have represented a big amount of work.
-training with last two weeks let us give more importance to last games played for a team: the last games give a better idea of how the team is going to perform next time. Finlayson uses what he calls “time weighting factor” that improves his predictions for international rugby games.
A Home Advantage Factor (which would have given the results as if played in neutral field) could not be implemented because it permanently provoked fluctuations to the iteration trend.
To assure convergence, parameters were chosen so that the value of a game was always between 0 and 100.
Using parameters: A = 30; B = 20; = 0.35; = 0.5 and giving an initial value of 50 for the iteration (bias) to all, an 80% of results is correctly predicted. The complete ranking is shown as an appendix.
As a reference, Dr Ray Finlayson’s world rugby ranking predicts 77% of results, and 84% after the time weighting.
Unfortunately, a comparison with other college football rankings is not possible since they usually only provide the top 16 or 25 teams, but all of them are needed
The following table depicts the correlation among most reputed computer rankings and the one proposed here. Teams are sorted by the BCS (official) ranking. Highlighted cells show how the “bugs” incurred by the ANN ranking have correspondences in the other rankings: Washington ranked 23rd by the ANN and 4th by the BCS, is ranked 10th by Billingsley and 11th by Dunkel; Nebraska, ranked 4th by the ANN and 8th by the BCS, was also 4th according to Sagarin, etc.
Where the headers mean: Billingsley, Dunkel, Massey, New York Times, Rothmans, Sagarin, Scripps, Seattle Times, the ANN discussed here
- Develop a net in which the difference in the ranking gives the expected score difference in a game between that teams (as Finlayson and Sagarin do)
- Rewrite the Home advantage factor so that does not provoke fluctuations in the trend (Sagarin).
- Improve the time weight factor, to give more value to a game played last week as compared to any game played before (Finlayson).
- Develop a parameter to give more value to games played between top ranked teams.
The objectives targeted by this work were partially accomplished. The 80% predicting sounds exciting and overall the rank shows being very similar to other computer rankings.
A deficit is the failure in implementing the H/V factor, which gave instability to the convergence of the iterations.
VG = value transferred by the connection (value the team gets from the game)
VO = value of the node connected (value of the opponent team) (since this is an iterative process VO is VG for the opponent in the previous iteration)
W = weight of the connection (score difference of the game)
A + B = intercept for the “win” curve
A = intercept for the “loss” curve
a + b = slope for the “win” curve
a = slope for the “loss” curve
Fausett, L. (1994). Fundamentals of Neural Networks. First Edition. Upper Saddle River. Simon & Schuster.
Wilson, R. (1995). “Ranking College Football Teams: A Neural Network Approach”. Interfaces 25: 4 July-August 1995 (pp. 44-59).
On the Internet:
http://www-cs-students.stanford.edu/~rsf/finlayson.html; http://sports.yahoo.com; http://www.ncaa.org; http://collegefootball.net; http://www.planet-rugby.com/rankings/explained.html/PR/ENG; http://www.live.com/wrr/
Other College Football Rankings:
http://www.usatoday.com/sports/sagarin.htm; http://www.dunkelindex.com; http://www.mratings.com/rate/cf-m.htm; http://www.cae.wisc.edu/~dwilson/rsfc/rate/rothman.txt; http://espn.go.com/abcsports/bcs; http://college.espn.go.com/ncf/rankings
Final ANN Ranking:
(actually 115 teams were ranked)
That was the paper, over time I decided the network actually more closely resembles a Markov Chain than a typical neural network (feedforward et al) where the “states” are the teams and the probabilities are games’ scores.
Years later, in 2007, I joined the crowd of computer rankings enthusiasts and created SporTheory, my very own page where I publish the rankings resulting from these techniques and rank college teams in Football and Basketball. I used to also rank Baseball, Softball and Hockey teams.
If you are curious about neural networks, check the textbook we used: