Ok, here is the methodology used by Mark at the Soccer Rankings app for his analysis of the comparison between each league.
Thanks! I should have read down that page as well. Average of all teams is a defensible option to rank 'em.
He's really good about replying to emails and tweaking things based on user comments, so download his app - pay for the pro version - and then dig further by asking him questions if you really want to get to the bottom of the question: "which league is best?"
Yep - he's quite responsive, and a couple of the tweaks I have suggested over the past few months have made it into the app.
I think it’s hard to compare individual teams when they aren’t regularly playing each other.
Maybe, but that is the question that SR tries to partially solve as objectively as possible, rather than just subjectively. With enough game data, you can fairly compare two teams - even if those two teams themselves never play eachother directly. The farther away in age, geography, league, all factors in to how valid the ratings comparison might be - but overall the predictability appears to be higher than some would want to admit.
On the girls side, I pay attention to the 07 age group since my DD plays in that age group on an 07 ECNL team. Last year’s 07 GA final saw FCV beat NEFC 2-1. We live in the north east, and those two have played some local ECNL teams in the last few months:
SUSA 2 -NEFC 0
Scorpions 1- NEFC 1
PDA Blue 4- FCV 1
Penn Fusion 1- FCV 0
Penn Fusion 1- FCV 2
World Class 2- FCV 1
That’s 1W-1D-4L for the two 07GA finalists, however most of those games were competitive. I think there are 5-6 other 07 GA teams who would have similar competitive results against these same teams, but I’d bet the depth drops off substantially after that.
SUSA 2 -NEFC 0 (11/25)
Scorpions 1- NEFC 1 (11/26)
PDA Blue 4- FCV 1 (11/19)
Penn Fusion 1- FCV 0 (11/20)
Penn Fusion 1- FCV 2 ?
World Class 2- FCV 1 ?
I can't find the last two games looking through any of the team histories, what dates should they show up on? But regardless, the ratings for all 7 of these teams appear to fairly represent their performance, even in just these individual game examples. In only 1 case of the 6, did a lesser rated team beat a higher rated team (Penn Fusion over FCV 11/20). (World Class and FCV are within .02 of each other)
With GotSport down, it was harder to get an official list of GA teams to review, but SR was helpful to provide a decent list. It looks like the ratings follow your intuition here. There are only 4 GA teams nationally with a higher rating (Nationals GA Blue, SC Wave GA, Lou Fusz, and Ukrainian Nationals Philly. Below FCV there are teams like Cincinnati United, and IMG Elite. Looked at another way, GA shows zero teams in the top 10, 3 in the top 20, 12 in the top 50, 26 in the top 100 (see below).
My suspicion is that ranking apps weight current data over older data + if GA teams arent doing tournaments right now their rankings would slowly move down.
While that makes intuitive sense, I have seen the opposite over time for teams that don't play for a month or 3. It's not that ratings go down much over time, it's that they stay the same numerically, or even move up slightly, right up until they drop out entirely to unranked if the team stops playing for 7 months. I believe there are a couple factors that could explain that behavior. At all ages other then the very young, over time there are more teams dropping out and stopping playing entirely rather than new teams created, so as teams drop out of the rating systems, the remainder continue to get bumped up. I also think there is some age-weighting factor that is continually applied to teams throughout the year. It's shown by why 2007's are higher than 2008's are higher than 2009's etc. Yes - any interplay between the ages causes those ratings shifts directly for the two teams immediately involved, but it seems there is an upward nudge for teams that haven't played at all as well. In a specific case, one of our teams hasn't had a scored game since 11/20, and their rating has gone up almost a full point even prior to their first spring tournament in a week or two.
