# Youth Soccer Rankings: Disregarding >5 Point Blowouts?



## Woodwork (May 10, 2021)

Prefacing this by saying no one should care much about these rankings or other youth rankings, for a wide variety of reasons.  I would recommend looking at actual head to head results in league play and the actual team play and coaching styles before anything else. 

Hypothesis: YSR disregards blowouts over 5 points, or at least limits them to 5 points.

Two examples:  (1) https://youthsoccerrankings.us/team.html?teamId=3500229 (Not to rag on this team at all - they likely had a slow start coming out of the pandemic and have steadily improved and solidified.  Kudos to them.).  And (2) https://youthsoccerrankings.us/team.html?teamId=3478498.

In case (1): If you take the strict average of the goal differentials (e.g., for a 15 goal loss to a 30.83 team, a point score of 15.83), 214.12/9 = 23.79.  However, if you limit the losses to 5 point differentials,  you average 25.68.  That is within .02 of the YSR score.

In case (2):  If you take the strict average of their differentials (which goes back about 18 months, a seeming cutoff date for YSR), you get 410.38/11 = 37.30.  But by limiting victories to no greater than 5 each, you get 396.38/11 = 36.03, which is within .01 of the YSR score.

The end result, it seems to me, is that YSR doesn't work well in ranking teams that play way out of their league or sandbag, because it cannot derive enough information from blowouts.  A team that regularly wins/loses by 5 points is effectively the same as one that wins/loses by 10.


----------



## PracticeWYpreach (May 11, 2021)

Woodwork said:


> Prefacing this by saying no one should care much about these rankings or other youth rankings, for a wide variety of reasons.  I would recommend looking at actual head to head results in league play and the actual team play and coaching styles before anything else.
> 
> Hypothesis: YSR disregards blowouts over 5 points, or at least limits them to 5 points.
> 
> ...


To that point, a flight 1 team that gets blown out in all their games is usually ranked higher than a flight 2 team that wins all their games by a few goals. It probably needs those tournaments where teams who don't normally play each other do.


----------



## futboldad1 (May 11, 2021)

YSR is great for the Youngers ages but once all our kids are spread across different leagues it is very inaccurate.........a team in ECRL or flight 1 who blows everyone out is ranked above that SAME club's ECNL team which makes no sense.........YSR rewards blowouts too much....winning close harder games are not recognized as much......

So I think I'm disagreeing with the OP but it is early so I might have misread his words......but I do agree with him that H2H is much better than any rankings


----------



## rainbow_unicorn (May 11, 2021)

futboldad1 said:


> YSR is great for the Youngers ages but once all our kids are spread across different leagues it is very inaccurate.........a team in ECRL or flight 1 who blows everyone out is ranked above that SAME club's ECNL team which makes no sense.........YSR rewards blowouts too much....winning close harder games are not recognized as much......
> 
> So I think I'm disagreeing with the OP but it is early so I might have misread his words......but I do agree with him that H2H is much better than any rankings


YSR does not "reward" blowouts too much.  It's a pretty simple algorithm on the surface which is taking into account as much data as it can and using the game results/goal differentials to generate an expected future game differential between teams.  Imposing a goal limit will actually make YSR less accurate.  What should be happening is that teams (or players) should be playing in appropriate leagues where they are not blowing out (or getting crushed) on a consistent basis.


----------



## futboldad1 (May 11, 2021)

Then how do you account for the same clubs RL teams being ranked higher than their ECNL teams? And teams who are in ECNL that used to thrash teams that are stuck in flight 1 now being ranked below them as the F1 teams win 5-0 every week? That makes no sense at all.......


----------



## dad4 (May 11, 2021)

You more want to disregard mismatches than blowouts.

If you expect an even game, but it is 8-0, don’t disregard it.  You just got information.

On the other hand, if YSR predicts a 4 point gap, and it is 8-0, then you got no information.

One solution is just toss any game where the winning team was expected to win by 3 or more.  It means you disregard the easier games for a team that is hanging out in a low league.  

That’s what we all do anyway.  If a good team goes trophy hunting in a lower tier tournament, the rest of us don’t think better of them for it.  We ignore the mismatches and ask how they do against real opponents.


----------



## Desert Hound (May 11, 2021)

futboldad1 said:


> Then how do you account for the same clubs RL teams being ranked higher than their ECNL teams? And teams who are in ECNL that used to thrash teams that are stuck in flight 1 now being ranked below them as the F1 teams win 5-0 every week? That makes no sense at all......


YSR does fine at younger ages. Once teams move off into various leagues (ECNL, ecRL, etc), the rankings start to move all over the place.

Long story short?

At older ages and when teams in different leagues the YRS reliability falls somewhat apart.


----------



## Woodwork (May 11, 2021)

Desert Hound said:


> YSR does fine at younger ages. Once teams move off into various leagues (ECNL, ecRL, etc), the rankings start to move all over the place.
> 
> Long story short?
> 
> At older ages and when teams in different leagues the YRS reliability falls somewhat apart.


YSR depends on head to head competition within a few degrees of Kevin Bacon to compare teams that have never played each other.  That does cause some variation but I think it is relatively minor compared to the apparent 5 goal differential limit.  

First, that limit causes YSR to have extremely poor predictive value for teams that routinely win/lose by more than 5 goals.  Two contrasting examples: (1) https://youthsoccerrankings.us/team.html?teamId=3472201 (Team that is routinely within 5 goals has fairly good predictive outcomes) and (2) https://youthsoccerrankings.us/team.html?teamId=3458985 ("Outlier team" routinely over 5 goal differential has YSR surprised every week, although the scores are consistent with its real average score of 26.7 rather than 29.66).

Second, to the extent YSR depends on a few instances of teams playing outside their league and/or level to draw conclusions about teams across the board, the outlier team will have a disproportionate impact because it is more likely to have head to head games with lower ranked teams when tournaments come around.  So an outlier team becomes the measuring stick.

Assuming the purpose of the 5 goal limit is to prevent anomalies from impacting the rankings, I think the 5 goal limit is arbitrary and too low in the context of youth sports.  Rather, it would make more sense for YSR to trust its own algorithm, disregarding one result outside of a standard deviation for each team (once there is a pattern, then it would be reincorporated as within the standard deviation).


----------



## Woodwork (May 11, 2021)

dad4 said:


> You more want to disregard mismatches than blowouts.
> 
> If you expect an even game, but it is 8-0, don’t disregard it.  You just got information.
> 
> ...


Or perhaps point differentials as a whole are over-rated for predictive value.  Teams behave differently once they are ahead and behind.  Some teams park the bus, some pile it on, some play subs and rest starters.  Some tournaments have benefits for goal differential, some don't.  Maybe a better system would ignore GD and base the rankings entirely on head to head W/L (still applying degrees of kevin bacon).


----------



## dad4 (May 11, 2021)

Woodwork said:


> Or perhaps point differentials as a whole are over-rated for predictive value.  Teams behave differently once they are ahead and behind.  Some teams park the bus, some pile it on, some play subs and rest starters.  Some tournaments have benefits for goal differential, some don't.  Maybe a better system would ignore GD and base the rankings entirely on head to head W/L.


WL- There is good information in the difference between a 3-4 game and a 4-0.  Going to simple W/L loses some accuracy.  It also makes it even harder to shift information across regions.

Outliers-. I am not sure what an outlier is in this context.  Do you mean a team that plays mismatches, or a team that crosses over to compete in a different pool?  You used "outlier" for both.

Mismatch seekers are useless as data sources.  

Crossover teams that enter a new competitive pool to find close games are incredibly good information sources.  The pandemic caused a temporary shortage of these, which hurts.


----------



## paytoplay (May 11, 2021)

The concept of “team” is problematic. In normal years, there’s some consistency within league play, but rosters for tournament play can change the team drastically. Maybe rankings should be a measure of the former, with an alternate ranking including tournaments. And how about weighting the different leagues? ECNL 10, SCDSL 6, Rec 1, etc


----------



## Woodwork (May 11, 2021)

dad4 said:


> WL- There is good information in the difference between a 3-4 game and a 4-0.  Going to simple W/L loses some accuracy.  It also makes it even harder to shift information across regions.
> 
> Outliers-. I am not sure what an outlier is in this context.  Do you mean a team that plays mismatches, or a team that crosses over to compete in a different pool?  You used "outlier" for both.
> 
> ...


I was using "outlier team" to describe a team whose results are not accurately predicted by YSR due to the 5 point limit.  When I say an outlier team playing in a different pool, I mean the poorly-predicted team playing in a different pool.


----------



## MSK357 (May 11, 2021)

Woodwork said:


> I was using "outlier team" to describe a team whose results are not accurately predicted by YSR due to the 5 point limit.  When I say an outlier team playing in a different pool, I mean the poorly-predicted team playing in a different pool.


I would like to think when a team is up by 5, they start to take their foot off the pedal.  At least moving players around to play other positions.  whether its 5-0 or 10-0 its clearly one sided, and shouldn't really matter at that point.  I'm sure many teams down by that much are demoralized anyway and aren't even playing to their best capability.


----------



## Woodwork (May 11, 2021)

paytoplay said:


> The concept of “team” is problematic. In normal years, there’s some consistency within league play, but rosters for tournament play can change the team drastically. Maybe rankings should be a measure of the former, with an alternate ranking including tournaments. And how about weighting the different leagues? ECNL 10, SCDSL 6, Rec 1, etc


Tournaments are basically the only way you can have head to head between different leagues, though.  E.g., concacaf club tournament pits MLS vs Liga MX and other leagues.  It becomes a comparison point for how leagues compare.  Otherwise you have endless subjective speculation.  I think drastic changes in rosters at tournaments happens but not enough to worry about as a whole.  It isn't a systematic problem like the 5-goal differential limit.


----------



## dad4 (May 11, 2021)

paytoplay said:


> The concept of “team” is problematic. In normal years, there’s some consistency within league play, but rosters for tournament play can change the team drastically. Maybe rankings should be a measure of the former, with an alternate ranking including tournaments. And how about weighting the different leagues? ECNL 10, SCDSL 6, Rec 1, etc


As long as there are politics in league admissions, weighting leagues hurts accuracy.  

Tournament rosters are funky.  But tournaments are 100% of your long distance connectivity for lower tier teams.  If you throw them out, you get massive drift between different states.


----------



## Woodwork (May 11, 2021)

MSK357 said:


> I would like to think when a team is up by 5, they start to take their foot off the pedal.  At least moving players around to play other positions.  whether its 5-0 or 10-0 its clearly one sided, and shouldn't really matter at that point.  I'm sure many teams down by that much are demoralized anyway and aren't even playing to their best capability.


In my experience, teams winning at the older ages rarely take their foot off the pedal.  Think USA vs. Thailand.  Perhaps each goal can become worth less in terms of predictive value, gradually.  But there is a difference between a team that loses 12-0 and one that loses to the same opponent 5-0.  The difference isn't 0.


----------



## MSK357 (May 11, 2021)

Woodwork said:


> In my experience, teams winning at the older ages rarely take their foot off the pedal.  Think USA vs. Thailand.  Perhaps each goal can become worth less in terms of predictive value, gradually.  But there is a difference between a team that loses 12-0 and one that loses to the same opponent 5-0.  The difference isn't 0.


I think other factors can be at play also.  If you have 2 games that day or game the next day, save your starters when you're up 5-0.  USA vs Thailand is a little different, they probably could have started the bench and got pretty close to that.  And if I remember correctly they did save some players for the next game.  So in that sense, that 12-0 could have been worse.


----------



## NorCalUSN (May 11, 2021)

Bringing a different perspective. (BTW I always hated the guy that started a sentence "When I was in..." but here it goes.) You may have seen my post about my daughters playing in Japan.  We were at one of those games where my daughter's team was just dominating, it was a like a 20-minute half but the score was already 5-0.  I thought for sure the coach would ease off the gas, change playing style or something.  But the 2nd half started off like the first soon it was 8-0 and I was feeling pretty ashamed about running up the score on a U10 boys game.  I asked my Japanese wife why is he not taking it easy? or something.  She asked, found out it was considered more "insulting" to take it easy on the other team then to keep going at regular speed.  I didn't necessarily agree but it did open up my perceptions to seeing it differently. Culture or not culture I don't think I would have done the same way but there has to be some balance.


----------



## focomoso (May 12, 2021)

futboldad1 said:


> .........a team in ECRL or flight 1 who blows everyone out is ranked above that SAME club's ECNL team which makes no sense.........


I agree there are deficiencies in YSR's model, but this isn't necessarily incorrect. Not all clubs treat their ECNL/ECRL teams as true A/B teams. Often they're made up of existing teams that come in as a unit, so it's not crazy that an RL team might be better than the NL. (And this is especially true for boys as RL is so new.)


----------



## RedCard (May 12, 2021)

One thing that's wrong with the YSRs right now is that they are not including any of the games from the year's SCDSL league season, yet all the games from this year's ECNL league and boys EA league are added.


----------



## MARsSPEED (May 13, 2021)

Always love a good YSR debate. Part of the problem with older ages is that when the new ECNL/ECRL/GA teams are created, they lose the ranking of the team that built it's way up. Errors should be reported on the site to combine or dislocate new and old team data. Often this does not happen or the wrong team are combined. YSR does have it's own system to catch mistakes but it's rare.

As for blowouts, it is what it is. IMO, that's the biggest issue when two teams are split by 3 or 4 goals on the index. Many more times the spread is larger in the actual games.


----------



## crush (May 13, 2021)

MARsSPEED said:


> Always love a good YSR debate. Part of the problem with older ages is that when the new ECNL/ECRL/GA teams are created, they lose the ranking of the team that built it's way up. Errors should be reported on the site to combine or dislocate new and old team data. Often this does not happen or the wrong team are combined. YSR does have it's own system to catch mistakes but it's rare.
> 
> As for blowouts, it is what it is. IMO, that's the biggest issue when two teams are split by 3 or 4 goals on the index. Many more times the spread is larger in the actual games.


I think your disclaimer will need to be changed soon Mars.  I have a new one for you if you would like it


----------



## futboldad1 (May 13, 2021)

focomoso said:


> I agree there are deficiencies in YSR's model, but this isn't necessarily incorrect. Not all clubs treat their ECNL/ECRL teams as true A/B teams. Often they're made up of existing teams that come in as a unit, so it's not crazy that an RL team might be better than the NL. (And this is especially true for boys as RL is so new.)


May be true for boys...... would be VERY rare for girls...... a club's best teams are all ECNL


----------



## MARsSPEED (May 14, 2021)

crush said:


> I think your disclaimer will need to be changed soon Mars.  I have a new one for you if you would like it


This should be good. No politics here so send me a PM, haha.


----------



## crush (May 14, 2021)

MARsSPEED said:


> This should be good.* No politics* here so send me a PM, haha.


Never, ever bro.  How hard is it to get gas so you can still drive your dd to soccer practice?  I will PM my disclaimer for you.


----------



## Woodwork (May 14, 2021)

MARsSPEED said:


> Always love a good YSR debate. Part of the problem with older ages is that when the new ECNL/ECRL/GA teams are created, they lose the ranking of the team that built it's way up. Errors should be reported on the site to combine or dislocate new and old team data. Often this does not happen or the wrong team are combined. YSR does have it's own system to catch mistakes but it's rare.
> 
> As for blowouts, it is what it is. IMO, that's the biggest issue when two teams are split by 3 or 4 goals on the index. Many more times the spread is larger in the actual games.


I would actually suggest YSR limit the data set to 12 months.  The purpose of the rankings is to see current standings, not historical.  The older data also may poison the whole ranking if that team does a lot of cross-league play.

YSR's inability to predict blowouts is kind of my point.  The inaccurate 3-4 point predicted differential appears to be a symptom of capping blowouts over 5 goals - even if it is persistent.

And please don't reply Mars if you still have that headache of a signature.  I like the sentiment but I may have to block it to be able to read anything you post in.


----------



## dad4 (May 14, 2021)

Woodwork said:


> I would actually suggest YSR limit the data set to 12 months.  The purpose of the rankings is to see current standings, not historical.  The older data also may poison the whole ranking if that team does a lot of cross-league play.
> 
> YSR's inability to predict blowouts is kind of my point.  The inaccurate 3-4 point predicted differential appears to be a symptom of capping blowouts over 5 goals - even if it is persistent.
> 
> And please don't reply Mars if you still have that headache of a signature.  I like the sentiment but I may have to block it to be able to read anything you post in.


How do you propose to include blowouts?

Suppose the Pink Flamingos rack up another 12-0 win over a team with a score of 19.  Do you want to put a 31 into the average?

I’d say the Flamingos are in for a rough day if they ever have to play a team whose 31 came from close games.


----------



## Woodwork (May 14, 2021)

dad4 said:


> How do you propose to include blowouts?
> 
> Suppose the Pink Flamingos rack up another 12-0 win over a team with a score of 19.  Do you want to put a 31 into the average?
> 
> I’d say the Flamingos are in for a rough day if they ever have to play a team whose 31 came from close games.


Yes, average it in, and deal with outliers differently.  I mentioned previously dropping any game which is way outside of the predicted range (say, 2 standard deviations).  The problem comes when you presume blowouts are outliers when, between certain teams, they are the norm.

Also, I think you would probably see a larger spread in the point system for YSR.  The top team would be more like 40 rather than 38.  The bottom team in an age group would be more like 20.  Right now, because of the 5 point cap, I suspect, The spread is more like 10 points rather than 20.

Sparkling Pink Flamingos was literally one of my daughter's teams.  How dare you.


----------



## dad4 (May 14, 2021)

Woodwork said:


> Yes, average it in, and deal with outliers differently.  I mentioned previously dropping any game which is way outside of the predicted range (say, 2 standard deviations).  The problem comes when you presume blowouts are outliers when, between certain teams, they are the norm.
> 
> Also, I think you would probably see a larger spread in the point system for YSR.  The top team would be more like 40 rather than 38.  The bottom team in an age group would be more like 20.  Right now, because of the 5 point cap, I suspect, The spread is more like 10 points rather than 20.
> 
> Sparkling Pink Flamingos was literally one of my daughter's teams.  How dare you.


Averages don’t help when you have a team as dominant as the Flamingos.  

How do you detect an outlier when the norm is blowouts?  The Flamingoes won by an average of 8 points last season.  The outlier was the 4-3 loss against Purple Unicorns.  If we throw out outliers, we should exclude the Unicorns game.


----------



## Woodwork (May 14, 2021)

dad4 said:


> Averages don’t help when you have a team as dominant as the Flamingos.
> 
> How do you detect an outlier when the norm is blowouts?  The Flamingoes won by an average of 8 points last season.  The outlier was the 4-3 loss against Purple Unicorns.  If we throw out outliers, we should exclude the Unicorns game.


You are dangerously close to my DD's other AYSO team.  Are you my wife?  If so, you are right and I am wrong.

You don't detect outliers by just raw score differentials.  That is what I am advocating against.  An outlier is would be determined by how far off from the predicted outcome the game is - the predicted outcome based on YSR's total aggregate data and algorithm.  In your scenario, the 4-3 loss could be an outlier if YSR has Purple Unicorns at 24 and Pink Flamingos at 28 (it expected Flamingos to win by 4).

If it happens once, outlier.  If it happens twice, you move the bell curve over and include both data points.


----------



## dad4 (May 14, 2021)

Woodwork said:


> You are dangerously close to my DD's other AYSO team.  Are you my wife?  If so, you are right and I am wrong.
> 
> You don't detect outliers by just raw score differentials.  That is what I am advocating against.  An outlier is would be determined by how far off from the predicted outcome the game is - the predicted outcome based on YSR's total aggregate data and algorithm.  In your scenario, the 4-3 loss could be an outlier if YSR has Purple Unicorns at 24 and Pink Flamingos at 28 (it expected Flamingos to win by 4).


if you have 8 blowouts and one 4-3 game, the YSR algorithm will gradually keep raising the team rating.  Eventually, it has them at 31- supported by nothing but the blowouts.  (We threw out the 4-3 game, because it didn’t match predictions.)

That’s a problem for the Flamingos.  I don’t think they’re quite ready for Supercopa or Dallas Invitational just yet.  But their rating says they are.


----------



## Woodwork (May 14, 2021)

dad4 said:


> if you have 8 blowouts and one 4-3 game, the YSR algorithm will gradually keep raising the team rating.  Eventually, it has them at 31- supported by nothing but the blowouts.  (We threw out the 4-3 game, because it didn’t match predictions.)
> 
> That’s a problem for the Flamingos.  I don’t think they’re quite ready for Supercopa or Dallas Invitational just yet.  But their rating says they are.


If they decide their first time playing a high level team must be in Dallas, so be it.  Once they go, those teams will beat them by 9 points each.  The higher level teams will have 40 points.  Flamingos are 31.  The other teams in Flamingo's league are 23.


----------



## dad4 (May 14, 2021)

Woodwork said:


> If they decide their first time playing a high level team must be in Dallas, so be it.  Once they go, those teams will beat them by 9 points each.  The higher level teams will have 40 points.  Flamingos are 31.  The other teams in Flamingo's league are 23.


Not quite. When they went to Dallas, they were scheduled against teams ranked 30-32.  It was predicted to be close, but they lost every game 8-0.  

That means every game in Dallas was an outlier.  They didn’t match predictions, so we threw those games out.  Flamingos are still listed at 31. (oops)

See the problem?  Flamingos are a fake 31.  They’re a 26, but all offense.  They can beat any 23 by huge margins.  Against a 25-27, they have a close game.  Against a real 31, they get crushed.  

The algorithm need to converge anyway.  Ideally, it converges to something near 26, because those are the good games.


----------



## GT45 (May 14, 2021)

Woodwork said:


> I would actually suggest YSR limit the data set to 12 months.  The purpose of the rankings is to see current standings, not historical.  The older data also may poison the whole ranking if that team does a lot of cross-league play.
> 
> YSR's inability to predict blowouts is kind of my point.  The inaccurate 3-4 point predicted differential appears to be a symptom of capping blowouts over 5 goals - even if it is persistent.
> 
> And please don't reply Mars if you still have that headache of a signature.  I like the sentiment but I may have to block it to be able to read anything you post in.


Agree. Plus team rosters can change significantly a year later. Data from a team that has a nearly completely different roster is not accurate.


----------



## Woodwork (May 15, 2021)

dad4 said:


> Not quite. When they went to Dallas, they were scheduled against teams ranked 30-32.  It was predicted to be close, but they lost every game 8-0.
> 
> That means every game in Dallas was an outlier.  They didn’t match predictions, so we threw those games out.  Flamingos are still listed at 31. (oops)
> 
> ...


I don't know if this is what you are getting at, but I would agree that goals are not a stable currency.  Let me put it this way:

D1 is +3 to F1
F1 is +8 to F2

What is D1 to F2's differential then?  Logically, you would think +11.  But, in head-to-head, because goals come easier when D1 plays against F2, it is probably more like +13 or +14.  So, yes, D1 could theoretically rank higher by playing F2 all year rather than F1.  But I don't think this justifies a +5 cap on GD.  Goals after the first 5 in blowout games are cheaper, but not worthless.

Maybe the only way to get a reliable GD ranking is for all the teams to each play the bottom team once.  Whichever team scores the most goals is the best one.  Presumably SD Surf ECNL is a plus 30.


----------



## focomoso (May 15, 2021)

All good points here. The one thing I'd say in favor of a cap is that once a team is up by 4 or 5 goals, some coaches will take their foot off the pedal and have the players work on specific things (usually possession) over scoring because 1) they can and 2) it doesn't do anyone any good to roll over a lessor opponent. Teams that don't do this will end up with higher YSR ratings than teams at the same level that do do this.


----------



## notintheface (May 15, 2021)

Waiting to see how long a thread goes about an algorithm judging a bunch of 9 year olds. Do we think YSR takes "stayed up past bedtime to watch a new episode of their favorite Disney+ show" into consideration? Come on folks.


----------



## dad4 (May 15, 2021)

notintheface said:


> Waiting to see how long a thread goes about an algorithm judging a bunch of 9 year olds. Do we think YSR takes "stayed up past bedtime to watch a new episode of their favorite Disney+ show" into consideration? Come on folks.


You mean I will be any less ridiculous when my kid turns 15?

Don't bet on it.

Go Flamingos!!!!!!


----------



## myself (May 21, 2021)

notintheface said:


> Waiting to see how long a thread goes about an algorithm judging a bunch of 9 year olds. Do we think YSR takes "stayed up past bedtime to watch a new episode of their favorite Disney+ show" into consideration? Come on folks.


Considering that the process of gathering and manipulating the data is probably 95% automated, no.


----------

