And here was what the predictions were asking the team to perform
Youth Soccer Rankings ?
- Thread starter dawson
- Start date
What I find odd is that Solar RL remains pretty much unaffected by their 4 losses this league season and being towards the bottom of their league.. I guess 41 goals in 13 matches outweighs the 4 losses and being in 5th place? In contrast, Slammers has 8 wins out of their 8 matches and 30 goals for the season and only 3 goals conceded
Attachments
RandomSoccerFan
GOLD
I'm happy that we're looking at the same data, but I'm not sure why we are seeing it so differently. Those predictions are what is necessary for a team to maintain their current rating according to their peers. Go above it, rating moves up. Go below it, rating moves down. All the ratings of all teams that play each other are interrelated. Over time - the average rating of an age group as a whole moves up in tandem (compare U11 to U12 to U13 etc.)
You're wildly exaggerating the requirements for your team to maintain the rating. For the rest of the league games all year, there is one game where it is expected to be blowout (7-0 against Heat). Only two games require a 4 goal difference. Everything else is 3 or lower. I just reloaded each team, and here are those predictions again:
Slammers FC HB Koge RL 2-0
Pateadores RL 3-0
LAFC So Cal RL 2-0
Eagles SC RL 2-0
LA Breakers FC RL 4-0
Eagles SC RL 2-0
Phoenix Rising RL 3-0
San Diego Surf RL 3-0
Heat FC RL 7-0
Sporting California USA RL 3-0
So Cal Blues ECNL RL 1-0
LAFC So Cal RL 2-0
Sporting California USA RL 3-0
Utah Royals FC-AZ RL 4-0
Beach FC RL 1-0
Legends FC RL 1-0
If you go out of league to play other teams, and you are wildly better than them - the expectation is that you will beat them soundly. If you go out of league to play other teams, and they are in fact rated higher than you - if/when you beat them your own rating will go up and theirs will go down.
If you think there is a special "if team name = Solar then do something hinky with ratings" is baked into the app, I'm not sure what to tell you. There are hundreds of thousands of teams keeping track of literally millions of games, all fed into the same program - and the data being shown is what it comes up with.
You're wildly exaggerating the requirements for your team to maintain the rating. For the rest of the league games all year, there is one game where it is expected to be blowout (7-0 against Heat). Only two games require a 4 goal difference. Everything else is 3 or lower. I just reloaded each team, and here are those predictions again:
Slammers FC HB Koge RL 2-0
Pateadores RL 3-0
LAFC So Cal RL 2-0
Eagles SC RL 2-0
LA Breakers FC RL 4-0
Eagles SC RL 2-0
Phoenix Rising RL 3-0
San Diego Surf RL 3-0
Heat FC RL 7-0
Sporting California USA RL 3-0
So Cal Blues ECNL RL 1-0
LAFC So Cal RL 2-0
Sporting California USA RL 3-0
Utah Royals FC-AZ RL 4-0
Beach FC RL 1-0
Legends FC RL 1-0
If you go out of league to play other teams, and you are wildly better than them - the expectation is that you will beat them soundly. If you go out of league to play other teams, and they are in fact rated higher than you - if/when you beat them your own rating will go up and theirs will go down.
If you think there is a special "if team name = Solar then do something hinky with ratings" is baked into the app, I'm not sure what to tell you. There are hundreds of thousands of teams keeping track of literally millions of games, all fed into the same program - and the data being shown is what it comes up with.
RandomSoccerFan
GOLD
That standings page of ECRL Texas is a bit broken as well. It looks like someone decided to sort it by PPG instead of PTS. If you sort the teams by points instead, it looks like below: (I added the current ratings for each team as well)
The position in the league closely tracks the rating, as one would expect. Solar shows a bit higher, so does FC Dallas RL. Any number of reasons for this - but the only one that matters is that for the opponents they played, they achieved a certain goal differential (pos or neg), and that factors in to their score every time. One indicator is how many more goals those teams have scored (41 & 39). And their goal differential is much better than anyone else in the league (+24 and +28).
The position in the league closely tracks the rating, as one would expect. Solar shows a bit higher, so does FC Dallas RL. Any number of reasons for this - but the only one that matters is that for the opponents they played, they achieved a certain goal differential (pos or neg), and that factors in to their score every time. One indicator is how many more goals those teams have scored (41 & 39). And their goal differential is much better than anyone else in the league (+24 and +28).
RandomSoccerFan
GOLD
As of today, it looks like the average for the Socal RL division is roughly 1 goal lower than the Texas RL division. Everything else being equal, it will be more challenging for a Socal team to match a rating with a higher rated Texas team, if they play equivalently (whatever equivalently means), and if that's an expectation or goal.
An algorithm based on averaging goal differential will do exactly that. A 3-4 loss and a 10-0 win earn you a higher ranking than a pair of 3-0 wins.
you also have to include the Sonoran division as Mojave also plays each team cross bracket during the season. That being said, the two highest rated teams Slammers RL and SoCal Blues are both around the same point total.
I guess in the end it’s more high stakes for these two teams to play lower teams should they not beat their predictions. I think those goal differentials are more achievable in 9v9 but not so much in 11v11 once these teams get their bearings on playing a big field.
I think it’s also kind of always been my gripe with the website and now the program that teams can play in lower leagues and hide in bottom bracket tournaments and it inflates their rankings because of goal differential.
I’m very familiar with how the ratings work, one higher rated team has a predictive value and another has their predictive value and that difference is goal differential. But a 2010 RL team rated 35 is predicted to lose to the top 2014 team. I can’t imagine a group of 8 year olds beating on 12 year olds. I think there should be some type of cap for rating (wins, losses, goal scored, goals against, age, and difficulty of bracket).
RandomSoccerFan
GOLD
Almost - there needs to be a caveat that the opponents are identical in this setup. If the games are against separate teams, it's entirely feasible that a 3-4 loss against a stellar opponent will help a rating more than a 3-0 win against a cellar-dwelling team.
RandomSoccerFan
GOLD
First - let me say congratulations on what looks to have been a fantastic start to this season. The team is clearly playing very well, and has become the standout team in their league. All of that should be a much more important realization than any of this silly rating discussion.
Maybe it would help to think about it slightly differently. The current rating of the team (any team) *is already* a reflection of how they've played in the past. Their exact performances in the past, the goal differentials against teams of certain ratings, all of the previous team results - this is reflected in their current rating. To keep their current rating - they essentially have to do nothing more, and nothing less, than they've already done. Looking at the current predictions in the future and thinking "that will never happen, they are too onerous" is too negative - those predictions are based on what has already happened looking backwards.
Now if the goal is to improve the current rating, and take a team for example from 41 to 43 within a certain time period - the math is pretty simple. For those same expectations, they need to overperform by about 2 goals every time (on average). If they can do that for a month or two - that's exactly what the rating would show. And if they have injuries or other challenges and they underperform by 1 goal (on average) for a month or two, their rating should go from a 41 to a 40. Whether it's done by ensuring the blowouts come in exactly on target, or a close game becomes a little less than a close game, the benefits (or penalties) would be the same. The interesting part is by the end of that journey, the predictions will start to take into account the new performance, and then the team will need to perform at those new expectations going forward, to keep that improved rating.
This is a bit inflammatory, and no matter how many times these insinuations are stated as fact - I haven't been able to find any actual examples that show this occurring. I looked in Southwest, Texas, Florida, NY, IL, and I couldn't come up with a representative example of a highly ranked team (Top 25 nationally) that had a limited game history that was just against weak opponents in their own league where they had an inordinate amount of blowouts. There may be ones out there, and I'm hoping that someone can point me in the right direction. What teams or leagues are you referring to? But in this thread - Solar (ECRL and ECNL) are both counter examples that show this not to be true. There is a ton of game history outside their own leagues that ensures that the ratings assigned take into account all sorts of competition, which both help and hurt from time to time depending on the opponent.
A cynic might say that Slammers RL are guilty of what you're accusing others of doing. Undefeated in league, 3 goals against them all year, yet the team isn't chasing enough worthwhile opponents to actually test their mettle in tournaments where they have a reasonable chance to lose. How many of these talented girls should be on an ECNL team instead once reaching 2009G?
There is a well-respected club near me that runs their youngers 1 year up all the way through U12 for their top team, and they are quite successful. In some tournaments they are entered 2 years up. Turns out that a very good 2013 team can in fact embarrass an average (or weaker) 2011 team. Watching a bunch of tiny sprites fly around the field against a bunch of larger but often slower and less talented players, can be fun to watch. Once kids get much older than this though, it starts to become a safety concern and the size/strength differences can't be ignored forever. I'm with you, 8 year olds playing 12 year olds would be a bit of a stretch. And validating whether a "35" at 2014 is exactly the same as a "35" at 2010, isn't something that can probably be confirmed in the real world. The cross-play from age group to age group to age group is just too far a chain of results to ensure that there isn't drift between the rating scales of the two teams. So it's a bit of a moot point. We don't know whether the 35's mean the same in that comparison, and while we can debate it - it can never be settled authoritatively.
You, and I, and probably most people other than Mark and any of his team members/helpers, have almost no idea about what goes in to the algorithm(s) - other than it's related to goals scored in prior games. The types of tweaking, caps, weightings, timings, and all of the other manipulations that can be done to past data to see if it better predicts future data is exactly what Mark has been doing for years. And continues to do in this app. What would you do if you had a system with millions of games, and had the chance to run countless models on the data to see how well different factors and operations can be applied to better predict game results (that have already happened)? All of the hypotheses that one could come up with ("weight last months result 1/3 less than this months, or weight it 1/2 less." "Drop results older than 6 months to .3 weighting vs. .2 weighting" "Refactor goal differential so 5 goal differences are only slightly higher than 4 goal differences to minimize blowout effect") are the types of things that you get to optimize for when you have the data. And all of that was massaged over time at YSR, but now is even more possible with the app. Just look at the predictions - there is now an estimate for what the score is likely to be. There are percentages given for chance of win / tie / loss. These don't come from a random number generator - they are provided as the back end of a ton of past data on team performance.
Is any of it ever going to be smart enough to predict the future with no variation - of course not. The real world doesn't work like that, for reasons that are obvious to all. But the griping about a simplistic algorithm or bad results feels off-base - the griping instead should be if the predictions turn out to be off or unusable on a regular basis. If someone has a better way of looking at team data on their own and predicting future performance more effectively than this - prove it to yourself! Use the data you have, look at the upcoming games, and document your predictions. If it turns out that the app isn't any better than what's possible without - save the $10 and don't give it a second thought.
Maybe it would help to think about it slightly differently. The current rating of the team (any team) *is already* a reflection of how they've played in the past. Their exact performances in the past, the goal differentials against teams of certain ratings, all of the previous team results - this is reflected in their current rating. To keep their current rating - they essentially have to do nothing more, and nothing less, than they've already done. Looking at the current predictions in the future and thinking "that will never happen, they are too onerous" is too negative - those predictions are based on what has already happened looking backwards.
Now if the goal is to improve the current rating, and take a team for example from 41 to 43 within a certain time period - the math is pretty simple. For those same expectations, they need to overperform by about 2 goals every time (on average). If they can do that for a month or two - that's exactly what the rating would show. And if they have injuries or other challenges and they underperform by 1 goal (on average) for a month or two, their rating should go from a 41 to a 40. Whether it's done by ensuring the blowouts come in exactly on target, or a close game becomes a little less than a close game, the benefits (or penalties) would be the same. The interesting part is by the end of that journey, the predictions will start to take into account the new performance, and then the team will need to perform at those new expectations going forward, to keep that improved rating.
This is a bit inflammatory, and no matter how many times these insinuations are stated as fact - I haven't been able to find any actual examples that show this occurring. I looked in Southwest, Texas, Florida, NY, IL, and I couldn't come up with a representative example of a highly ranked team (Top 25 nationally) that had a limited game history that was just against weak opponents in their own league where they had an inordinate amount of blowouts. There may be ones out there, and I'm hoping that someone can point me in the right direction. What teams or leagues are you referring to? But in this thread - Solar (ECRL and ECNL) are both counter examples that show this not to be true. There is a ton of game history outside their own leagues that ensures that the ratings assigned take into account all sorts of competition, which both help and hurt from time to time depending on the opponent.
A cynic might say that Slammers RL are guilty of what you're accusing others of doing. Undefeated in league, 3 goals against them all year, yet the team isn't chasing enough worthwhile opponents to actually test their mettle in tournaments where they have a reasonable chance to lose. How many of these talented girls should be on an ECNL team instead once reaching 2009G?
There is a well-respected club near me that runs their youngers 1 year up all the way through U12 for their top team, and they are quite successful. In some tournaments they are entered 2 years up. Turns out that a very good 2013 team can in fact embarrass an average (or weaker) 2011 team. Watching a bunch of tiny sprites fly around the field against a bunch of larger but often slower and less talented players, can be fun to watch. Once kids get much older than this though, it starts to become a safety concern and the size/strength differences can't be ignored forever. I'm with you, 8 year olds playing 12 year olds would be a bit of a stretch. And validating whether a "35" at 2014 is exactly the same as a "35" at 2010, isn't something that can probably be confirmed in the real world. The cross-play from age group to age group to age group is just too far a chain of results to ensure that there isn't drift between the rating scales of the two teams. So it's a bit of a moot point. We don't know whether the 35's mean the same in that comparison, and while we can debate it - it can never be settled authoritatively.
You, and I, and probably most people other than Mark and any of his team members/helpers, have almost no idea about what goes in to the algorithm(s) - other than it's related to goals scored in prior games. The types of tweaking, caps, weightings, timings, and all of the other manipulations that can be done to past data to see if it better predicts future data is exactly what Mark has been doing for years. And continues to do in this app. What would you do if you had a system with millions of games, and had the chance to run countless models on the data to see how well different factors and operations can be applied to better predict game results (that have already happened)? All of the hypotheses that one could come up with ("weight last months result 1/3 less than this months, or weight it 1/2 less." "Drop results older than 6 months to .3 weighting vs. .2 weighting" "Refactor goal differential so 5 goal differences are only slightly higher than 4 goal differences to minimize blowout effect") are the types of things that you get to optimize for when you have the data. And all of that was massaged over time at YSR, but now is even more possible with the app. Just look at the predictions - there is now an estimate for what the score is likely to be. There are percentages given for chance of win / tie / loss. These don't come from a random number generator - they are provided as the back end of a ton of past data on team performance.
Is any of it ever going to be smart enough to predict the future with no variation - of course not. The real world doesn't work like that, for reasons that are obvious to all. But the griping about a simplistic algorithm or bad results feels off-base - the griping instead should be if the predictions turn out to be off or unusable on a regular basis. If someone has a better way of looking at team data on their own and predicting future performance more effectively than this - prove it to yourself! Use the data you have, look at the upcoming games, and document your predictions. If it turns out that the app isn't any better than what's possible without - save the $10 and don't give it a second thought.
You have to remember slammers had 2 ECNL teams and 2 ECRL teams per age group. These are the girls who didn’t get picked for either ECNL teams. So there’s no sandbagging on this one, just a lot of talent at slammers and not enough appeal for players to go elsewhere. Team did win Surf Cup and as the 3rd Slammers team was placed in their appropriate bracket. Games were competitive in the subsequent games including the Beach match that was a 1-4 loss. Even though this team has had closer games when they played previously against each other, but I digress
As for Solar RL, I think there’s plenty of data to show they should drop off with 4 losses to teams rated significantly lower than them and a series of underperforming match outcomes yet they remain very highly ranked as an RL team. But either way, time will have everything play out and stabilize.
We’re looking at a transition year moving into 11v11 and some teams are not reflective of their current team because of that transition as well as roster changes that normally come with moving to 11v11 and new leagues.
I don’t know how much historical data weighs versus current results, but there’s teams that are stacked high because of their matches from nearly 2 years ago (slammers included). What I’d like to know is, when we are rating teams, are we scoring them based off what their rating was at the time they played versus what their rating is now?
An example is Slammers McCarty Black was highly rated (top 13 in SoCal, top 25 Ca) when it was 9v9, but that team no longer has that roster (95% of the team went to NL/RL teams). So naturally their ranking drops, but at the time they played higher rated teams, is the algorithm accounting for their rating at that time and not the current rating? Because I’m seeing the matches between that team and the current team showing as a deduction in points even though at that time, the predictive value probably would not have changed either team’s points value.
Don’t get it twisted, I’m a supporter of Mark and what he’s doing, but these are questions that people are asking
RandomSoccerFan
GOLD
All of the game history is assigned to a team entity, and the rating of that team entity is affected by every game as it happens. To see what ratings are affecting a team, it's just the listing of all games on the team page itself, with the links to the source data included at the bottom. From the app's perspective, a team is just that - a collection of game data. As game data is added (either 1 game by 1 game over time), or in bulk (as a new data source from a tournament or even a league is assigned to a team entity), that new data is used to adjust the team rating. There is no concept of roster changes, 9v9 vs 11v11, or anything else. When a new game result is added, all that is necessary is to tie this team entity to the opponent team entity, and to see if the score of the match is greater or lesser than the predicted score of the match, which is the difference in ratings between the two entities. If the opponent isn't rated at all, it seems to sit there dormant and not affecting anything until or if the opponent ever does become rated. That's really it.
Where some of the opaqueness comes in, is trying to figure out what happens when old data is assigned to a new team. If I realize that last year's team X in this tournament is actually the same as team XX that I'm looking at, and I pull in 4 games from February 2022, how does that affect current rating? Basically - is it looking at a comparison of what the rating was for both opponents back in February 2022, accounting for that new data, then coming back to present day to calculate current rating. Or, is this new data applied to current rating, but unweighted very significantly because it's 8 month old data anyway. My hunch is that it's the latter. Otherwise - there would have to be a daily history of existing rating kept for each team forever - and that seems unlikely from a data management standpoint. Especially since we know the app is calculating ratings in real time as soon as new data is added; there isn't a daily batch process. We know that there is some history kept for the rankings graphs to be plotted - but it's unclear whether the rankings are calculated once and then static, or if those rankings are variable due to past ratings being variable.
What I do feel is that older game data may matter less than one would think. We had significant team history issues in the migration from GotSoccer to GotSport. Team history between several teams in the club got munged together and then actually assigned to the wrong team when entered into GotSport. It went on long enough before being noticed, that by the time it was, it actually become easier to just leave them as is (initially wrong) than try and sort everything back out. This not only hosed GotSport rankings for awhile, but affected the team's ratings in YSR (and now the app). But after a few months of new and accurate game data, the rating for our teams shot up significantly (5 points in less than a year). To move it 1 point, it really can be done in just a few weeks of games. A team that performs at level X for 6 months just isn't going to be at the rating X+2 because of what happened a year ago or earlier, the ratings are much more fluid.
I'm trying to make sure I'm looking at the same team you are referring to. Is it CDA Slammers North McCarty Black Whittier? It looks like one that is playing in the SOCAL Fall League, most recently losing to Chelsea SC Langsford 1-2 on 10/30? It looks like it's now 115 in state. The game results look pretty spot on going back all the way to May, I count 32 games - of which 28 of them they performed as expected. In 4 of them they underperformed. You have to go all the way back to January before finding any games where they overperformed enough to affect their rating much positively. But back to present day - they currently show a 38.07, and that's the rating that would be used to predict a game tomorrow, regardless of any history.
I see that there is a "CDA Slammers Whittier McCarty Drop" team entity in the Unranked teams area. It has game data assigned to it as recent as February 22, at the SoCal State Cup G2010 Super. Earlier data includes the 2021 Silverlakes Fall Showcase, the 2021 Players Challenge Cup, and some more. None of this game data is being used to rank any team at all right now, as it's not assigned to any team that has played in the last 7 months. Does this represent the older team you're referring to?
Yes, that team dissolved right after their last match with Beach FC Ayala. Coach pulled the team out from state cup for a variety of factors. That group of girls became the Arsenal ECRL team for a few months before there was a coaching change and the team dissolved there after with girls now spread amongst various teams (Blues NL, Slammers RL, Pats NL, Arsenal NL). However, that team name lives on as his flight 2 team, McCarty White is now using the McCarty Black team name (signifying that this is the top team from the branch).
The Arsenal 2011 Pre-ECNL team is now the 2010 Arsenal ECRL team (and despite trying to merge these two, they continue to be separated).
So back to the prior results, you can see the Slammers North team performed very well until February 2022 at which point that roster was no longer the same (only 1 player from that roster is still on the team). But when it was performing, the rating was higher. You can see the same type of thing happen with other teams (Legends ECNL was ranked lower than McCarty’s team at that time, so those victories improved Legends ranking when they were played, however, if you look today, they are actually bringing Legends ranking down.
Attachments
Here’s a screenshot of what the rankings were in January 2022 back when McCarty Black had its original performing roster that was higher ranked than the Legends ECNL team at that time. Today, Legends is being penalized for their 3-1 victory over that Slammers Whittier Black team
Attachments
RandomSoccerFan
GOLD
I'm looking at the Legends FC ECNL history that you attached, and I see what you mean about some of the games back in the April 2021 time period. They show beating the Slammers North McCarty Black Whittier team (3-1), and a draw (1-1). At the time, if Legends were higher, it would have helped their rating. But now the ratings are reversed, and it looks like that loss is instead hurting their rating. It's a fair question - and it's not impolite to ask Mark directly for his thoughts on how this is accounted for or dealt with. My hunch is that it simply doesn't matter that much, as games older than 1 month, 2 months, 3 months, 6 months, 12 months, get discounted in weighting so much that the effect on the present rating is minimal if it's even noticeable.
This is testable, if you are curious and want to do it with a team you have permission to tweak ratings for. Just delete all source data from over a year back, and see how much (if any) the rating changes. Any change will show immediately; ratings are calculated on the fly when source data is added/removed. Then just add that source data back to the team so there is a fuller game history once again.
This is testable, if you are curious and want to do it with a team you have permission to tweak ratings for. Just delete all source data from over a year back, and see how much (if any) the rating changes. Any change will show immediately; ratings are calculated on the fly when source data is added/removed. Then just add that source data back to the team so there is a fuller game history once again.
Mark’s a busy guy. I try not to bother him unless it’s necessary. I’m sure he gets plenty of emails. He’s usually very prompt when I’ve reached out in the past, and I’m sure he’ll iron this all out.
RandomSoccerFan
GOLD
This should be fixable with a few clicks (by either of us, or anybody else who can confirm the team info). I can find the Arsenal 2011 Pre-ECNL team.
But I can't find anything that looks like a 2010 Arsenal ECRL team. There is nothing named "Arsenal" in 2010 Girls in either Ranked or Unranked teams that looks to be what you are describing. Can you share any link to where they are currently playing, or any other info that might help troubleshoot why they aren't showing up?
Sporting California USA = Arsenal
RandomSoccerFan
GOLD
Right - I went through every one of those.
None of them have played a game since August 2021. All appear to be dormant teams. Is that correct? If so - who cares? Any team who hasn't played a game in 7 months becomes unranked anyway.
RandomSoccerFan
GOLD
Wait, I understand what you mean now. Apologies. The team isn't called Arsenal at all now. Is this the team? And it should have the Arsenal 2011 team data tied to it?
Yes sir