RandomSoccerFan
PREMIER
First - let me say congratulations on what looks to have been a fantastic start to this season. The team is clearly playing very well, and has become the standout team in their league. All of that should be a much more important realization than any of this silly rating discussion.
Maybe it would help to think about it slightly differently. The current rating of the team (any team) *is already* a reflection of how they've played in the past. Their exact performances in the past, the goal differentials against teams of certain ratings, all of the previous team results - this is reflected in their current rating. To keep their current rating - they essentially have to do nothing more, and nothing less, than they've already done. Looking at the current predictions in the future and thinking "that will never happen, they are too onerous" is too negative - those predictions are based on what has already happened looking backwards.
Now if the goal is to improve the current rating, and take a team for example from 41 to 43 within a certain time period - the math is pretty simple. For those same expectations, they need to overperform by about 2 goals every time (on average). If they can do that for a month or two - that's exactly what the rating would show. And if they have injuries or other challenges and they underperform by 1 goal (on average) for a month or two, their rating should go from a 41 to a 40. Whether it's done by ensuring the blowouts come in exactly on target, or a close game becomes a little less than a close game, the benefits (or penalties) would be the same. The interesting part is by the end of that journey, the predictions will start to take into account the new performance, and then the team will need to perform at those new expectations going forward, to keep that improved rating.
This is a bit inflammatory, and no matter how many times these insinuations are stated as fact - I haven't been able to find any actual examples that show this occurring. I looked in Southwest, Texas, Florida, NY, IL, and I couldn't come up with a representative example of a highly ranked team (Top 25 nationally) that had a limited game history that was just against weak opponents in their own league where they had an inordinate amount of blowouts. There may be ones out there, and I'm hoping that someone can point me in the right direction. What teams or leagues are you referring to? But in this thread - Solar (ECRL and ECNL) are both counter examples that show this not to be true. There is a ton of game history outside their own leagues that ensures that the ratings assigned take into account all sorts of competition, which both help and hurt from time to time depending on the opponent.
A cynic might say that Slammers RL are guilty of what you're accusing others of doing. Undefeated in league, 3 goals against them all year, yet the team isn't chasing enough worthwhile opponents to actually test their mettle in tournaments where they have a reasonable chance to lose. How many of these talented girls should be on an ECNL team instead once reaching 2009G?
There is a well-respected club near me that runs their youngers 1 year up all the way through U12 for their top team, and they are quite successful. In some tournaments they are entered 2 years up. Turns out that a very good 2013 team can in fact embarrass an average (or weaker) 2011 team. Watching a bunch of tiny sprites fly around the field against a bunch of larger but often slower and less talented players, can be fun to watch. Once kids get much older than this though, it starts to become a safety concern and the size/strength differences can't be ignored forever. I'm with you, 8 year olds playing 12 year olds would be a bit of a stretch. And validating whether a "35" at 2014 is exactly the same as a "35" at 2010, isn't something that can probably be confirmed in the real world. The cross-play from age group to age group to age group is just too far a chain of results to ensure that there isn't drift between the rating scales of the two teams. So it's a bit of a moot point. We don't know whether the 35's mean the same in that comparison, and while we can debate it - it can never be settled authoritatively.
You, and I, and probably most people other than Mark and any of his team members/helpers, have almost no idea about what goes in to the algorithm(s) - other than it's related to goals scored in prior games. The types of tweaking, caps, weightings, timings, and all of the other manipulations that can be done to past data to see if it better predicts future data is exactly what Mark has been doing for years. And continues to do in this app. What would you do if you had a system with millions of games, and had the chance to run countless models on the data to see how well different factors and operations can be applied to better predict game results (that have already happened)? All of the hypotheses that one could come up with ("weight last months result 1/3 less than this months, or weight it 1/2 less." "Drop results older than 6 months to .3 weighting vs. .2 weighting" "Refactor goal differential so 5 goal differences are only slightly higher than 4 goal differences to minimize blowout effect") are the types of things that you get to optimize for when you have the data. And all of that was massaged over time at YSR, but now is even more possible with the app. Just look at the predictions - there is now an estimate for what the score is likely to be. There are percentages given for chance of win / tie / loss. These don't come from a random number generator - they are provided as the back end of a ton of past data on team performance.
Is any of it ever going to be smart enough to predict the future with no variation - of course not. The real world doesn't work like that, for reasons that are obvious to all. But the griping about a simplistic algorithm or bad results feels off-base - the griping instead should be if the predictions turn out to be off or unusable on a regular basis. If someone has a better way of looking at team data on their own and predicting future performance more effectively than this - prove it to yourself! Use the data you have, look at the upcoming games, and document your predictions. If it turns out that the app isn't any better than what's possible without - save the $10 and don't give it a second thought.
Maybe it would help to think about it slightly differently. The current rating of the team (any team) *is already* a reflection of how they've played in the past. Their exact performances in the past, the goal differentials against teams of certain ratings, all of the previous team results - this is reflected in their current rating. To keep their current rating - they essentially have to do nothing more, and nothing less, than they've already done. Looking at the current predictions in the future and thinking "that will never happen, they are too onerous" is too negative - those predictions are based on what has already happened looking backwards.
Now if the goal is to improve the current rating, and take a team for example from 41 to 43 within a certain time period - the math is pretty simple. For those same expectations, they need to overperform by about 2 goals every time (on average). If they can do that for a month or two - that's exactly what the rating would show. And if they have injuries or other challenges and they underperform by 1 goal (on average) for a month or two, their rating should go from a 41 to a 40. Whether it's done by ensuring the blowouts come in exactly on target, or a close game becomes a little less than a close game, the benefits (or penalties) would be the same. The interesting part is by the end of that journey, the predictions will start to take into account the new performance, and then the team will need to perform at those new expectations going forward, to keep that improved rating.
This is a bit inflammatory, and no matter how many times these insinuations are stated as fact - I haven't been able to find any actual examples that show this occurring. I looked in Southwest, Texas, Florida, NY, IL, and I couldn't come up with a representative example of a highly ranked team (Top 25 nationally) that had a limited game history that was just against weak opponents in their own league where they had an inordinate amount of blowouts. There may be ones out there, and I'm hoping that someone can point me in the right direction. What teams or leagues are you referring to? But in this thread - Solar (ECRL and ECNL) are both counter examples that show this not to be true. There is a ton of game history outside their own leagues that ensures that the ratings assigned take into account all sorts of competition, which both help and hurt from time to time depending on the opponent.
A cynic might say that Slammers RL are guilty of what you're accusing others of doing. Undefeated in league, 3 goals against them all year, yet the team isn't chasing enough worthwhile opponents to actually test their mettle in tournaments where they have a reasonable chance to lose. How many of these talented girls should be on an ECNL team instead once reaching 2009G?
There is a well-respected club near me that runs their youngers 1 year up all the way through U12 for their top team, and they are quite successful. In some tournaments they are entered 2 years up. Turns out that a very good 2013 team can in fact embarrass an average (or weaker) 2011 team. Watching a bunch of tiny sprites fly around the field against a bunch of larger but often slower and less talented players, can be fun to watch. Once kids get much older than this though, it starts to become a safety concern and the size/strength differences can't be ignored forever. I'm with you, 8 year olds playing 12 year olds would be a bit of a stretch. And validating whether a "35" at 2014 is exactly the same as a "35" at 2010, isn't something that can probably be confirmed in the real world. The cross-play from age group to age group to age group is just too far a chain of results to ensure that there isn't drift between the rating scales of the two teams. So it's a bit of a moot point. We don't know whether the 35's mean the same in that comparison, and while we can debate it - it can never be settled authoritatively.
You, and I, and probably most people other than Mark and any of his team members/helpers, have almost no idea about what goes in to the algorithm(s) - other than it's related to goals scored in prior games. The types of tweaking, caps, weightings, timings, and all of the other manipulations that can be done to past data to see if it better predicts future data is exactly what Mark has been doing for years. And continues to do in this app. What would you do if you had a system with millions of games, and had the chance to run countless models on the data to see how well different factors and operations can be applied to better predict game results (that have already happened)? All of the hypotheses that one could come up with ("weight last months result 1/3 less than this months, or weight it 1/2 less." "Drop results older than 6 months to .3 weighting vs. .2 weighting" "Refactor goal differential so 5 goal differences are only slightly higher than 4 goal differences to minimize blowout effect") are the types of things that you get to optimize for when you have the data. And all of that was massaged over time at YSR, but now is even more possible with the app. Just look at the predictions - there is now an estimate for what the score is likely to be. There are percentages given for chance of win / tie / loss. These don't come from a random number generator - they are provided as the back end of a ton of past data on team performance.
Is any of it ever going to be smart enough to predict the future with no variation - of course not. The real world doesn't work like that, for reasons that are obvious to all. But the griping about a simplistic algorithm or bad results feels off-base - the griping instead should be if the predictions turn out to be off or unusable on a regular basis. If someone has a better way of looking at team data on their own and predicting future performance more effectively than this - prove it to yourself! Use the data you have, look at the upcoming games, and document your predictions. If it turns out that the app isn't any better than what's possible without - save the $10 and don't give it a second thought.
