State Cup

[JB805]

The old YSR site said that it was 70% predictive for a game's result (0% meaning no better than a coin flip, 100% meaning getting every single game prediction correct). Whatever the metric is, it's not a guess or an estimate; it can be easily calculated by comparing predictions with actuals. I'm curious how SR is doing more recently, as it's been staggeringly good for at least the teams/leagues that I'm following. I wonder if Mark would be open to again monitoring and posting that overall stat on the SR FAQ.

So far for the Super Cup, we're at 75% accuracy.

NBA moneyline favorites were accurate 67.25% of the time from 2017-2022.
NFL moneyline favorites were accurate 66% of the time whenever this was research was conducted.



For the 3 tournaments that I've tracked, SR is at 68.18% accuracy . I'd say that's pretty damn accurate.. I'm also tracking the Goal Differential predictor but that will be another post in it's own when I have time.

 

[RandomSoccerFan]

How did you choose to define if the prediction is accurate or not? If it says Team A is going to win and Team A wins, it's clearly accurate. If Team B wins, it's inaccurate. But if Team A ties Team B - is that accurate, inaccurate, or defined differently? Alternatively, if the app predicts a tie, and instead one team wins, is that an inaccurate prediction? The app does give percentages for win/loss/tie predictions, but for obvious reasons the chance of a tie is comparatively low, so the chance of accurately predicting a tie correctly is low as well.

Another variable (that can be controlled for, to some extent), is the quality of the game data assigned to each team isn't all at the same level. Teams with more problematic names, or clubs who sometimes go by initials, sometimes full names, or often change the name of individual teams sent to tournaments, are less likely to get all of their game data loaded up correctly into one team entity. To get the best predictions between two teams - it can be helpful to look into the game history and see if there are any obviously duplicated teams that are sitting in the unranked pile. Pro subscription users can update any team's data if they notice problems/discrepancies - but you shouldn't mess with teams unless you're essentially 100% sure, with the risk of losing editing rights entirely if it's shown that your changes/fixes aren't in line with SR's goals. A few weeks back a small local tournament up north was held during the gotsport outage, and a spreadsheet of the results was loaded manually into SR; none of the team data actually got assigned to any existing team for any match; it was all unranked. I spent some time one evening and went through the brackets to connect 'em up for the teams I was confident of. For the SoCal State Cup Youngers in particular, it looks like there was a partial load of the data to SR, and then at a later point there was a more complete load. It was likely related to GotSport issues as well. This resulted in some teams having duplicate game data for a few games if the names matched and they were pulled together automatically. In some (but not all) cases, the game data wasn't linked so the partial data stayed in unranked, and much of it probably didn't pollute the main team's data. Mark fixed this on Monday, and all of the partial load appears to have been blanked out, so the full game history (and therefore the team rating) is at least somewhat more accurate now for these SoCal teams than it was a few days ago.

 

[RandomSoccerFan]

OK - traded messages with Mark. For the data set as a whole, SR picks the right winner 85% of the time, for games that have a winner. Ties are ignored. Interestingly, for the top 100 teams in each age/gender, SR picks the right winner about 82% of the time. Evidently as teams get better, less goals are scored, which injects just a bit more randomness. He plans to post these stats/details up in the FAQ at some point.

 

[JB805]

How did you choose to define if the prediction is accurate or not? If it says Team A is going to win and Team A wins, it's clearly accurate. If Team B wins, it's inaccurate. But if Team A ties Team B - is that accurate, inaccurate, or defined differently? Alternatively, if the app predicts a tie, and instead one team wins, is that an inaccurate prediction? The app does give percentages for win/loss/tie predictions, but for obvious reasons the chance of a tie is comparatively low, so the chance of accurately predicting a tie correctly is low as well.

An accurate prediction in my data is when the outcome is the same as the app predicts. In a knockout game where the game cannot end in a draw, I will move to whichever team the app says has the higher Win %.

 

[JB805]

OK - traded messages with Mark. For the data set as a whole, SR picks the right winner 85% of the time, for games that have a winner. Ties are ignored. Interestingly, for the top 100 teams in each age/gender, SR picks the right winner about 82% of the time. Evidently as teams get better, less goals are scored, which injects just a bit more randomness. He plans to post these stats/details up in the FAQ at some point.

I believe that. I'm looking at a small sample size - 2012 girls age group, most competitive bracket in the most competitive competitions in this region. It's a tiny amount compared to the number actual games that are being tracked in the app.

 

[Carlsbad7]

Evidently as teams get better, less goals are scored, which injects just a bit more randomness. He plans to post these stats/details up in the FAQ at some point.

This is 100% true.

As teams move to 11v11 theres a bigger field, defenders are better, players are closer to the same size, goalies kick the ball to the half line, + coaches/clubs put teams in appropriate brackets. Individual effort usually cant create goals by itself. The net effect is less goals scored.

 

[RandomSoccerFan]

An accurate prediction in my data is when the outcome is the same as the app predicts. In a knockout game where the game cannot end in a draw, I will move to whichever team the app says has the higher Win %.

Got it. Nothing necessarily wrong with your interpretation, but keep in mind that it penalizes the soccer percentages much more than the compared NBA or NFL stats, as ties are almost unheard of in those leagues - while they are relatively common in soccer. It's no longer apples to apples.

 

[RandomSoccerFan]

This is 100% true.

As teams move to 11v11 theres a bigger field, defenders are better, players are closer to the same size, goalies kick the ball to the half line, + coaches/clubs put teams in appropriate brackets. Individual effort usually cant create goals by itself. The net effect is less goals scored.

Yes - but that's not necessarily the point Mark's making. The top 100 teams all the way up and down the age groups have less prediction fidelity. The better the teams are (and the better their competition is), the lower the goal differential, which means a small but measurable drop in prediction quality.

 

[TheStr1kerz]

Surprise surprise its Lime vs Surf in the final

 

[RandomSoccerFan]

Surprise surprise its Lime vs Surf in the final

National #1 vs National #3 (CA #2). Perhaps it's not a surprise to many, but it's still quite the final!

Here's how SR weights it. 64% Rennie Lime takes it; 21% Surf wins as the underdog. (15% tie)

 

[TheStr1kerz]

I predict 3-1 Lime

 

[ColdPizza]

3-0 Surf. Lots of firsts there. Will it move them into 1st place? Probably not. Congrats to both teams.

 

[TheStr1kerz]

Congrats to Surf for winning the Super Cup! No doubt about who was the better team yesterday. Hearing that will be the last time that we see Blues Limes with that roster and they will be retooling and players leaving...

Also Rangers won the Elite Cup!

 

[espola]

The old YSR site said that it was 70% predictive for a game's result (0% meaning no better than a coin flip, 100% meaning getting every single game prediction correct). Whatever the metric is, it's not a guess or an estimate; it can be easily calculated by comparing predictions with actuals. I'm curious how SR is doing more recently, as it's been staggeringly good for at least the teams/leagues that I'm following. I wonder if Mark would be open to again monitoring and posting that overall stat on the SR FAQ.

Coin flips are 50% predictive. If their methods can really hit 0% they would have something worth looking at.

 

[RandomSoccerFan]

Coin flips are 50% predictive. If their methods can really hit 0% they would have something worth looking at.

Correct, it's all about defining terms and understanding what they represent in the right context. A coin flip will get the right answer (in a scenario where the two answers are equally likely), exactly 50% of the time, and it will trend ever closer to that 50% the more times you run the scenario. In this case, a coin flip is defined as 0% predictive. Not because it gets it wrong every time, but because it gets it wrong exactly 50% of the time. If picking the winner via a proposed method is exactly as good as a coin flip would be, the method being compared is 0% better than coin flip - essentially useless. It is 0% predictive. 0% predictive = getting exactly half the guesses right, 100% predictive means getting every single guess right, 50% predictive means getting 75% of the guesses correct.

SR is 70% predictive overall. That translates to picking the correct winner, for games that have a winner, 85% of the time. (divide the 70% by 2, add it to 50%, and you'll see how it becomes 85% ). For the top 100 teams, it's 64% predictive, picking a winner correctly 82% of the time.

But even if something has a very high likelihood of happening, that doesn't mean it's going to happen every time - as is clearly demonstrated by the finals result here!

 

[way up]

Correct, it's all about defining terms and understanding what they represent in the right context. A coin flip will get the right answer (in a scenario where the two answers are equally likely), exactly 50% of the time, and it will trend ever closer to that 50% the more times you run the scenario. In this case, a coin flip is defined as 0% predictive. Not because it gets it wrong every time, but because it gets it wrong exactly 50% of the time. If picking the winner via a proposed method is exactly as good as a coin flip would be, the method being compared is 0% better than coin flip - essentially useless. It is 0% predictive. 0% predictive = getting exactly half the guesses right, 100% predictive means getting every single guess right, 50% predictive means getting 75% of the guesses correct.

SR is 70% predictive overall. That translates to picking the correct winner, for games that have a winner, 85% of the time. (divide the 70% by 2, add it to 50%, and you'll see how it becomes 85% ). For the top 100 teams, it's 64% predictive, picking a winner correctly 82% of the time.

But even if something has a very high likelihood of happening, that doesn't mean it's going to happen every time - as is clearly demonstrated by the finals result here!

YSR is really accurate and a great tool. Of course, this is soccer though which is a game of luck at times. As kids get older, games get closer and games are decided by 1 goal often and/or even pks. This is what sucks about soccer. The best team does not always win some would say, but others could argue otherwise. I've seen games where one team had 6 missed goal box score opportunities and could not get the ball in the net while the other team got luck with a bad call by a ref or even just lucky hand ball in the box that led to one opportunity that resulted in their tie or winning goal. Soccer can be very frustrating especially when coaches are good with strategy and teams are close competitively.

Also, don't forget guest players. We've had many tournaments where everyone would predict the win, yet the game was very close, because of a few guest players raising the bar for the other team. So tournaments can elevate or drop the rankings on some teams depending on their use of guest players.

All in all, YSR is very good at ranking as a whole especially considering how 1 goal can distinguish a winner and pks can almost be like a coin flip anyway.