Indeed. For lower levels and rec leagues, I have a very high tolerance for lack of competence, and generally I am the most forgiving of ref mistakes among the parents of kids on my son's team (having been a ref for around a decade in my youth).
I have had probably an average of 2-3 games per year where I though the ref was incompetent to the point where I would file a complaint, if there was an easy way to do so. These would be things like missing easy calls repeatedly, and/or letting the game get out of control to the point where the kids were in danger of getting injured. The team plays probably 50 games a year, so that's a pretty low percentage.
I have had one game in the last 3 years where I though the ref was clearly biased and paid off to cheat for the other team (the aforementioned State Cup game, at the Legends owned facility, against the Legends team). He called a penalty against our player for a slide tackle, which he admitted to the player on the field was 100% clean after the fact, but he called the penalty for a perceived dangerous play (which, even if that was a correct call, which it was not, would have been indirect). He called another foul later when our player got pushed over from behind, blatantly... on our player. He was 100% corrupt, laughably, and not even trying to be particularly subtle about it... to the point where we all stopped getting angry, and just told the players that sometimes the refs are just dirty, and they were clearly cheated out of that game. Apparently Legends cheating is fairly well-known, though, to be fair (I learned this afterwards).
Also, to be clear, I'm not even asking for penalties against bad refs, I just want a path for parents to identify and report perceived bad refs, to mitigate some of the entirely one-sided handling by the leagues and tournaments. If there's no way to weed out bad refs over time, then parents just get more and more frustrated, which will lead to escalations. More one-sided responses will not help this ongoing and increasing problem.