Yes, average it in, and deal with outliers differently. I mentioned previously dropping any game which is way outside of the predicted range (say, 2 standard deviations). The problem comes when you presume blowouts are outliers when, between certain teams, they are the norm.
Also, I think you would probably see a larger spread in the point system for YSR. The top team would be more like 40 rather than 38. The bottom team in an age group would be more like 20. Right now, because of the 5 point cap, I suspect, The spread is more like 10 points rather than 20.
Sparkling Pink Flamingos was literally one of my daughter's teams. How dare you.