The issue is that some think the stat is worthless and some put more store in it than they ought to. It's the best single indicator of performance that we have. But it still isn't very good, particularly on its own.
The truth lies somewhere in the middle. It's probably a little bit better better than corners but, probably, much worse than corners and possession used together.
I would largely agree that pointing to xG alone and claiming that we were unlucky shouldn't wash with fans, particularly over a sequence of games. Either the results or the xG tend toward the mean. However, in our case, just watching games should be enough to suggest we, probably, deserve a couple more points.