I missed this at the time, but I think it's worth a reply.
The thing is, the ref is not always in control of whether you "notice" them. There's the obvious case of players steaming in to terrible tackles or starting a fight for no reason, when the ref clearly has to be involved, but there's more subtle stuff like the way the TV directors focus on the officials after any decision.
If a 50/50 ball bounces off both players at the same time and goes out, in the amateur game the ref will give a call one way or another, the bloke who loses out will probably say "****'s sake" and get on with it. In the Premier League, they'll get out the Ultra HD slow mo and spend ten seconds working out whose shoe lace was last, then have a lingering close up of the ref or linesman.
That's not the officials' fault, but you're noticing them, because someone else has decided you will.