It is the selling club. Not the fans.
But player's agents would challenge a selling club if they were being ridiculous. Agents have a good knowledge of the market.
Let's say
Villa express an interest Rogers publicly
Rogers is interested by offer.
Boro demand £20m for Rogers.
Villa tell them to F Off
Rogers tells his agent he's unhappy he's not been given a chance to speak to them.
Agent assesses situation. If £20m is a fair valuation he would explain this to the player. However if isn't, a bit of pressure will be rightfully applied by said Agent.
Maybe an agreement can be made, wait 6 months, renew contract etc.
Maybe not and player ends up disgruntled
But it's not as simple as just saying, well he's got a 4 year contract so he's worth whatever we say he's worth. That's not how it works. They are human beings not commodities.