I don't want to put words into Robs mouth, but I think he is talking about how humans are quite tribal and naturally make judgements and decisions around people who are "outsiders" on a daily basis. Not just on race, but on all aspects of what makes a person - race, nationality, gender, and so on. And some of this behaviour is reinforced through years of hearing parents, grandparents, peers say negative thins about certain groups of people.
For example, if a group of people were placed together in a room - you would probably find that people of the same nationality, who knew the same language, and yes have the same skin colour, would gravitate towards each other.
You would probably find that this behaviour increases when the subject is put into a position of danger or confrontation (e.g. walking alone on a night).
I agree with Rob (or at least my understanding of what he said), but it is quite hard to articulate - especially as people are so scared of being labelled as a racist (because the social punishment is so severe) that it is hard to have a conversation about it without people keeping their guard up.