If it's embedded into their character and it's not a situational thing, it drives me completely up the wall. It actually makes me aggressive.
If it's situational, it depends. It might be endearing, slightly annoying, or I just never make contact with them.
With your ISTP person, is it his presence that makes you nervous?