I haven't observed things that would lead me to believe that people disproportionately base estimates of others' intelligence on social skills; on the contrary, social skills are generally considered to be "obvious" or intuitive (hence social non-conformists are assumed to be attention-seekers, as*holes, etc., rather than lacking in intelligence) One's social behavior would have to be quite immature (or interpreted as such), or composed of very blatant social gaffes, before anyone began to suspect an intellectual problem.
However, I do think that estimates of intelligence are somewhat based on behavior in general; namely, if one moves or "looks ret*d," people will assume a cognitive disability. Having a blank expression, appearing to mill about aimlessly, staring at things, lack of body language, apparent unself-consciousness, etc.