The Man Who Found Time: James Hutton and the Discoverty of the Earth's Antiquity, Jack Repcheck, New York: Perseus Publishing (Basic Books), 2003.
page 138:
‘ . . . He [Adam Smith] cut quite a figure. He always carried a cane, but never used it—rather he rested it on his shoulder “as a soldier carries his musket.” He dressed well, but not extravagantly. Like Hutton, he struck observers with his eccentricities because he often talked to himself, and his head turned from side to side while he walked. One of his biographers commented, “Often, moreover, his lips would be moving all the while, and smiling in rapt conversation with invisible companions.”
'Like Hutton and Black, Adam Smith was unmarried. Though he was sociable and fond of company, accounts of the period indicate that he was shy and spoke up only if called upon. . . ’