When I read your subject line, I didn't know whether you meant to write, "elusive," or "illusory." And in a way, both apply. Not only is frienship elusive, but you perceive the value as illusory.
I will stay, categorically, that there is a difference between friends and acquaintances, and there is tremendous value in friends. But friendships don't come ready-built. The payback from friendship comes with time, and all friends start out as acquaintances, first.
That being said, different people have different needs. If your family are providing you with the social environment that you need, then that is all to the good. But, bear in mind that children grow up and leave home, and while your wife may always be your best friend, that does not mean that one person can fulfil all your social needs. Having friends complements your family, it doesn't detract from it.
_________________
--James