I go to the University of Connecticut, which is a big school with over 18,000 undergrads. It also has a dedicates AS Program (albeit a small one). But the point is, at a school this size, there are going to be people like you. But with 18,000 other students out there, don't expect to sit around and have them come to you. Because that's not happening. I am actively involved in UConn Students for Liberty (a Libertarian student organization) and the Math Club (actually, I haven't attended any meetings yet, but I'm a freshman and last semester their meeting time conflicted with a class, and our only meeting this semester was postponed because of the snow - but I will be attending once we actually have a meeting I can attend). I also meet people at sporting events - sports are huge here. I've looked at small schools too during my college search process. The organizations there are more active in recruiting than those at UConn, because they have to be. But it's also harder to find those that interest you. The fact is, at UConn, there's going to be at least one (and probably way more) clubs or activities that you'll find similar people in. But you've got to find those clubs. We have an involvement fair each semester, but it's cluttered and overwhelming, and you're not going to be able to talk to every single organization, nor should you want to.
On the other hand, my high school's graduating class had fewer students than my freshman calculus class (seriously). that being said, our school had some good extracurricular, and since those people were also my classmates in many cases, it was easier to hang out with them outside the activity. But most high schools of our size don't have so many activities. And even the large ones often aren't the greatest. But it depends on the high school/college. Some large universities have very few activities, while many small ones (Ivies especially) have a lot. But you're not going to get anywhere by sitting around. Keep that in mind.