Could you buy some appropriate clothes at the thrift store? If you buy secondhand, you won't be supporting sweatshop labor. In fact, many thrift stores are associated with various social causes, so you could find one whose position you agree with and support their efforts by spending your money there.
It might seem counterintuitive to look for "nicer" clothing at a thrift store, but many of them have a good variety, depending on what exactly you're looking for. Since you're not planning to follow the latest silly trends and want to avoid wasteful consumerism, I think that shopping at a thrift store is a good option.
Avoiding animal products is easier than it seems, and most cruelty-free products have the added bonus of meeting your "sweatshop-free" criterion. In addition to buying regular, cruelty-free clothes at thrift stores, you might enjoy browsing the Vegan Essentials website:
www.veganessentials.com
Also, not sure if this matters to you or not, but I'll suggest it anyway. I still wear the leather belt and sandals I bought before I became a vegan. I decided that throwing them away and buying new stuff before I really needed it was silly and wasteful. I obviously don't buy new leather products anymore, and when my existing ones wear out, I'll replace them with vegan alternatives, but I thought it was more sustainable to wear stuff I already had rather than consuming unnecessarily. Perhaps if you have some similar things in your wardrobe already, you can keep using them until they wear out and then buy more ethical replacements. I won't be offended if you disagree, because I understand that it probably seems a bit hypocritical.
Good luck. I hope I helped; I'm not sure that this was exactly the kind of answer you sought.