Working out your lower body can help with your movement, balance, athletic ability and can help you perform better with compound exercises.
Also, having more muscle mass in general means your body's energy expenditure increases (more fat loss). Plus, the exercise it takes to do the leg muscles also burns some calories.
It may not be a priority for everyone, but I wouldn't call it a waste of time.