So I think this depends a fair amount on your age bracket, but I do know that a lot of women are led to believe that you have to be slow to reply to a guy so he doesn't think you're clingy or too interested. For example, I've been told that if you call twice in less than an hour it makes you look needy and possibly crazy. I believe the same goes in reverse, just maybe not as strong. For younger people shorter reply times are expected, so many younger women will take it as a sign you're not interested if you take more than a few hours to reply. So I think you're seeing a few effects colliding - 1) you don't reply quickly, so they think you're not interested, 2) when you reply if you call and text repeatedly in a short period of time you may make them think you're TOO interested, 3) arielhawksquill is probably right about some women, they may just be put off by finding out that you are sleeping into the afternoon (again, age dependent, the older you are the more likely this is to be something they care about).
In short, it's all a giant, exhausting game. The easiest thing to do is is reply once with minimal information, and then let her react however she's going to.