I have fluctuations in speaking ability, but not from lack of use. I've been living like a hermit for about 10 years, but can still usually manage basic, short exchanges (I was never very talkative in the first place), and the occasional extended one, if necessary. I do get a little rusty, but it seems limited. I was about to say that maybe the headaches and exhaustion were worse, but they're probably just more noticeable because they aren't happening all the time, anymore.
Other things like lack of sleep or overload are what give me those "blank out" moments. For me the big problem with conversations is how slow my brain is at processing the words.
But I've never been very talkative, so maybe you're (OP) talking about a higher level of conversation that I ever get to in the first place.