Always! The trouble has always been that I write too slowly to get everything down on paper properly.
The real skill would be to learn to listen and immediately distil (for memory or for note-taking) the salient point being made. I've always admired people who could do this: I'm hopeless at it. I have, however, learned (and taught others) some useful lessons about how to go back later and mine the factual gems from the verbal spoil heap.
I ALWAYS provide comprehensive handouts, and will allow students copies of my own speaking notes if they feel they need them.