If you do get Respite Care services, you could start out small. One of my son's caregivers is only here for 1 1/2 hours and my husband is usually in the garage working during that time.
You may also be able to hire people yourself, depending on how it works there. I used to hire a good mom friend to babysit. Then the system got a lot more difficult to use and I switched to having an organization deal with it for me.
We also have some special organizations that take respite hours - for example, there's a horseback riding facility that specializes in therapeutic riding for disabled kids. I would just read a book for an hour but he was never out of earshot.
Anyway, of course you should do what makes you comfortable, but do check out all the options before closing the door. All parents need some decompression time and having special needs kids can make it harder to find.
_________________
Sharing the spectrum with my awesome daughter.