I read your post this morning and my heart went out to you. You are juggling far more than anyone I know! Are there other children in the home as well?
Have you considered asking the visiting teacher for more activities, projects that he could do at home? Because he does talk a lot at home, it sounds like he is curious and wants to interact with you. If the "schoolish stuff" came from you, and you kept it light and fun, he might willingly cooperate. And there's no reason why you couldn't work with him throughout the year, even without the visiting teacher. See if she will be your ally in teaching your son.
Because he does sound curious and involved, how about starting with reading to him A LOT. Start with books below his level and get him hooked on spending that special time with you. Most of the stuff we learn in school can be learned from reading anyway.
If you decide to take the plunge and homeschool full time, stop by again. There are lots of special needs math and reading programs out there that could be a help.
Whatever you decide to do, I hope you take regular time to put your feet up and pamper yourself a bit. Let us know how you're doing.