How do I treat outside potted herbs before bringing inside for winter?

I have a few potted plants including basil, thyme, rosemary, hot peppers, aloe vera and some kind of tropical palm tree like plant. I know i can spray water and phosphate free soap like Ivory on the non-edibles to kill bugs off, but wouldn't that be bad for the herbs? What do you suggest?