"I just wanted to thank you for your time and consideration, but I've decided to pursue other opportunities."
I tried to get it all out quickly to pre-empt an interruption and loss of courage on my part, so I can't remember what exactly it was I said.
The hiring manager, who I've heard from at least 4 different reliable sources is super-nice, sounded super-nice on the phone, and asked if it was with their company or somewhere else, so I said somewhere else.
And I couldn't help but to apologize: "I hope I haven't wasted any of your time," and he said, "No, I appreciate you telling me now."
So I did it. I'm off the hook. I don't know if I just made a mistake, but it's done.