You bumped into a little known "hitch" when giving authorization to assign payments to your credit card. Once you have done so, you cannot take away the authorization ....only the organization to whom you've assigned the right can do so.
That has not been my experience. I have canceled automatic credit card payments with more than one organization without having to cancel the card itself. Of course, it might vary with other businesses.