... I can always go back to my original church...
Are you sure? You might want to check with that church first.
I've gone to a lot of churches in my life and have carried there and known others that carried there. I've never seen open carry in church, even from LEOs. I would also consider it an unnecessary distraction. I'd cover it up and go on about my business. YMMV.