I left my old Southern Baptist church without saying a word. Since his church keeps you on the member list pretty much forever and does not practice church discipline at all, especially not for not attending, nobody has even contacted me. My family is still there and doubtless the members have noticed I don't walk in the door with them. My PCA church has a membership class in two weeks and when I join, I will either ask for a letter from the old one certifying my baptism there, or I will just ask them to remove me from their membership.
"I wish it were in my power, after this day, to cry down all love but the love of Christ, and to cry down all gods but Christ, all saviours but Christ, all well-beloveds but Christ, and all soul-suitors and love-beggars but Christ." -Samuel Rutherford, Letter CCIII. to William Gordon at Kenmure