I do not believe Jim nor myself are the only Christians who feel as we do about the cross.
I'd also suggest that , and this is a personal opinion, it is best not to worry what anyone else thinks as to why you wear a cross. If they think it a pagan symbol let them dare ask. And that may then be God's doing. Affording the opportunity to witness to the inquisitive one who presumes and can then be set straight. And perhaps set to go straight toward the narrow road. You could actually lead a soul to find eternal salvation for which the cross means everything. And does not change in that importance due to anyone's contrary belief about it.
Remember, there are atheists who think we're fools when they see us wearing a cross. There are Satanists that feel the same way. Does their opinion mean anything ? No. What should mean something, the only thing, the only opinion that counts, is yours.
"I wouldn't want anyone to think I'm wearing it for any other reason and I want all the glory to go to God and not to some weird pagan group"
You're not able to control what people think.
If they think hard enough they may dare to ask about your cross. That's a blessing isn't it.
As for me, I don't worry about what people think. That they do is a positive. That they opine about my personal attire is their prerogative. That their opinion would make me change anything about me? Not a chance. I'm a temple that carries the most holy spirit God within. And made in the image and likeness of God I am living breathing proof he has a sense of humor and is into abstract art.
And I LOVE him for that!