Actually, at least in the US, these things are pretty clean. I live by a commercial oyster farm and get to see what they go through. The water has to be VERY clean and free of pollution for the oysters to be allowed to be harvested and sold. There aren't many places left in the US suitable for oyster (and other shellfish) farming that can pass those tests. The water and the oysters themselves are constantly tested and quite often shut down from harvesting until some temporary pollutant is gone and has worked it's way out of the oysters. Sometimes all it takes is a little rain after a dry spell that creates polluted runoff from cow pastures to shut them down. So if you buy oysters in the store or from any kind of commercial outlet (restaurant, etc) in the US it's a pretty safe bet they didn't come from some polluted backwater swamp.
Now, chicken, beef, or other farm animals pumped full of drugs and hormones all their lives... That's a different story. Seems the government doesn't care about that kind of pollution. The big corporations that produce that kind of food don't seem to be under the same kind of scrutiny.