Pattaya has been growing too fast during the past decades, and so has pollution. There are some improvements now, but I've never been swimming here and I never will (you might come out green colored with blue bubbles...).
A better option is visiting one of the small islands near Pattaya (Ko Lan) or the far islands where the diving guys go (Ko Rin, Ko Man Wichai). You could rent a boat on the beach that will bring you there (not too expensive if you can share it and go for a day). Or just go diving ;-)
