520xi muffler

The baffle(s) in the muffler of my 520xi have disintegrated or are otherwise gone. The muffler doesn't leak and is not perforated but it is very noisy. Is there any possible damage to the engine if I don't replace the muffler?