When I set up my tank, I added some coral skeletons, a cup and what looks to be a fairly large staghorn. I figured they would get covered with bacteria and would help with nitrates. Now I just saw something about not using crushed coral due to it releasing phosphates

.
Am I causing my own pollution problems?
The cup especially is just covered with hair algae, but it's also where my clownfish sleeps at night.