If some people, e.g. Wahabis and Salafis, regard dargahs as places of idol worship ( because dargahs are shrines built on graves of sufi saints ) they need not go there. Nobody is compelling them. But why should they physically attack dargahs ?
Everybody should be free to profess and practise what he/she wants. By going to a dargah am I breaking the head of anyone or chopping off his limbs ? Secularism and tolerance is the only way to peace and prosperity, particularly in our sub continent, which has so much diversity.
That is the path shown by our great Emperor Akbar, the real Father of our nation, who gave equal respect to all religions and sects.