A Labor Minister has labelled Liberal comments "incredibly unfair" to the Muslim community in the wake of the Bondi attack.