Broom is listed as a prohibited plant under the Pest Plant and Animals Act 2005. It is an offence to grow a prohibited plant with a maximum fine of 50 penalty points ($500,000). Thus you would be doing the landowner a favour. Recklessly propagating a plant (ie telling you not to remove it) is a further offence.