Mostly but there is no gate on the track at the top of the quarry. I have not walked right around the fence , so I don't know if there are other openings.
I was at Mugga Mugga 2 days ago and had another look at the fence. There are other places where there are wide gaps in the fencing and areas where a goat could easily get under a fence. This article from Canberra times suggests goats may have been there for some time: http://www.canberratimes.com.au/act-news/canberra-life/tim-the-yowie-man-canberra-times-readers-still-in-the-grip-of-tom-roberts-fever-20160201-gmiguc.html
My photos of Apr 16 and Jan 17 were further north than this sighting so don't form a timeline. There is only the one goat as far as I know. It will have a good view of the filling in of the quarry.