Upon waking it struck me as semi-plausible. The fencing is more designed to keep dogs and cats in and so the half height wire and the full height deer mesh are ziptied to the poles on the interior. We did have one deer get into the fenced area and she clearly spent time throwing herself at the fence, causing zip ties to pop off. On the inside, though, the tension of the fencing material itself is a barrier. From the outside pushing in, if the corner zip ties failed, i suppose it could come down.
I assume i'm dreaming about deer because, in the past week, a doe and older fawns are walking through the front yard, in view of where i sit and work.