In "larger" format games, trees are placed on a cloth or similar. There, it's perfectly fine to "make room". But a skirmish game has terrain for specific reasons: mostly to get in the way.
I was participating in an IRA game by Leo Cronin at a convention. Some scoundrel tried to move buildings so his truck would fit in an alley. That was definitely beyond the pale! I bet the scalawag would've also liked to exit through the sides too! Balderdash! Harrumph!