Meaning – to remain neutral in an argument or in a decision
In Context – The two of you can fight it out between yourselves. I am not going to take sides in your argument. I’m on the fence here.
Note – You can also say, ‘to sit on the fence’. There is no change in meaning.