You are literally making things up:
"Imagine you have an empty box..."
Don't you see the problem here? You first create an empty box out of thin air, then put it in another box just to prove there is an object in it! Is this the way mathematicians perform their 'magic'? Fooling the public by suggesting there is something there, when there isn't?
But wait... this is math, THERE IS NO EMPTY BOX! And if there is, what is it made from then? 'Set-material'?
If you take an actual box and put it in another box, I agree, there is an object in it. But in Math, a set is only there if there is something in it, the set IS the collection of objects NOT the container around it.
If you have... whatever it is you have, and give it away - some old chair - , there is simply nothing there anymore, not even the memory of it (except in your head, and perhaps the imprint on your rug or the circle where the rug did not fade). So no empty set there. Our would you say there is an empty 'chair-set' standing in your room? In that case there is also an empty 'nucleair submarine-set' standing there, next to the empty 'fallen deities set'...
And yes you are right, once we had zero, we had negative numbers, but that's ONLY in our minds and our conventions. Once we put a threshold or reference somewhere we can go below this threshold and call it 'negative', but our thresholds are arbitrary. Zero degrees is when water freezes, but we could have chosen the freezing point of Nitrogen as zero degrees.
But hey, there is an actual absolute zero! And can you go beyond that? Guess...
Zero in nature is absolute, you cannot go beyond it. If I give you a cow, that's NOT the same as you giving me minus one cow. But only in Math it is!
But what we CAN do is define a direction and call that 'positive' so we can call the opposite direction 'negative'. If I give you a cow, to me that is a flow of value in the negative direction, but to you in the positive direction. Just way of defining flow and direction, the cow stays an actual cow, regardless of the direction.
In a mirror universe everything is reversed (you know with all the CTP symmetries) and it works the same. Positive and negative are just points of reference, frames of reference. What's positive to me is negative to you. Only a matter of perspective and direction. A negative charge just means an opposite spin related to positive spin and thus charge.
And so, we can work with zero and negative values because we have conventions about the flow and direction of things, money that flows into my bank account is positive, money that flows out is negative. But have you ever seen a pile of negative dollars? A pocket full of negative coins? BS! Even if you lend someone one dollar (and your fortune grows with -1 dollar) it is still an actual dollar bill.
And if i drive with my car and plot this route on a Cartesian grid, the resulting vector may be some number, zero when I return to my starting point, perhaps a negative direction but the actual distance I've traveled ar real miles (or km here in the Netherlands). Nothing negative there.
You are right, calculus needs the zero as a placeholder and so it is something real, but ONLY in math, and thus our modern world would fall apart without it, but that is also true for money. You can't have a monetary system without 'money' and the flow of it. Money is another invention, a way of standardizing 'worth' - another abstract concept - but it is arbitrary and if the system fails, you end up with a pile of green paper that is only useful for wiping your arse. Can we do without it? No. Is it real? No. It's only worth something within the system of conventions. And that is because 'worth' does not exist. What is a Picasso worth? If you are dying of thirst and walking around in the desert - with your precious Picasso - and someone want to exchange your Picasso for a bottle of plain 'ol water.... Your Picasso is worth a bottle of water! Our modern society is based on all kinds of lies and without them the system would fail. There is no left and right, no rich or pour, no master or servant, it's all lies we believe in.
Don't you think it is strange that in computer programming we have various way of saying 'zero' or 'nothing' that ar not compatible? There is 'zero' as a Number, there is 'nothing' as in 'null' and there is 'nothing' as in 'undefined'. Why is that, you think? Because it means something different in every situation because it is a symbol, not an actual number. Programmers understand the nature of zero apparently better than mathematicians.
Don't you think it's strange that we need imaginary numbers to do calculations in vector spaces?
It's all and only because our reference point is zero. If you would take an actual arbitrary large positive number for x, y an z as a reference - just like in the real world - and start rotating your vectors, they would never hit a negative value.
And because the convention of some mathematicians long ago about the square of a negative number is a positive number, you need i to correct this error - this flaw in math - in vector spaces with a zero reference point. Imagine the convention of -5 squared would be equal to -25 (i.e. - (5*5) instead of - 5 * - 5). We would not need i! And that would be the right thing to do because even a negative value in a vector space is in the end just a positive number, a vector with a real positive length, only in the opposite direction.