First time I've seen this thread.
The problem with your assumption is that you call numbers "measurements" then you call them a "quantity of units."
If you keep using a vague definition for a number and constantly change that definition, then it's simple to confuse anyone.
A number is a dimensionless unit to represent a quantity. A quantity of what? Anything! That's the point of it being a dimensionless unit.
And by definition, a unit in mathematical terms is "a single thing."
So, when you look at an equation like "2 + 2 = 4" you're really and truly saying "2 units + 2 units = 4 units" where "unit" can be anything in the world (so long as whatever you replace "unit" with, you have to do so across the entire formula).
So, when you replace the formula with "2 apples + 2 apples = 4 apples" you effectively proved my post for me.
And, done.