Here's another way to look at it. The temperature difference between two states is related to the heat added or taken away from the system. So imagine a curve that connects the temperature of the thermal reservoir that the system is in contact with on the x axis, and the heat that can still be extracted from the system on the y axis. The shape of that curve is set by the physics of the system, but in the Kelvin scale, we want the curve to hit zero heat when the temperature is zero (so the curve hits the "origin" of the graph). Now given all this, you can scale (stretch or compress) either axis completely arbitrarily-- it's just the unit of temperature and the unit of energy that controls the scaling of the axes. The unit of energy is chosen some other way and doesn't matter to your question, but the unit of temperature in the Kelvin scale is chosen such that, if the system is water, the difference in temperature between the point where the water freezes and the point where it boils is 100. Once you set that scale, the shape of the curve will dictate that the difference in T between where it freezes, and the origin, is 273. So it is the choice of the 100 for water that sets the 273. The Celsius scale uses the same convention of 100 between freezing and boiling, but it decides to take the zero temperature to be the freezing point. That's why it differs from Kelvin by 273.
So, you are wondering why 273 seems so arbitrary, and it's because the 100 for water is not arbitrary, combined with the desire to make a step in the Kelvin scale correspond to the same difference in heat as a step in the Celsius scale.