- The fact that food is not 100% digestible is a good point.
- The fact that amino acids are also not completely digested adds to the complications (normally, amino acids in a bomb calorimeter are completely oxidized but in the body, they are not, rather they are oxidized incompletely to urea & creatinine, which still have energy).
- The fact that each of the 20 amino acids has slightly different variables only makes matters worse (by a small factor, but hey, scientists sometimes want to know these factors).
The values for carbohydrate (4), protein (4), and fat (9) are based on
Wilbur Atwater's early studies that built on
Max Rubner's bomb calorimeter experiments in the late 1800s. Rubner's early research only counted the energetic value of fully oxidized (combusted) hydrocarbons (of which fat, protein, and carbohydrates are more or less a subset) so that his values were larger than 4,4,9. Though Rubner figured out that protein in the gut is incompletely oxidized (as compared to complete oxidation in a bomb calorimeter), he never went as far as account for fecal losses. Atwater, on the other hand realized that fecal losses were an important variable and must vary according to the food, so he did what a true scientist would do...
he stuck the caca in a bomb calorimeter to determine its caloric content
He did this for all foods consumed so he could subtract fecal losses from food intakes to provide better estimates of what he called "digestibility".
Thanks to Atwater, we finally had an idea of how much energy we can get from certain foods. He showed that digestion is amazingly efficient, but not perfect. He showed that high fiber foods definitely reduce the availability of calories. He also refined the use of bomb calorimeters for nutrition studies (which up to then had been used for non-digestive studies and in some ways, not as accurate for digestive studies). His original values were 4, 4, 8.9, but in 1910, the USDA rounded them off to 4, 4, 9.
He figured out much more than that, but unless you are a metabolism nerd, you probably don't want a long, soporific outline of how he influenced dietary thought. Suffice it to say that
most questions you might have about calories, he probably addressed in a very deep way... go read his works to get an idea.
- 1955 Widdowson: Assesment of the Energy Value of Human Foods . a great outline of the history of the calorie and its measurement
- http://www.nal.usda.gov/fnic/foodcomp/Data/Classics/ah74.pdf. the actual charts for thousands of foods and the methodology. everything you want to know about how it's done is here.
There are many more ways in which energy efficiency is lost, but most of this is of academic importance only... not of much use in daily life as the losses are small in comparison to the truckloads of food we consume. Nevertheless, if you are curious, I made a chart a while back that outlines some of the many ways in which it happens. (https://docs.google.com/open?id=0B05o7uyVl6pab09jRWczNXFqeG8)