While it seems logical that you only need to consider the deficit or surplus on the energy balance to see if you'll gain or lose weight, this is not going to work because the body will regulate the metabolic rate to keep a certain amount of energy reserves. How much fat reserves your body decides to keep will depend on your physical fitness and physical activity levels, the intake of minerals that are essential for maintaining physical fitness, how much sleep you get etc. etc. In general, when you live an unhealthy lifestyle, your body's regulation of its metabolic rate will tend to lead to larger fat reserves.
From a theoretical point of view, this is quite easy to understand. Whatever the precise biochemical mechanisms are that are involved in regulating metabolism (not everything is known), it remains the case that all these mechanisms have evolved in order to maximize survival probability of animals in Nature who obviously don't do calorie counting.
One of the problems evolution had to solve was how to make sure you don't starve to death due to a small shortage in the energy balance that you cannot make up for. Suppose you eat one sandwich a day worth 100 Kcal a day less and walk a bit more so that you expend 100 Kcal more per day. While this could lead to some weight loss, it cannot be the case that you'll continue to lose weight without limit. However, naive the calorie counting hypothesis suggests that a 200 Kcal deficit per day would lead to a long term weight loss trend of 1 kg of fat per 40 days. So, in a little over 2 years you would lose 20 kg of weight, which is clearly nonsense.
Animals living in the wild may find themselves having to deal with a bit less food that is also a bit harder to find. If they were to lose weight because the metabolic rate is cannot be actively regulated, it would only increase due to physical exertion, the animal would be doomed. This doesn't make sense for warm blooded animals that we know have mechanisms to regulate the metabolic rate, and who have metabolic rates that are ten times higher than what they need to just barely survive.
Instead, it makes far more sense to make the metabolic rate dependent on the degree to which the fat cells are filled. So, if there is a shortage on the energy balance, the animal will initially lose weight, but then the metabolic rate will be down regulated, correcting the energy balance, a slight surplus will be created, allowing the fat cells to be filled.
While the biochemical mechanisms the body uses for this are not well understood, but recently it has been found that fat cells produce leptin, the more filled a fat cell is the more leptin is produced and besides regulating the appetite, leptin will let the hypothalamus produce more TRH, and TRH will let the pituitary gland produce more TSH and TSH will let the thyroid gland produce more thyroid hormone.
Then the body will likely also make the set point for the fat reserves dependent on factors such as the amount of food intake, stress levels, sleep etc. The whole point of the fat reserves is to maximize survival probability, so the probability of a food emergency, the time it can survive without food etc. will all influence the set point for its fat reserves. It then makes sense that the outcome of evolution would be that the set point would be set higher when the animal has less to eat, has more stress doesn't get the optimal; amount of sleep. In that case, a food emergency is more likely and when it happens it is less likely to survive on some given amount of fat reserves. So, the smart thing to do is to save more energy under these circumstances.
In contrast, when you sleep better, eat more and exercise more, the body will think that the prospects of a food emergency are smaller, and if that were to happen you would be in stronger position to take measures to reverse the situation. So, you'll not keep as much fat reserves, because doing so does come at the cost of having to carry all that fat ballast with you all the time.
