In general, the average person will experience worsening metabolism with age. What makes it harder to lose weight for someone who is 40 versus say 20 assuming that the only difference is their age?
In general, it is the basal metabolic rate (BMR) of the organism which in a person who doesn't exercise more than average, consumes most of the calories of the daily intake. It is dependent of age, sex, population and fat-free mass, i.e. the mass of muscles in your body, which consume a lot of energy just for maintenance. Studies show that it can vary individually between 1000 and 2500 kcal/day within just one population. Many factors and energy consumers contribute to the basal metabolic rate, but approximately 70% of it is used for maintenance of the body's main organs, and the rest is used for physical activity (in an average human) and for thermogenesis and digesting your food.
It is not known why exactly the BMR decreases with age. While it can be partly attributed to lifestyle changes after early adulthood with decrease in exercise and physical activity, studies show that it is not the only contributing factor. Part of it may be attributed to "an alteration in tissue energy". But when you compare age 40 with age 20, another part is that human adolescence and body growth are active until approximately age 17.5 in women and age 19 in men, which also heavily contributes to BMR.
So, the decrease in BMR with age is multifactorial, and thus may also be influenced in many ways, the most popular one being the increase of physical activity and thus fat-free mass, which is a big contributor to it as we learned here and here.