In short: According to several recent systematic reviews of studies, milk consumption is not associated with significant side effects, increased mortality, cancer, osteoporosis, heart disease or stroke.
Milk and dairy products: good or bad for human health? An assessment of the totality of scientific evidence (PubMed, 2016)
very few adverse effects have been reported.
milk and dairy intake was inversely associated with colorectal cancer,
bladder cancer, gastric cancer, and breast cancer, and not associated
with risk of pancreatic cancer, ovarian cancer, or lung cancer, while
the evidence for prostate cancer risk was inconsistent.
There was no consistent association between milk or dairy intake and
cardiovascular disease, coronary heart disease or stroke
Food Sources of Saturated Fat and the Association With Mortality: A Meta-Analysis (PubMed, 2013)
...high intakes of milk, cheese, yogurt, and butter were not associated with a
significantly increased risk of mortality compared with low intakes.
Milk and acid-base balance: proposed hypothesis versus scientific evidence (PubMed, 2011)
Recently the lay press has claimed a hypothetical association among
dairy product consumption, generation of dietary acid, and harm to
human health. This theoretical association is based on the idea that
the protein and phosphate in milk and dairy products make them
acid-producing foods, which cause our bodies to become acidified,
promoting diseases of modern civilization. Some authors have suggested
that dairy products are not helpful and perhaps detrimental to bone
health because higher osteoporotic fracture incidence is observed in
countries with higher dairy product consumption. However, scientific
evidence does not support any of these claims.
Dairy product consumption and risk of hip fracture: a systematic review and meta-analysis (PubMed, 2018)
Consumption of total dairy products and cream was not significantly
associated with the risk of hip fracture. There was insufficient
evidence to deduce the association between milk consumption and risk
of hip fracture. A lower threshold of 200 g/day milk intake may have
beneficial effects, whereas the effects of a higher threshold of milk
intake are unclear.
Disclaimer: My conclusion is not that milk is healthy and I am not trying to encourage anyone to drink it. I just haven't found convincing evidence that it is unhealthy.