- Seconding Carey, it is most likely (not impossible) that your "saggy" chest is due to Pseudogynecomastia, which is caused in your case by losing weight too fast.
If male breast enlargement is caused by glandular proliferation, it is
defined as gynecomastia. If it is caused by increased fat deposition,
it is defined as pseudogynecomastia
Erol S, Orhan E, Sevin A, Erdogan B. Trauma: a new pseudogynecomastia
cause. Aesth Plast Surg. 2010;34:404–405. doi:
If you truly have gynecomastia then a doctor can diagnose you with it (usual things to look for are large, puffy nipples). In that case, your are correct with asserting that adding chest muscle will only serve to highlight the condition.
You are correct about your comment regarding poor diet causing it, but this is usually restricted to the older gentleman, excerpt from Wikipedia:
Declining testosterone levels and an increase in the level of
subcutaneous fatty tissue seen as part of the normal aging process can
lead to gynecomastia in older men. This is also known as senile
gynecomastia. Increased fatty tissue in these men leads to increased
conversion of androgenic hormones such as testosterone to estrogens.
When the human body is deprived of adequate nutrition, testosterone
levels drop while the adrenal glands continue to produce estrogens
thereby causing a hormonal imbalance. Gynecomastia can also occur once
normal nutrition is restarted (this is known as refeeding
Cuhaci N, Polat SB, Evranos B, Ersoy R, Cakir B (March 19, 2014). "Gynecomastia: Clinical evaluation and management". Indian J
Endocrinol Metab. 18 (2): 150–58. doi:10.4103/2230-8210.129104. PMC
3987263free to read. PMID 24741509.
- You severely underestimate the effort required to put on muscle, especially pectoral muscle. Once does not "accidentally Arnold" their way into being build like a brick-sh*t-house.
In the case of pseudogynecomastia, continued strength training and a continued reduction on bodyfat percentage would probably help the most. But without pictures and a proper inspection it is hard to say if it is just that you have no muscle or you have loose skin (or a combination of both).