I have heard that there can be adverse effects from masturbation, such as difficulties with having children and/or decreased sex drive.
Can masturbating actually decrease my fertility or sex drive?
Masturbating does not decrease fertility or sex drive
Historically, masturbation was discouraged for a variety of reasons including the thought that it lead to mental health problems. However, that was disproved. The abstract of Coleman, E. (2003) states
Research on masturbation has indicated that, contrary to traditional beliefs, masturbation has been found to be a common sexual behavior and linked to indicators of sexual health. While there are no general indicators of ill health associated with masturbation, it can be powerfully negative or positive for many individuals. As an example, it is widely used in sex therapy as a means of improving the sexual health of the individual and/or relationship. Promoting masturbation as a means of a public health strategy for sexual health is highly controversial; however, there are arguments and evidence that suggest that this may be an important part of any public health approach to improving sexual health.
Masturbation was a diagnosable psychological condition until DSM II in 1968. (Ley, 2014) The American Medical Association consensually declared masturbation as normal in 1972.
Due to the false ideas on masturbation and mental health, there are not a lot of documented studies but I am going to concentrate on fairly recent studies to eliminate the falsehoods.
Another point of note mentioned in Elzanaty, S. (2008) is that
Compared with clinic-collected semen, home-collected samples had statistically significantly higher values for sperm concentration, total sperm count, rapid progressive motility, and total count of progressive motility. Semen volume, proportion of normal sperm morphology, neutral α-glucosidase, prostate-specific antigen, zinc, and fructose did not differ significantly between groups. [Therefore] results demonstrate superior semen quality in samples collected by masturbation at home compared with at a clinic. This should be taken into consideration in infertility investigations.
Since sperm are short-lived, they must constantly be replenished, so the testes produce 1,500 sperm per second. Plus sperm is produced during the whole of the male life. If the dead sperm is not released through sex or masturbation, it will automatically be released by the body.
Some males find that they are more prone to nocturnal emissions during times of less frequent sexual activity because they are not ejaculating as frequently from masturbation or sex with a partner, and in fact — as @Gabri pointed out, and studies have confirmed — high ejaculation frequency was related to decreased risk of total prostate cancer (Leitzmann, et al., 2004).
There is even less around for female masturbation, but one study suggests that female orgasm induces ovulation (Pavličev, and Wagner, 2016) so that would suggest that the chances of conception would be higher. The difference between men and women is that there is only a finite amount of eggs. The human female has all their eggs at birth and no more is being produced. Once they are all released, that's it.
Coleman, E. (2003). Masturbation as a Means of Achieving Sexual Health Journal of Psychology & Human Sexuality 14(2-3): pp 5-16; DOI: 10.1300/J056v14n02_02
Elzanaty, S. (2008). Comparison of semen parameters in samples collected by masturbation at a clinic and at home Fertility and Sterility (Journal of American Society for Reproductive Medicine) 89(6): pp 1718–1722; DOI: 10.1016/j.fertnstert.2007.05.044
Leitzmann, M.F.; Platz, E.A.; Stampfer, M.J.; Willett, W.C. and Giovannucci, E. (2004). Ejaculation Frequency and Subsequent Risk of Prostate Cancer JAMA 291(13): pp 1578-1586; DOI: 10.1001/jama.291.13.1578
Ley, D.J. (2014). The Myth of Sex Addiction. Rowman & Littlefield. p. 12. ISBN 978-1-4422-1305-0.
Pavličev, M., and Wagner, G. (2016). The Evolutionary Origin of Female Orgasm Journal of Experimental Zoology 326(6): pp 326–337; DOI: 10.1002/jez.b.22690
There are no relevant researches that links the two things. Moreover, some specialists say that masturbation can prevent, or at least decrease, the risk of develop a prostate cancer. Keep in mind that these are new studies, so data are too few to create a rule. With the informations that researchers obtain, they can say that the risk is reduced (1). But as I stated before, there are not enough data to confirm this (2).
In addition to this, the spermatogenic cycle is different from the female one: female have a fixed number of egg follicles, while men production of sperm is cycle (Spermatogenesys take 64 days, and it's continuous). So, if sperm cells are not used for fertilize egg cells, they will die and after short time the cycle of production start again (please, keep in mind that the process is way more difficult and specific; this is a overview to explain what the user asked).
So, no. Masturbation doesn't decrease your fertility and if studies are confirmed, it may prevent from prostate cancer.