Generally, based on pharmacokinetics and safety or efficacy.
Drugs are metabolized and eliminated from the body at some rate, with the rate and mechanism depending on the specific drug. Both acetaminophen (Tylenol) and ibuprofen have a half-life of around 2-4 hours, for example, and take about 1 hour after oral administration to reach peak serum levels.
Many drugs have side effects that get worse at higher doses. The key is not just the acute dose but how much of the drug is circulating at any given time.
Looking for repeat dose PK measurements, the first study I came across happened to involve IV administration of acetaminophen in neonates every 6 hours:
(from Palmer, G. M., Atkins, M., Anderson, B. J., Smith, K. R., Culnane, T. J., McNally, C. M., ... & Hunt, R. W. (2008). IV acetaminophen pharmacokinetics in neonates after multiple doses. British journal of anaesthesia, 101(4), 523-530.)
You will see that at this frequency, the peak concentration levels off after a few doses and doesn't continue increasing.
Other drugs are meant to deliver a roughly constant therapeutic effect (this is more relevant for long-term prescription medications, such as drugs to regulate blood pressure, hormonal birth control, etc), and the frequency of dosing is meant to keep the therapeutic effect within an appropriate range (with some balance for convenience). Missing a dose might even lead to a rebound effect and make a symptom worse.
You are right, however, that the exact numbers are a bit fuzzy, both because different people metabolize a given drug at different rates and because there isn't some magic danger time: if a bottle of ibuprofen says every 4-6 hours, realistically there is not much difference if someone took a pill at 3 hours and 58 minutes after their last dose. The numbers printed on a label for an OTC drug are going to be safe for most people and indicate a dosing scheme that has been tested in clinical trials.