Using any medication, including ibuprofen, in doses higher than recommended, is very unwise. There is a reason that patient information leaflets exist, and that health professionals insist that the patients read and understand these leaflets (and ask questions when necessary) prior to consumption.
First of all, ibuprofen is a non-steroid anti-inflamatory drug (NSAID). If the cause of your toothache is tooth decay (cavities) it will treat the pain, i.e the symptom, but not the underlying cause of your toothache. There might be another reason for your toothache, but nonetheless: if you are experiencing unbearable pain this is definitely a reason to see a doctor, or in this case a dentist.
Recommended daily dose of ibuprofen for an adult is 1.2g to 1.8g in divided doses which means that you could take up to 1800 mg during a day - that is during a 24 hour period, but definitely not all at once. And yes, taking them 30 minutes apart is pretty much the same as at once, given ibuprofen's kinetics. These recommendations are based on an average adult, with a healthy liver and kidneys.
Sometimes this dose may be increased up to 2.4g (UK) or even 3.2g (USA) and this would be the maximal daily dose. (Martindale) However, this is not a common practice for a toothache.
There are side effects associated with the use of ibuprofen, and many of them are dose-dependent.
From NHS:
Ibuprofen can cause a number of side effects.
For this reason, take the lowest possible dose of ibuprofen for the shortest possible time needed to control your symptoms. The same source lists some of the possible side-effects.
Common side effects include:
- nausea
- vomiting
- diarrhoea
- indigestion (dyspepsia)
- abdominal pain
Some less common:
- headache
- dizziness
- fluid retention (bloating)
- raised blood pressure
- gastritis (inflammation of the stomach)
- a stomach ulcer
- allergic reactions – such as a rash
- worsening of asthma symptoms by causing bronchospasm (narrowing of the airways)
- kidney failure
In high doses there are additional risks, such as those for the cardiovascular system (heart and blood vessels):
Data from meta-analyses and epidemiological studies indicate that there is an increased risk of cardiovascular events (such as myocardial infarction or stroke) associated with the use of high-dose ibuprofen (at or above 2,400 mg per day).1-4
The risk with high-dose ibuprofen is similar to the risk seen with some other non-steroidal anti-inflammatory drugs (NSAIDs), including COX-2 inhibitors and diclofenac.
(from: European Medicines Agency).
Ibuprofen might interact with and increase the toxicity of some other concomitantly used medication, such as: aspirin, ciclosporin, fluconasole, voriconasole, colestiramine, mifepristone, tacrolimus, methotrexate, lithium, zidovudine, warfarin, ciprofibrate, baclofen...
If you are taking any other medication with ibuprofen, the best course of action is to consult with a healthcare professional first.
Finally, although ibuprofen is an OTC, higher potency preparations, such as those of 600mg per dose are actually prescription only in many countries.
A side note: I don't even know how to comment on your: "let's take a lot of this medication and see what happens" approach. Medicines are not candy. If you behave so recklessly about your own health, who else is supposed to take care of it for you?
References:
Martindale: The Complete Drug Reference, 34th edition, 2005, The Pharmaceutical Press
NHS: Side effects of ibuprofen
EMeA: Updated advice on use of high‑dose ibuprofen