I accidentally pressed a button on the remote control while it was pointing towards my eyes. Could that cause any problems?
No. The infrared light is not intense enough. If it was, the controller would be labeled with a warning sticker.
I've been criticized for giving a short argument. But it was a choice between recommending a method to measure the IR intensity or giving a credible argument from a design perspective.
There are just logical barriers that make a dangerous intensity highly improbable; legislation and power efficiency are in 99.99% the prime reasons why it should be safe.
Even looking directly into a 1 mW infrared laser beam would be safe. This article reports about the results of an experiment where volunteers looked into an infrared laser beam. They were able to perceived the infrared light as if they were looking at light with half the wavelength. This is caused by single photoreceptors absorbing two infrared photons at once, which then leads to the same result as when it would absorb a single photon with half the wavelength. Needless to say, the required intensity of the infrared light needed for this (safe to perform) experiment is massively larger than the infrared light emitted by your remote control.