First of all, know that you regularly consume protein and have been all of your life. If you did not, you would be in quite terrible health.1 That being said, IGF-1 is involved in bone growth, muscle growth, connective tissue growth, motor neuron growth, and so on. It's also produced within the body.2 Your body uses it for growth and healing, and this has nothing to do with whether or not you drink protein shakes or work out.
The human brain has an incredible capacity to make connections where they're not obvious; one could argue this is the source of human intelligence, but sometimes it gets these connections wrong. I'm telling you definitively that IGF-1 was not the cause of your breakout, and neither was protein. Most likely you just so happened to break out at the same time you were drinking protein shakes. Feel perfectly comfortable drinking them now.
Furthermore, your body relies on the amino acids in protein to rebuild itself, and keep you alive. The fact that you're breathing right now means that you consume protein regularly. When you work out, you are causing tiny amounts of muscle damage, and the muscle building basically results from the body overcompensating as it heals the muscle.3 This healing relies on these amino acids, which are taken from protein as it is broken down in digestion. You can't heal (and therefore grow) muscle without protein, just as you can't live without protein.
Moral of the story: Drink protein shakes if you like, eat plenty of meat and fish if you don't. It won't cause acne.