Don't keep it pulled back. I fell asleep with it retracted, when I woke up it was swollen and painful. It's happened twice.
I'd say it's better to have the foreskin right where it's supposed to be.