You're probably right about late payment hurting your scores. Even low level late payments (30 and 60 day variety) can hurt your scores especially if you have several of them. The statute of limitations for negative information, most of it anyway, is 7 years. That means that your late payments will remain on your credit files for 7 years from the date they occurred.

Now, that's the bad news...here's the good news...

They will lose negative value over time (organically) if for no other reason than they are aging. So you'll get points back as they get older but you have to stop missing payments.

The take away....

Automating your payments is a good idea especially if you're so busy that you forget to make your payments on time.