You really should not keep account information on your computer. If you are infected with a virus or you misconfigure a file sharing program, that information can be leaked. I have done a little research in the past and have found multiple incidents where a misconfigured file sharing program leaks full PDF tax returns, bank statements and numerous other personal information. A trojan can allow someone to aquire these PDF's also. My suggestion is, if you concerned about keeping everything paperless, look for a online storage medium. Even something like Gmail will work. Just email youself the PDFs. Just make sure to protect your email with a decent password. Also, check with your bank and see if they keep a certain number of statements. For example, I think chase keeps 12 months of statements. If that isnt enough, then DL them and upload them to your storage medium.

Another possibility is using something like Yodlee Moneycenter or Mint to keep track of purchases and expenses. Then if you absolutely need a statement, you can pay a small fee to have it recovered and sent to you from your bank.

Possible information that can be leaked from a statement:
Account number
Personal address
Name
Purchase Information
Pay day

Payday is a rather important to someone trying to steal. With this information, they can time a withdrawal via a online bank or service to maximize their theft. Purchase information allows them to commit shadow fraud by trying to mimic your purchases (Such as buying at X online store to reduce you noticing it). All that information can be used to social engineer their way to full account control.