Requirements vary by state, but generally require that you be physically present in the state for a specific duration (e.g., 183 days in Virginia) before you can be considered a resident. There are exceptions for temporary absences for education or military service, as well as members of Congress. You may be able to accelerate the process by filing a declaration of domicile.
However, even if you are not a New York resident, you may be required to file a New York income tax return because you received income from New York sources and for services you perform in the state. See page 5 of www.tax.state.ny.us/pdf/2008/inc/it203i_2008.pdf. So if you move to Florida you may have to pay taxes in both states, although generally you can reduce your tax burden in each state by the amount paid to another state. This means you will still end up paying the higher New York taxes.
Of course, if you work for an Internet firm and move permanently to another state and no longer maintain a New York residence, it might be possible to work remotely from your new home and avoid the higher income taxes.
Rules are different for state residency for in-state tuition purposes, which generally require a longer period of physical presence in the state (1-2 years). See www.finaid.org/otheraid/stateresidency.phtml for details.