How about establishing automated regression-tests?
All "existing functionality" that should still work is defined in regressiontests. All "new features" need to be evalutated against the specifications/requirements (functional & non-functional requirements) and will end up in the regressiontests.
At a certain point it might be useful to prioritise your regression-test-testcases.
Parts of the source or the programm which seldom throw errors/exceptions wont be tested as in detail as new features or parts of the programm that are not stable.
Hope this helps.
If not - please get more in detail with your question :)
I agree with Peter. If your application is not so big then you can also verify existing functionalities by doing regression testing manually. We are doing the same. That also depends on how often the application is changing means how frequent the updates are being made. If they are less then we can also consider manual regression testing since it will be cost-effective way for small companies having less budget on QA. This is just my opinion. Correct me if I am wrong...