While you are in an app, if you hit the windows button the app doesn't close but is tombstoned (stay in the background). Most apps when running in background doesn't consume any processing power or resources although some does. To close the apps that are tombstoned follow this method.
Tap and hold the back button. This will show you the App Switcher with the recent apps that have been tombstoned. Select any App and press the back button to close it down.
For most apps you don't have to bother closing them down because the OS suspends all threads related to the app and only store the last state of the app in the memory. Only the event listeners relating to the app are still running but this is not a concern because they don't strain the system significantly and thousands of them can be active at one time. But some apps like Nokia Drive can drain your system. On Nokia website they recommend blocking such apps from running in the background by selecting
Settings → Applications → Background Tasks. Then tap the app you want to block, and tap block.
Especially Nokia Lumia 520, 620, 720, 810, 820, 822 and 920 users; please check that Nokia Drive+ Beta app is blocked (set to off) in background tasks.
If you exit an app by pressing back button it will be closed, not tombstoned, but if you exit in any other way it is tombstoned. This means at a later point you can resume it in the last state you left it easily from the app switcher or by selecting it from the menu.
At any point if you want to close all background apps (not that you need to bother, but I like to check that everything is closed before I leave it idle) you can do by the tap and hold back button method.