The malt is providing the sweetness and the hops are providing the bitterness plus the lingering flavour.
Depending on what type malted grains the beer is brewed will effect its sweetness. This is due to how much fermentatble sugar can be extracted from the grains, any non-fermentable sugars will stay in the beer and provide effect how the beer sweetness.
Hops are used in different stages of the brewing process to provide different functions.
Hops are added at the intial point of the boiling process (which is usually about 60mins) - The hops oils released from the hops at this stage provide the beer with head retention, bitterness and also natural perservative.
Hops are then again added during the final stage of the boiling process (last 10-15mins) - At this point the hops are providing the lingering taste that you are noting, plus any aroma notes.