What made brewers in the 19th century decide to start using bottom-fermenting yeast and cooler fermenting temperatures? Were they trying to solve some specific problem, like making the best of available ingredients or dealing with cooler-than-ideal environments? Or were they just experimenting and trying to create something new?
I think it would be right to say that lagers were discovered rather than invented. Lager yeast is apparently the result of an unlikely mating of ale yeast and a wild yeast from South America. It's ability to ferment at lower temperatures probably led to its establishment in some brewery in some cold part of Bohemia centuries ago.
According to Wikipedia, lagering became more commercially viable with the appearance of refrigeration systems. But White & Zainasheff describe why this was desired. After Pasteur showed that yeast are responsible for fermentation, the Carlsberg brewery isolated S. carlsbergensis, now known as S. pastorianus, for commercial use. This is a lager yeast, and since many ales were brewed with mixed (S cerevisiae plus wild yeasts, bacteria) cultures, it created the perception was that lager, as a style, was cleaner than ale, and the improved shelf life made it a more attractive product to produce. And so, lagers took over the beer world after that, despite so many small breweries that make high quality ales.