I've had the best luck using brown or dark glass growlers. Glass is one of the most inert food-storage materials. The effect on taste of beer from different material containers is similar to the effect that drinking water from different material containers has.
Using metal can sometimes impart a flavor to the beer, though medical-grade (316) stainless steel should be as inert as glass, which is why it's used in a lot of brewing equipment. Not all stainless steel is up to this task, though - make sure to check which alloy the containers are made from. Some will rust over time, despite the "stainless" designation! Passivation of the material's inside surface is also helpful - this creates a very thin layer of metal oxide material on metals that will not react with the liquid stored inside. (The growler you linked to is passivated, for example.)
Plastics definitely can impart a flavor into beer or other drinks, and are difficult to keep sanitized / free of scratches and bacteria. I'd avoid this where possible, though you may find it is fine for taste if you drink the beer quickly enough.