One would hope that there aren't too many negatives associated with these, but are there proven differences? Is it the same as a bottle vs. can, or does the (usually) limited time in the container make the material...immaterial?
The biggest impact tends to be the color of the container. Green and clear bottles let in more of the harmful light that spoils the beer. The darker the container, the better.
Plastic also lends itself to being scratched if re-used (which most growlers are). Even powdered cleaners with soft cloths can scratch the plastic (if the cleaner isn't disolved well enough) and make grooves where bacteria can sit and ruin future beers.
While it isn't the only choice: Dark Colored, glass growlers that have an air-tight seal would be the best for longevity sake.
I've had the best luck using brown or dark glass growlers. Glass is one of the most inert food-storage materials. The effect on taste of beer from different material containers is similar to the effect that drinking water from different material containers has.
Using metal can sometimes impart a flavor to the beer, though medical-grade (316) stainless steel should be as inert as glass, which is why it's used in a lot of brewing equipment. Not all stainless steel is up to this task, though - make sure to check which alloy the containers are made from. Some will rust over time, despite the "stainless" designation! Passivation of the material's inside surface is also helpful - this creates a very thin layer of metal oxide material on metals that will not react with the liquid stored inside. (The growler you linked to is passivated, for example.)
Plastics definitely can impart a flavor into beer or other drinks, and are difficult to keep sanitized / free of scratches and bacteria. I'd avoid this where possible, though you may find it is fine for taste if you drink the beer quickly enough.