The likely answer is somewhere between carbonation pressure and marketing.
It's hard to find numbers for how much caps can handle vs corks but you should notice that most corked beers also come in bottles with very thick glass, this is because the beer inside is at a higher pressure than most other styles.
Most beer styles will fall pretty close to 2.0 or 2.5 volumes of CO2 (just the term, think of the number as a relative baseline), but a lot of Belgian strong beers like Bierre de Garde or Tripel will be carbonated with 3.3+ volumes. Lambics and other sour beers can go up to 4.5 volumes. This is a linear scale so that becomes possibly twice as much pressure as a normal bottle...so yeah, thicker glass.
At such high pressure you might find corks will stay put more reliably than a cap since it has more surface area in contact with the bottle. It also might provide a more airtight seal than a cap for long-term aging.
Those would be technical considerations.
Corking also predated capping so there's also a very traditional feel about a bottle with a cork and a cage, which can fit into the traditional image a lot of breweries market themselves using. I say I'm not sure and it could just be marketing because a lot of Wheat styles like German Wheats, Belgian Wits, etc are fairly highly carbonated (near the 3.3 volumes range) but most of them have no problems being capped.
As to the affect on taste? Nada, apart from hypothetical aging concerns...unless mistreated you should not taste the cork.