Had a similar issue with a friend's car. Turned out to be the wastegate actuator was funky. Instead of replacing it we were able to use a cheap mechanical boost controller in line with it's vacuum line to delay when it kicks in (since it was kicking in too early, just like your symptoms).
The loss being different depending on the gear might be explained by different engine load and/or manifold pressure (and flow).
This isn't the smartest thing to do, but you could disconnect the line so that the gate stays shut, and see if you have a loss... while being extremely careful not to overboost (not familiar with how easy that engine can overboost but I'm guessing it wouldn't handle a large spike very well).