if you dont want to destroy the stocker and possibly the snout Get the puller tool.. rent it or something.

Do you have a scanner? to scan for knock, O2 values, MAF freq, ect?

I dont suggest going to a smaller pully unless you can scan for the above. most cars can get away with your mods and run a 3.4 or 3.5 but they have too much knock to say its safe. I would add a 3in downpipe if you havent and get a AFC2.0 at the minimum to help richen up the areas you need it.
cause you will need it.
also if you have a lot of miles i would replace the lower motor mounts and pop in some poly mounts up top to help get rid of any false knock from the motor moving.