Come on, those loads don't even come close to normal working pressure. Of course the brass isn't expanding enough to seal! The primers are backed out because pressure never gets high enough - and because your cases are too short (head-shoulder) to fit the chamber, giving the same result as if the rifle had excess headspace (which itprobably doesn't).
You need to get pressure a lot closer to normal for your cases to expand fully, not to mention providing sufficient load density for consistent ignition and accuracy. Your 32.5 grain load gives only about 70% load density. If you want to shoot reduced-velocity loads with those pricey bullets, use a faster powder designed for good ignition and combustion at low density - one that will also develop enough pressure to expand cases.
Sample QuickLoad estimates:
Cartridge: 6.5 x 55 Swedish Mauser & Norwegian Krag (Pmax 3800 bar, 55114 psi)
Bullet: .264, 140, Sierra SPBT 1730
Cartridge O.A.L.: 3.150"
Barrel Length: 24.0"
Powder: Varget
Step Fill. Charge Vel. Energy Pmax
% % Grains fps ft.lbs psi
-25.0 63 30.00 2089 1357 26482
-22.5 65 31.00 2148 1434 28389
-20.0 67 32.00 2206 1513 30414
-17.5 69 33.00 2264 1594 32560
-15.0 71 34.00 2321 1675 34835
-12.5 73 35.00 2378 1758 37245
-10.0 75 36.00 2434 1842 39797
-07.5 77 37.00 2490 1927 42500
-05.0 80 38.00 2545 2013 45360
-02.5 82 39.00 2599 2100 48388
Pressure figures are in psi (CIP piezo) - not CUP.