A lot depends on how big you are (if you want to look proportional), what kind of clothes you wear, and how you're planning on wearing the packer. The 5" (small) can be better for taller guys, broader guys, or guys who wear baggy pants or prefer boxers. The extra small is a good all arounder (I know plenty of bigger/taller/broader guys who use it) but it's particularly good if you're short, small, wear tight pants, and/or wear mostly briefs, boxer briefs, or trunks.
The thing about the extra small is it is very squishable, so as long as you're not wearing it free in boxers, it'll squish flat very easily. I'm very short, so the 5" creates a much bigger bulge for me than the 3 1/2", and the 3 1/2" is also easier to "tuck" or shift around if I'm wearing pants that show it off more than I'd like.
In boxer briefs and briefs, the extra small generally will squish flatter than the small - the small has smaller balls, but because it's longer than the extra small, you have to either point the shaft up, or tuck it under, which creates the same profile problem that the extra small's balls do. (The 5" is in general not *quite* long enough to hang down one's leg, which would minimize that issue).
Everyone's different, though, and I know plenty of people who prefer the 5", especially if they're tall or wear boxers.
Also, the 5" is easier to manage as an STP, if you plan on converting one, because you get better clearance... but if you're a small guy you may have the aforementioned issues packing with it. The 3 1/2" can be converted just as well, but clearance is an issue unless you're very coordinated and have easy to reach anatomy.