Mix it Up
I'll admit, I'm a big offender of this because it's more natural for me to hold my camera to take a landscape oriented shot. But it may make for some repetitive scrapbook layouts.
If you find yourself shooting most of your photos in horizontal/landscape mode, why not try changing it up and also shooting in a vertical/portrait orientation?
Another plus is that when you are taking photos of people, a vertical photo allows you to focus in on more of the person and have less distracting background on either side. If you have photo-editing software, you an also change a photo from horizontal to vertical with the cropping tool after the fact.