LOL my wife did the same thing. Got me a Bread album, a Michael W Smith, and (best of all) a greatest hits double album of the Bee Gees (yes!). Was gonna hook up the player to my Edifier bookshelf speakers that i bought a couple years ago for my PC, but we had a little extra dough so she got me some mid-century lookin Crossley powered speakers.
I've no idea where she got the albums (Christmas secret), but there's several options in my area, a place called Vintage Vinyl, and a hole-in-the-wall on route 66 that supposedly sells albums. With vinyl becoming a craze, I'm sure there's options near you, just gotta turn over some mossy covered rocks & find them.