It's probably more important to use performers as a starting point. You could end up with a great piece of music and a crappy performance...and that will most likely discourage you. I'm sure you can look up all the big, famous classical pieces yourself...Maybe this will lead you in a better direction:
Avoid Karajan (German conductor) like the plague. His vintage stuff is good...but he tends to make music sound inaccessible and sluggish.
Avoid Charlotte Church (Welsh opera "singer") like the plague. If you like Church's singing, you don't really like music. Same with that Celtic garbage.
Definitely look up Fritz Reiner (Hungarian conductor): get anything from the RCA/BMG "Living Stereo" series.
Reiner notables include:
Bartok's
Concerto for Orchestra...etc
Dvorak's
9th Symphony
Rimsky-Korsakov's
Scheherazade
Charles Munch is another great (French) conductor...if you can find his Walter Piston cycle, LET ME KNOW where you got it!
Glenn Gould (piano):
It's good to have J. S. Bach's
Goldberg Variations...both 1955 and 1982 recordings are good. You can get them both together on Sony's
"A State of Wonder" release. I know...what a dumb title! It's a quote from Gould himself. Seriously though, this 3CD set is a great bargain at a little over ten bucks...well, great considering Sony's bait-and-switch tactics with its once "bargain" priced Glenn Gould series on CD. There's also a DVD you can get of the 1982 recording session (by Bruno Monsaingeon, GG's friend). Amazing to watch!
Look up BIS. It's a Swedish record label distributed by Qualiton Imports LTD in NYC. Basically, northern European music rocks (Kokkonen, Aho, Holmboe's symphonic cycle) The Martinu (Czech) symphonic cycle on BIS is killer (Neeme Jarvi conducts...he's great with the more obscure symphonic stuff, but he kind of spreads himself thin. He's made, like, millions of recordings.)
I got into classical music because of Arnold Schoenberg...more specifically, the chamber symphonies and other orchestral stuff. So you don't necessarily have to start out with Tchaikovsky and Mozart.
Oh, the Lasalle Quartet released a great boxed set of Schoenberg's, Webern's, and Berg's muic (2nd Viennese Schule).
Mahler is important to have...
Simon Rattle and Klemperer (much doomier) do great 2nds
In my opinion, George Szell does the best 6th
The Fifth, of course...Giuseppe Sinopoli, Leonard Bernstein, Solti.
The ninth, Sir John Barbirolli
If you can find E. Mravinsky's recording of Hindemith's
Harmonie der Welt and Honegger's 3rd symphony (on Melodiya/BMG), do not pass it up! It's metal.
Violinists: David Oistrakh, N. Milstein, Bartok's friend: Joseph Szigeti (on older recordings only), Mischa Elman (mostly older recordings)...
Pianists: Sviatoslav Richter, Emil Gilels, Martha Argerich, M. Pollini, A. Michelangeli,
Ivan Moravec...
Cellists: Janos Starker, Jacqueline Du Pré, Mstislav Rostropovich...
Horn: Dennis Brain
Almost forgot the best part, conductors: Fritz Reiner, Charles Munch, Leopold Stokowski, Otto Klemperer, Bruno Walter, Wilhelm Furtwangler, Sergiu Celibidache (hated recording, but there ARE some out there), Evgeny Mravinsky, Sir Thomas Beecham, Sir John Barbirolli, Serge Koussevitzky...and countless others.
Mercury Living Presence recordings are amazing if you can handle the "Presence" of the sound.
If you have a library nearby that lends CDs...seriously, that's the best place to start. Just go nuts.