I can listen to a song with a good rhythm and bad lyrics (as long as they're not too obtruse), but I can't listen to a song with bad rhythm and good lyrics.
I do think lyrics are very important, though; I can really enjoy some creative songwriting.
