There are a lot of games that work for 2+ players, but I want to focus on 2 player only games.
Abstract games as a whole are typically two players and provide a great challenge. Think of games like Chess, Checkers, Go, Backgammon, etc. I don't know enough about abstract games, nor do I play them often enough, to recommend them. Still, they are worth a mention in any list of 2 player games.
Lost Cities is a card game where players are going on adventures. There are 5 different adventures each player can embark upon. Players do so by playing cards in ascending order from 1-10. If they skip a value, they can never go back. For each adventure you start on, you need to get at least 20 points, else you'll have a negative score. Players typically play 3 rounds, and the highest combined score wins the game. I've only played it once, but it was a fun game. It's really easy to learn, and it only takes about 20 minutes to play 3 rounds.
1960: The Making of the President
Yes, you guessed it, a political game all about the 1960 US Presidential election. Without getting political, 1960 was a landmark election in the US. This game captures a lot of the back and forth of the race, and it's almost always close. Players use cards, which represent different moments in the campaigns, for one of 3 things: action points, card text, the debates. The game is somewhat long, typically 90-120 minutes, but it is a rich play experience.
No, this has nothing to do with the books or movies. The game encompasses the entirety of the cold war. One player is attempting to spread communism, while the other is trying to spread democracy. This is designed by the same person who designed 1960. I've only played this one time, but it's a very deep game. Expect to spend a bit learning the rules, and then around 2 hours playing. The game is well worth it, but it has a steep learning curve.
Memoir '44 (or anything in the Commands and Colors System)
Memoir '44 is a simplified take on some of the major engagements of WWII. Players primarily use infantry and tank units to accomplish various objects. Most engagements are a matter of eliminating the other side's units, but some involve controlling locations. Players use command cards to issue orders, and dice to resolve combat. The system is fairly easy, at least before expansions, and offers good variety with a plethora of scenarios.
Mr. Jack is a neat deduction game with one person taking on the role of Jack the Ripper, while the other player is the inspector hunting Jack down. Jack will secretly be one of 8 characters on the board. The inspector is trying to narrow down the possibilities and eventually make an accusation, while Jack is trying to escape as quickly as possible, or last 8 turns. I really like the mechanics of Mr. Jack. The deduction aspect is interesting and challenging. My problem with it is a lack of replayability. Since it's a shorter game, I want to be able to play it more often, but it just doesn't lend itself to that. Because every character is used in every game, the only real change is which role Jack takes, and then which side you're playing. If you're looking for a neat 2 player game to add to your collection for those rare times you have only 2, Mr. Jack would work well, but if you play a lot with only 2, then Mr. Jack wouldn't be the first game I'd buy.
1960: ~ Looks to be Out of Stock, copies exist in the BGG marketplace for around $40.
From a pure pricing standpoint, Lost Cities wins out. I think Lost Cities has the biggest potential to be played frequently, due to the short play time. It's easy to set up, and play a round while you're waiting for someone else to show up. It also works as a couples game for something quick to do before dinner or to wind down after a long day of work.
My pick for a 2 player only game to add to a collection is Lost Cities. I liked it in theory before I ever played it, and playing it only made me like it more.
I'm by no means an expert on 2 player games, I just don't play them that often. That being said, check out http://gameswithtwo.blogspot.com for more on two player games. They have a lot of good suggestions, and a really interesting series on Risk Legacy.