Reynolds singled off Yankees reliever Hector Noesi with two outs in the top of the 11th to score Matt Angle and break a 4-4 tie.
Angle reached with one out on another error by Yankees shortstop Eduardo Nunez. Angle stole his second base of the day, waited while Nick Markakis was walked intentionally and raced home on Reynolds' hit.
The Orioles loaded the bases with two outs, but Davis grounded out.
The good news: It was a ground out. A sixth strikeout would have tied the single-game club record held by Sam Horn, and it would have made the trip back to Baltimore tonight feel like it lasted around 10 hours - or the equivalent of a rain delay at Yankee Stadium.
Davis went 0-for-6, struck out five times, stranded seven runners and committed an error.
Jim Johnson, fresh off his three-inning relief stint on Monday, picked up the save after issuing a leadoff walk to Jesus Montero, who homered twice off him during the first game of this series, and giving up a two-out single to Curtis Granderson. Pinch-hitter Eric Chavez grounded into a force, with Robert Andino making another great diving stop.
Andino flipped to Ryan Adams, who was late covering the bag, for the out. It was a very close play, and it was a game-saver.
Pedro Strop recorded his first win as an Oriole with 1 1/3 scoreless innings. He fanned Alex Rodriguez, giving him five strikeouts in 3 1/3 innings since joining the Orioles - all on sliders.
The Orioles' bullpen allowed four hits in six scoreless innings.
The two teams won't see each other again until...tomorrow.