They rallied, but then so did Cleveland.
It was the fifth blown save for the O's closer, who had saved 12 in a row since a blown save May 26 at Toronto.
Johnson, seeking his 28th save, issued a four-pitch walk to Micheal Brantley to start the ninth and Jason Giambi doubled. An intentional walk followed and back-to-back ground balls scored runs as the Indians moved ahead. On the second one - off the bat of Drew Stubbs - the O's just missed turning an inning-ending double play that would have kept the game tied.
The O's went to the eighth down 2-1 and facing reliever Joe Smith, who had just come on. J.J. Hardy led off with an infield single. A walk and an Alexi Casilla single loaded the bases with one out. A Nick Markakis fielder's choice grounder tied the game at 2-2 and Manny Machado came up.
Machado ripped a liner off the base of the left-field fence for a long RBI single and a 3-2 lead. In the seventh, Machado hit his major league-leading 36th double to break up a Scott Kazmir no-hitter.
Kazmir left this game under strange circumstances. With his pitch count at just 78, he came out for the last of the eighth, but later walked off the mound with the trainer as Smith came on.
Machado scored in the seventh after reaching on a double down the left-field line. With one out, he went to third on an errant Kamzir pickoff throw. He scored on a Chris Davis sac fly. For Davis, it was his 73rd RBI of the year and his 21st in his last 14 games.
Cleveland had taken a 2-0 lead off Jason Hammel on a two-run homer by Jason Kipnis in the fourth inning.
Hammel gave the Orioles a solid outing after pitching to an ERA of 6.23 over his last four starts. Over seven innings, he gave up four hits and the two runs with two walks and five strikeouts.
Hammel's outing gave the Orioles two straight starts of seven innings. They now have gotten seven or more from their starting pitcher 11 times on the year.
Last year, the Orioles went 29-9 in one-run games. In 2013, they are 11-11.
With their fifth loss in the last six games, the Orioles fall to 43-36 and they'll host Cleveland Thursday night in the finale of this four-game series.