Goals by midfielder Robert Andrich, his second of the night, and winger Diaby in a five-minute burst late in the game swung the result for hosts Leverkusen.
"We were shocked to be 2-1 down and needed a bit of time to regain control," admitted Leverkusen's double goal-scorer Andrich.
"We had to remain patient and deserved to get the result."
While the Germans reach the knock-out stages as Group G winners, the away defeat means Celtic will finish in third spot after second-placed Real Betis won 2-0 at Ferencvaros Budapest.
"(Conceding) two late goals was disappointing and we came up short," lamented Celtic's Australian coach Ange Postecoglou.
"We lost some composure at the end and didn't make good decisions."
Andrich's early header for Leverkusen was cancelled out when Josip Juranovic equalised with a first-half penalty before Jota was left unmarked in the area to fire Celtic ahead for the first time just after the break.
Leverkusen midfielder Andrich equalised for the hosts eight minutes from time before Diaby's volley sealed the win.
"It's not easy to finish top of the group with a game left. At the end we were rewarded for our hard work," said Leverkusen coach Gerardo Seoane.
The result preserves Leverkusen's record of having never lost at home in the Europa League group stages, while Celtic must wait for a first win on German soil in European competition.
Leverkusen struck early when Andrich outleapt his marker Stephen Welsh at a corner and powered his header past Celtic's former England goalkeeper Joe Hart.
Celtic started applying pressure
A great chance went begging when Japan striker Kyogo Furuhashi got in behind the defence and his speculative shot to the far post flew just wide of Jota.
The Scottish giants drew level five minutes from the break when Juranovic chipped in from the penalty spot after home goalkeeper Lukas Hradecky fouled Furuhashi went he went to punch clear a cross.
Leverkusen made a strong finish to the first-half as Florian Wirtz and Moussa Diaby both hit the woodwork, but it stayed 1-1 at the break.
Hart came to Celtic's rescue early in the second-half when he scrambled back to deny Jeremie Frimpong moments after blocking a Diaby shot.
At the other end, Jota, who is on loan from Benfica, fired Celtic in front when Furuhashi spotted the Portuguese unmarked at the edge of the area to blast his shot past Hradecky.
Leverkusen roared back as Andrich beat Hart for the second time five minutes before Diaby volleyed in the winner after being picked out unmarked by a Florian Wirtz pass.