Goalkeeper Farouk Ben Mustapha was the hero, coming on for the shootout and saving Caleb Ekuban’s penalty.
Tunisia had been seconds away from winning in 90 minutes, but substitute Rami Bedoui headed into his own net in injury time, with his first touch.
They had led in Ismailia through Taha Yassine Khenissi’s goal from Wajdi Kechrida’s cross.
Leeds striker Ekuban was the only player to fail with his kick in the shootout, with Ferjani Sassi scoring the winning kick.
Tunisia now face Madagascar, ranked 108th in the world, on Thursday, despite not having won a game in 90 minutes. The Eagles of Carthage drew all three group games before their penalty shootout win over Ghana.
The Black Stars dominated the opening stages of the game, and Nuhu Kasim’s header hit the outside of the post.
They had a first-half goal controversially disallowed. Thomas Partey fed Jordan Ayew, who crossed for his brother Andre to score a backheel.
The referee ruled the goal out for a Partey handball, with VAR not coming in until the quarter-finals, but the Atletico midfielder controlled the ball with his chest.
Tunisia took control when Wahbi Khazri, who was an injury doubt, came on in the 68th minute. Moments later his corner was headed on to the bar by Khenissi.
The breakthrough came as Khazri backheeled the ball into the path of Kechrida, whose cross was swept home via the post by Khenissi.
Ghana’s Mubarak Wakaso had a 30-yard shot tipped on to the post and then had a hand in the equaliser.
Defender Bedoui came on for Khenissi in the last minute to help them see the game out – but headed Wakaso’s free-kick into his own net with his first involvement.
Both sides had chances in extra time, but failed to take them, so the game went to penalties.
BBC World Service’s John Bennett in the Ismailia Stadium
For the first time since 2008 Ghana have failed to make it to at least the semi-finals.
And for the first time in Cup of Nations history Tunisia have managed to beat them.
More penalty heartbreak for Ghana who have suffered so much in shootouts. Think back to 2010 at the World Cup and 2015 in Equatorial Guinea.
Jordan Ayew could have won it in extra time too. The in-form striker had a great chance.
Tunisia go through and like fellow quarter-finalists Benin they are yet to win a match at this tournament.