Shehbaz Sharif (centre) led an opposition alliance to vote Imran Khan out
The opposition coalition leader - who had worked to depose Mr Khan - won the support of a majority in parliament.
Shehbaz Sharif will now form a new government that can remain in place until elections are due in August 2023.
Mr Khan, 69, was voted out after days of high political drama.
He had attempted to block a previous attempt to bring a no-confidence motion against him by dissolving parliament and calling for a snap election.
However, the country's Supreme Court upheld an opposition petition that his actions were unconstitutional and ordered the no-confidence vote to go ahead.
On Monday Mr Sharif was elected in parliament unopposed and will be Pakistan's 23rd prime minister.
Rival candidate Shah Mahmood Qureshi had announced the Pakistan Tehreek-e-Insaf party led by Mr Khan was boycotting the parliamentary vote and staged a walkout. The former PM and most of his party's MPs resigned from their seats ahead of the vote.
Shehbaz Sharif submitted his candidacy for the top post on Sunday, before winning 174 votes in parliament the following day - two more than a simple majority of lawmakers.
In a speech after the vote, Mr Sharif accused Imran Khan's outgoing government of mismanaging the economy and said putting it back on track would be a huge challenge for his new government.
Imran Khan's supporters held large rallies across Pakistan's cities on Sunday night demanding his return
However, Imran Khan, whose supporters protested across major cities after he was dismissed, is still blaming a "foreign conspiracy" for his removal as prime minister.
He has claimed that his rivals colluded with the US to remove him because of his foreign policy stance on Afghanistan, Russia and China. Washington has strongly denied the allegations and they have been ridiculed by his opponents.
But the former international cricketer's claims fed on anti-American sentiment held by many of his supporters.
Mr Khan's opponents saw an opportunity to strike after months of discontent over his management of the economy, and a breakdown in his relations with Pakistan's powerful military.
Mr Sharif banded together with other opposition lawmakers to muster the numbers to vote Mr Khan out.
On 3 April, they challenged Mr Khan outright in parliament - launching a motion for a vote of no confidence in his leadership.
But the deputy speaker of the parliament - a political ally of Mr Khan's - blocked the motion, claiming it was influenced by foreign powers. His government then swiftly dissolved parliament and called for a snap election.
The furious opposition immediately challenged the legality of government actions in the Supreme Court. After a four-day hearing, it ruled that Mr Khan's government had acted illegally, in breach of the constitution.
Even then, Mr Khan and his party tried to prevent the vote in parliament for hours - with officials quitting, lawmakers filibustering and other disruptions.
But eventually, in the early hours of Sunday, his opponents secured his removal, with 174 lawmakers voting against him in the 342-member house.
-- Courtesy of BBC News