Last week O'Neill received 70 votes to Abal's 24 in a parliamentary vote for a new leader.
Before the vote Sir Michael Somare was still recognised as the country's prime minister, although he has undergone surgeries for his heart, lungs and kidneys since April and remains on medical leave.
Abal's argument that the position of prime minister was not vacant was dismissed by Judge Gavara-Nanu on Friday, according to the AAP.
The judge reportedly said the role was no longer vacant when Governor-General Michael Oglio swore in O'Neill after the vote earlier last week.
Abal's lawyer also argued that the speaker, Jeffery Nape, did not follow parliamentary procedures to declare that the position of prime minister was vacant, but the judge reportedly questioned why Abal participated in the vote instead of walking out.
Both Abal and O'Neill used lawyers to argue their case and were not present in the court room.
While it is unclear if there will be a further appeal to the Supreme Court, this court has already rejected an application by former Attorney-General Sir Arnold Amet to declare last week's election of O'Neill as unconstitutional.