According to Reuters the 11-judge panel used its precedent from dismissing a case against Royal Dutch Petroleum Company in April.
In this ruling the court interpreted that the Alien Tort Statute (enacted in 1789) was only relevant when violations concerned US territory with "sufficient force".
Around 10,000 Bougainville residents were involved with the lawsuit against Rio.
The original lawsuit filed in 2000 alleged the company was involved in destroying villages, razing the rain forest and slicing off a hillside to establish the Panguna open pit mine, in operation between 1972 and 1989.
The lawsuit argued that riverine tailings allegedly destroyed fish stocks and caused locals to have severe medical complications.
The lawsuit also claimed Rio helped the Papua New Guinea government transport troops to the island to curb protests against the mine, which led to an uprising and years of subsequent civil war.
Meanwhile, there is growing support on the island to reopen the Panguna mine to revive the local economy.