The plane, a smaller trainer aircraft, had just landed in the besieged city of Misrata when it was attacked, they say.
It is the first incident of its kind since enforcement of the zone began.
Dozens of coalition missiles have already hit military bases, with the aim of ending Col Muammar Gaddafi's ability to launch air attacks.
UK officials said on Wednesday that Libya's air force no longer existed as a fighting force.
Single engine Coalition forces have pounded Libyan targets for a fifth consecutive night.
Initial reports of the French action said the Libyan plane, a G-2/Galeb with a single engine, was in the air when it was hit.
But French military spokesman Col Thierry Burkhard later said the plane had just landed when the attack took place.
The French jet, a Rafale, fired an air-to-ground missile, other reports said.
Earlier, the French military said their planes had hit an air base about 250km (155 miles) south of the Libyan coastline, but did not give any further information on the location of the target or the damage.
Western military planes were also said to have hit the town of Sebha in southern Libya, according to residents and media reports.
Fresh fighting has meanwhile been reported in Misrata, scene of a bitter battle for control which has lasted for many days.
One doctor quoted by the AFP news agency said pro-Gaddafi forces had killed more than 100 people and injured 1,300 injured in the past week.
Misrata resident Muhammad told the BBC many large explosions were heard overnight in the city.
"Even now, we continue to hear the aeroplanes circling the air above Misrata," he said.
"Gaddafi's forces have occupied the main street - there are snipers all along the rooftops of that street. They are firing indiscriminately into the main street and the back streets.
"But the heavy artillery and shelling has stopped since yesterday [Wednesday]. In that sense, we are in a much better position."
Further east in the strategically important city of Ajdabiya, residents described shelling, gunfire and houses on fire. One report said rebels were moving closer to the city but remained outgunned by pro-Gaddafi forces.
In the main eastern city of Benghazi, rebel spokesman Mustafa Gheriani told the BBC that 17,000 fighters had set out from the city to join the battle to the west.
Although he admitted that the rebel forces were on a "learning curve", he insisted that they all knew how to operate their weapons and were committed.
"We will slowly advance," he said. "[Gaddafi's forces] have no reason to fight, no cause, while we do.
"We have given so much blood and we're willing to give some more if we have to."
Nato debate
Nato members have been holding talks about assuming responsibility for the no-fly zone over Libya, so far without agreement.
The US is keen to relinquish its current control, but there are disagreements about who should take on the role, and what the exact terms of engagement would be.
Turkey wants any Nato-led action to focus directly on enforcing the no-fly zone and arms embargo, rather than allowing any continuing strikes against ground forces.
The Turkish parliament has backed a government decision to join the blockade. Four frigates, a submarine and a support vessel are being provided by Turkey.
In a separate development, African Union chairman Jean Ping has invited members of the Libyan government and opposition to talks in Ethiopia.
Representatives of the European Union, the UN Security Council and neighbouring countries have also been invited to the talks, scheduled to be held in Addis Ababa on Friday
