The Information Commissioner's Office said today it had been "badly let down" by parliament, the courts and newspapers in its attempt to stop the "flourishing" trade in illegally obtained confidential personal information.
The information commissioner, Christopher Graham, who took over the role at the end of June, claimed that custodial sentences could end the practice "at a stroke".
He said the office had tried to "sound the alarm" about the scale of the problem. "We were let down by the courts, who didn't seem to be interested in levying even the pathetic fines they had at their disposal; we were rather let down by parliament in the end, with no legislation; and we were let down by the newspaper groups, which didn't take it seriously," he said.
Graham was giving evidence to the Commons media select committee, which is investigating phone-hacking and the use of illegal subterfuge by newspapers as part of an inquiry into press standards, privacy and libel.
MPs heard that police suspected mobile phones belonging to Prince William and Prince Harry had been tapped into by Clive Goodman, the former News of the World royal editor, and private investigator Glenn Mulcaire, who were both jailed for hacking into phones of royal aides.
The committee reopened that inquiry after the Guardian revealed in July that the News of the World paid three people, including the Professional Footballers' Association chief executive, Gordon Taylor, about £1m in costs and damages to settle civil actions bought against its owner News International after their phones were hacked into by reporters on the paper.
The commissioner said the sale of personal data was not just an issue for newspapers, describing it as a "wider societal problem", which also involved jury tampering and witness intimidation.
He also criticised the government for failing to introduce jail terms as part of the Data Protection Act. He claimed intensive lobbying from the newspaper industry had prompted a change in policy.
Graham said prison terms would be "the big stick in the cupboard. If people have to factor that risk in, it's a business that is not worth being in." He described the scale of the illegal accessing of personal information as "an appalling situation".
The Information Commissioner's Office published a report in 2006 showing that 305 journalists used private investigators.
Asked why no action was taken against them, Graham said there were more than 17,000 invoices or purchase orders and the office would have to prove in each case they acted illegally.
"You would have to go through [each individual case] forensically to achieve the standard of proof required in a court of law, attach each to a story … and work out if our lawyers could get the better of their [newspaper] lawyers," Graham said. "The appropriate response was to make a big issue of it … tackling it at source and at the top level by legislation."
John Yates, assistant commissioner of the Metropolitan police, who also gave evidence, defended the Met's investigation into the News of the World during the phone-hacking scandal – and explained why it did not widen its inquiry into other reporters at the tabloid.
Yates, and detective chief superintendent Philip Williams, told the committee that other reporters from the News of the World were not pursued because there was no evidence of illegal activity.
Williams was asked if he suspected that journalists had hacked into the mobile phones of princes William and Harry. He replied: "Yes, I think they may well have done."
A News of the World spokeswoman last night said: "News of the World knows of no evidence to support this allegation and it was never part of or disclosed in the police investigation."
The Met officers dismissed an email uncovered by the Guardian, which was sent by a junior reporter to Mulcaire referring to a "transcript for Neville". The email was taken to refer to News of the World chief reporter Neville Thurlbeck.
"It wasn't a viable live inquiry and I have to say [this decision] has been supported by leading counsel subsequently," Yates said. He said police did not have any other evidence of illegality to put to other News of the World reporters. Yates said the police investigated the paper and asked it to disclose all relevant information. A letter was sent to News International solicitors asking for information about "all persons involved" in potential phone-hacking.
But Yates conceded that Mulcaire was in contact with other journalists: "I am quite happy to concede that he did contact other journalists, that is his job. What we concentrate on is evidence."
Tom Watson, MP for West Bromwich East, asked Yates: "Doesn't it look suspicious that Mulcaire and Goodman may have been tapping royal princes' phones and committed a serious crime that undermined their own reputations and that of their employers and their employers give them a undisclosed pay-off and no one has gone back to them and asked what that financial arrangement is about?"
Yates replied: "It is not our business."
• This article was amended on 3 September 2009. The original referred to the Information Commission. This has been corrected.
