Welfare reform: will Iain Duncan Smith's benefits revolution happen?

The Jobcentre Plus office at 75 Union Street, Oldham, is at the centre of a welfare revolution – but on a warm summer's day you wouldn't know it. On the pavement outside, almost no one going in or out seems, for more than a year now, to have been aware that their town has been pioneering a great experiment.

It is an experiment that the work and pensions secretary Iain Duncan Smith – a survivor of the recent cabinet reshuffle – likens to William Wilberforce's mission to free people from slavery. The project with the utopian-sounding title of universal credit (UC) will, Duncan Smith promises, "put hope back where it has gone" and "give people from chaotic lives security through hard work". It will end the "benefits trap" that has meant that those on welfare have too often lacked financial and other incentives to embrace the world of work. It is, in his mind, nothing short of a panacea for some of the worst social ills of our time, reducing crime and the sense of hopelessness.

"Work is about more than just money. It is about what shapes us, lifts our families, delivers security and helps to rebuild our communities," says Duncan Smith. By changing the way benefits are paid, combining six benefits into a single payment and making sure that it will always be financially worthwhile to do more work – Duncan Smith is convinced that UC will make the search for more work as creative, dynamic and rewarding as a satisfying job should be. In a major speech on welfare last week, the minister was warming to his theme yet again, railing against a "dysfunctional welfare system that often trapped those it was supposed to help in cycles of worklessness and dependency". Practical progress in the former Conservative party leader's efforts to become the new Wilberforce, is, however, proving painfully slow.

Overlooked by the Pennines, Oldham was once one of the most thriving cotton and textile spinning towns in the world. Its last mill only closed in 1998 and its economy is still adapting. Those using the Jobcentre look blank when asked if they have heard of the big new idea to help them, and whether they were aware that Oldham had been selected as one of UC's biggest trial areas in April last year. Most haven't a clue what UC is.

Until, that is, two smartly dressed young men, both university graduates – one from Lancaster; the other from the University of Central Lancashire – answer in unison. "Yes, we're on it." Both are new claimants, single, have no children, live with their parents and have good qualifications. As guinea pigs go, they are perfect because their cases are simple for IT systems to handle and they will be easy to place in jobs. "I have had interviews already and reckon I'll be in work in a few weeks," said one. "I don't think I'll be on benefits for long either, fingers crossed," says the other.

They don't want to give their names because they don't want people to know they are reliant, even temporarily, on the state. They don't think UC is particularly efficient – the computers keep going down and they have turned up for work coaching sessions that staff have no record of – but they reckon it will do its job for them nonetheless. Half an hour later Martin Need, who has been unemployed for two years and says he is losing his confidence because he can't find a job, emerges with forms in hand. He says he is not on UC, but remembers that his 23-year-old son, who lives with his ex-wife, is. "Oh yes, I know about it because of Jack, but I don't think most people know what it is," he says.

The implementation of UC has already been massively delayed and scaled back. It is now being piloted among a limited cohort of the easiest new cases in just 34 Jobcentres across the country. At the last count, only around 12,000 people were on UC, a tiny proportion of the numbers originally intended to be claiming it by now. At the end of July offices dealing with UC began to take on couples, calculating their claims together, in another small lurch forwards.

The official word is that all is well. At the Jobcentre Plus office in Hammersmith, west London, journalists can meet staff who are trained to deal with UC clients and, it would seem, trained to impress the media, too. They extol UC's virtues from first-hand experience, under the beady eyes of officials from the Department for Work and Pensions. They explain how under UC people in part-time work who are offered more employment that would take them over 16 hours no longer have to come off job-seekers' allowance (JSA) as a result and go through the burdensome process of applying for tax credits. UC, they say, simply adjusts their total payment automatically and ensures that for every extra pound they earn a maximum of only 65p will be lost from their benefit payments.

Vandna Patel, who used to deal with JSA cases, is one of the new team of "work coaches" for UC at Hammersmith. She is wholly positive. Instead of spending her day helping people to fill out long forms as they come off and back on JSA and dealing with queries about delays in payments, as she did before, now she can concentrate on coaching people about how to find extra work and become better off. "It is great," says Patel. "It is so much better. Now I can focus on the job aspect and the claimants' needs."

All UC claimants have signed contracts saying what they must do to look for work. It is all creative and dynamic now. She had one client who was doing 16 hours of work a week provided by various agencies and then got offered the chance to increase it to 20. "He was so relieved to hear that the money would just be adjusted automatically and that he would not have to come off JSA," she said. Jim Hamilton, who is involved with the technology side of UC at Hammersmith, said the team had experienced no serious computer or other technical glitches. "If there is a problem, you fix it," he said.

There is little political argument about the theory of universal credit. All parties agree that a system that simplifies the benefits system, builds in incentives by making work pay and, thereby, again in theory, saves the state money in the long run by changing cultural attitudes, must be good. But where there is argument – even within government – is over the rate of progress, the handling of its implementation by the DWP and Duncan Smith, the soaring costs and, ultimately, the feasibility. It is a vast, complex enterprise, presenting an IT challenge that experts say is nothing short of mind-boggling.

At present the project seems to exist in two parallel universes. One is inhabited by Duncan Smith and the few well-primed DWP and Jobcentre Plus staff involved with the pilots. This is a world where problems do not seem to exist, targets can be missed and new ones set as if the delays had been planned all along, and where criticism is answered with more utopian promises about what wonders, one day, will come to pass.

Asked why people should still have faith when his plans are so behind target, Duncan Smith dodges and says he wants to go slow now for good reason. "We have done this because people's lives matter and we want to make sure that the UC system works as we move forward."

UC, he says, "is on track to roll out safely and securely against the plan I set out last year". Ultimately it will, he says, "make up to three million households better off by £177 a month on average and lift 300,000 children out of poverty".

The second universe is the one outside that bubble, where fears are growing that UC is developing into a catastrophe that will dwarf previous IT disasters. Even inside government there are profound doubts. After £40m was written off on IT systems that are no longer fit for purpose, it recently emerged that the body responsible for grading its implementation, the Major Projects Authority, had "reset" UC as an entirely new project, meaning that it was classed as having been sent back to the drawing board. It had failed all its tests, so it was starting again.

The Treasury, which refused recently to sign off on a new business plan for UC, is now said to be "drip-feeding" money to it, keeping it alive rather than committing to the huge funds it needs upfront. In April the all-party work and pensions select committee issued a report so scathing it read like a hatchet job. It complained that the original IT system for UC was already defunct and suggested that bad money was being thrown after good in the desperate chase to make new systems work. "The underlying issue ... is that the department has written off £40.1m on assets it will now never use and spent a further £91m on assets that will support only a limited service for five years, with clear consequences for public value."

The report also accused the DWP time and again of witholding information and failing to admit problems until forced to do so, and of a "deeply concerning lack of clarity" about how the eventual IT system – the so-called "end-state solution" – will work.

The National Audit Office has raised doubts about the new IT systems: when they will be ready, how much it will all cost and the competence of those managing it.

Amyas Morse, head of the NAO, said last year: "The department's plans for universal credit were driven by an ambitious timescale, and this led to the adoption of a systems development approach new to the department. The relatively high-risk trajectory was not, however, matched by an appropriate management approach. Instead the programme suffered from weak management, ineffective control and poor governance. Universal credit could well go on to achieve considerable benefits if the department learns from these early setbacks and puts realistic plans and strong discipline in place for its future rollout."

But there are still doubts at the highest levels of government about whether it has done so. Progress is not being helped by a high turnover of executives. John Manzoni, head of the Major Projects Authority, told the public accounts committee earlier this summer that the current head of UC, Howard Shiplee, had been working only part-time after falling ill around Christmas.

Rachel Reeves, the shadow work and pensions secretary, says the entire project is well intentioned but chaotic and gave no guarantee that it could be resuscitated. "Labour supports the principle of universal credit, but we will not accept the huge waste and delays which have brought the government's £12.8bn welfare reform programme shuddering to a halt," she said. "A Labour government will call in the National Audit Office to review universal credit. This review will enable Labour to take hard-headed decisions about whether universal credit can be rescued."

So what, precisely, is the problem that is holding up the delivery of a vision that Duncan Smith still seems sure can earn him a reputation as a great reformer in the mould of Wilberforce? Brian Wernham, an independent project adviser and author of Agile Project Management for Government, who has more than 30 years' experience in both the public and private sectors, said computer systems were being asked to deal with hugely complex shifting human and family situations. "The system has to make fiendishly complicated calculations based not just on the income of one person, but everyone else in the household," he said. These would change constantly and the software had to show it could respond. "To make things worse, there is the 'lobster pot' principle – once a claimant is in the universal credit system, he/she remains in it. No matter how complex their life becomes, the IT just has to deal with it."

Wernham added: "We accept the occasional flaw in our everyday use of IT, like GPS car navigation, such as no right turns being missed, because we over-ride these mistakes by applying common sense, hopefully." UC is different, and the problems bigger. It has to be foolproof as "when dealing with a £110bn benefits budget, the risk of a small percentage error in calculations cannot be allowed. Fraudsters will open up even the smallest crack in the logic of such systems and hundreds of millions of pounds will evaporate, as we saw in the VAT frauds a few years back." He predicts that existing problems with the DWP's "sprawling systems and processes will take 10 years or so to iron out".

Fear of fraud clearly hovers over the project. What if hackers entered the system and invented claimants who did not exist, as they can do with companies, asked one well-placed official linked to the project? Dame Anne Begg, chair of the work and pensions select committee, said UC had the long-term potential to cut fraud, but there was a danger this could be "undermined because of the uncertainty about how DWP will administer the housing element of universal credit without increased risks of fraud and error".

She said that, under the current housing benefit system, local authorities can cross-check claims across a range of data relating to other council services; unless the centrally controlled DWP was able to cross-check universal credit claims in a similar way, "it may be less effective in tackling fraud and error".

Debbie Gibbons, national chair of the Local Authority Investigation Officers Group, which represents staff investigating fraud at local level, said moves to centralise fraud investigations inside the DWP in future risk leaving the new system badly exposed. "The proposal to create a Single Fraud Investigation Service (SFIS) for welfare benefit investigations would be a step in the right direction. However, the decision to create this service within DWP without the input of the existing and successful local authority investigation services will dilute the service with the loss of local intelligence, a major factor in the successful anti-fraud measures in local authorities."

Gibbons has relayed her concerns to government. Pete Challis, national officer for Unison, says fraud is the "Achilles heel" of universal credit. "Organised crime is already sophisticated at using identity theft and ghost employees, and the risk of organised crime infiltrating the state benefit system is significant. Relying on remote central IT systems, removing local knowledge and human contact, will make it easier for fraudsters."

Duncan Smith has no truck with the doomsayers and, while hearing them out, bats them away. He says he himself decided to reset the project to zero this year. "My overriding concern was to ensure we introduced universal credit carefully so that no one would be negatively affected – as they were when Labour introduced tax credits – testing it at each step before expanding across the whole country." Progress , he says, is now being made according to a "test, learn, implement" template. It may be tortuously slow, but it is safe, he says. He assures us that by the end of this year one in eight jobcentres will be offering universal credit and that by 2017 more than a million should be on the system.

Out in the real world, minds are turning to how it will affect individual claimants, how they will handle the transition.

A feature of UC which Duncan Smith hopes will acquaint those on benefits more with the world of work is that people will receive benefits monthly rather than fortnightly. People now on housing benefit will have to settle their bills themselves a month in advance, rather than the money being paid direct to housing associations or local authorities. This huge change is causing its own worries.

Housing associations and local authorities are taking matters into their own hands. They are concerned that people will be unable to make the transition easily and some are, controversially, already urging people on benefits to put aside extra money now so that they will be in credit on their housing accounts by the time UC is introduced.

In one case known to the Observer, a housing association said it would not refund more than £320 to a woman who had overpaid "bedroom tax" because it wanted to ensure that the money was on her account before her switch to UC, although no one knows when she will move on to the new system.

"I think it is appalling that they are doing this to people who obviously struggle on what they receive as it is, and when they do not know when they will change to universal credit," the woman said.

The housing association, when approached by this paper to explain, said it would write to the woman and reach an amicable solution.

Citizens Advice is alarmed at such cases and by the lack of information available to people ahead of their eventual transfer on to UC. Gillian Guy, chief executive of Citizens Advice, said that "introducing such a complex reform on such a big scale without fully preparing people for the changes in store is like making the families affected walk a tightrope without a safety net".

She added: "A rollout which delivers universal credit slowly and safely rather than quickly and badly is crucial. As the introduction of the reform continues, ministers need to tackle emerging problems which have the potential to harm households' wellbeing if they are not addressed. Only people in the very simplest circumstances have so far moved on to the new single benefit and there are already warning signs for government to heed. Citizens Advice Bureaux are reporting problems with the new software's ability to deal with changes in personal circumstances, resulting in people being left without financial support.

"Huge numbers of people will need strong, personal support to manage with the new system. The changes to budgeting, managing money and online applications are a major shift. There is still a lack of information about how people with disabilities or complex personal circumstances will be supported through the transition onto the new system."

Some at Westminster believed Duncan Smith would lose his job in the recent cabinet reshuffle. They thought he would pay the price for the difficulties his big idea is encountering and the mounting costs. But David Cameron kept him in place, expressing his full faith in the secretary of state.

To have moved him, Tory MPs observed afterwards, privately, would have looked like an admission of failure and would have left a new man or woman with the poisoned chalice of sorting out the mess. With Andrew Lansley having being sacked as health secretary in mid-term and Michael Gove just shunted out of education, sacking Duncan Smith would have reflected too badly on this government's public service reform agenda.

So Duncan Smith soldiers on, still hoping that universal credit will one day cement his place in history.