The government is considering structural changes to the way services are commissioned for people with learning disabilities after a report found services are below par.
A Healthcare Commission audit concluded services for people with learning disabilities are not meeting the standards expected in the 21st century.
Most services are providing a poor standard of care and, even in the best services the safety, the quality of care is not adequate.
Tomorrow the government will publish details of a consultation designed to improve facilities and care for people with learning disabilities.
Among the proposals are plans to transfer responsibility for commissioning services from the NHS to local government.
The Healthcare Commission said "sweeping and sustained" changes are needed to services.
Concluding its audit, the commission said services risked depriving people of their human rights and dignity.
Although the commission found no evidence of physical abuse, six services have been referred to local authorities after concerns were raised.
Anna Walker, chief executive of the Healthcare Commission, said: "The first thing to say is that there is nothing in this audit that should lead to widespread concern about safety. but we will remain vigilant on this on behalf of service users and their families.
"This report, however, does paint a bleak picture. Services for people with learning difficulties are not generally unsafe but they are poor. These services are regularly neglected and too often old-fashioned and institutional.
"I want to be clear that there are many members of staff working hard for the people they serve. But they operate in a system where too many people are not given choices and control over their lives. Care is not personalised, living environments are poor and activities are few."
Responding to the report, care services minister Ivan Lewis said people with learning difficulties deserved to be treated with dignity and respect.
Mr Lewis said: "It is totally unacceptable for anyone with a learning disability to be treated in a way that compromises their human rights."
He said he was pleased the commission would be monitoring NHS trusts on learning disability services, adding: "All relevant NHS organisations will have to deliver action plans setting out what they are doing to address shortcomings in learning disability services identified by the Healthcare Commission's audit."