Just 20% of people in England and Wales hit the recommended levels for blood pressure, cholesterol and blood sugar, figures from Diabetes UK suggest.
It wants more people to get the recommended annual checks and action taken against parts of the NHS with the worst record.
The Department of Health said it wants to improve diabetes services.
The charity obtained the figures by analysing data from the National Diabetes Audit.
It showed that for type 1 diabetes, just 11% of people meet the targets which help prevent the risk of complications.
High blood pressure, cholesterol and blood sugar suggest the patient’s diabetes is not well controlled and can lead to complications such as kidney failure and stroke.
The charity also said it goes a long way to explaining why 24,000 people with diabetes die early every year in England and Wales.
Currently only 54% of people with diabetes in England get the nine annual checks recommended nationally to prevent future problems.
As well as calling for more patients to receive this annual review, Diabetes UK said it wanted the government to hold poor performing areas to account because in some places fewer than 20% of people with the condition were getting the checks.
Care needs to improve to cope with the increasing numbers of people living with diabetes, the charity said.
It is estimated that the NHS spends about £10bn a year on diabetes – 10% of its entire budget – and about 80% of this goes on treating complications that could often have been prevented.
There are currently around three million people diagnosed with the disease.
Barbara Young, chief executive of Diabetes UK, said: “Given that diabetes is serious and can lead to early death if not supported to manage their condition, it is extremely worrying that so few people have it under control.
“The high rate of preventable complications is the inevitable consequence of a healthcare system that has all too often not been good enough and this highlights the need for local services to put in place self-management support programmes to help people manage their diabetes well.
“At the moment there is virtually no access to the ongoing education and support to help people manage their diabetes and so help avoid complications reduce their chance of early death.”
A spokesperson from the Department of Health said: “We are determined to improve NHS services across the country for people with diabetes and end the unacceptable variation in care that still exists.
“We have set clear objectives for the NHS to improve the care and management of people with diabetes and we will be monitoring NHS England to make sure this is delivered.”