A firm has been stripped of NHS contracts after hundreds of tonnes of clinical waste from hospitals was allowed to pile up at its sites.
Health Minister Stephen Barclay said new arrangements have been made to replace the service by Healthcare Environmental Services (HES).
HES says although it did also collect some body parts, this anatomical waste was correctly stored as per guidelines.
It removed waste from a number of hospitals in England and Scotland.
The Environment Agency first raised the alarm about the problem at the end of July, the government said.
The government says there was "absolutely no risk" posed to the health of patients or the wider public.
In a statement to Parliament, Mr Barclay said NHS Improvement concluded that HES "failed to demonstrate that they were operating within their contractual limits.
"Consequently, 15 NHS Trusts served termination notices to HES formally to terminate their contracts at 4pm on Sunday," he said.
New arrangements have been made with Mitie to "step in and replace this service" and "NHS services continue to operate as normal", Mr Barclay told MPs.
Mr Barclay said the primary concern was that too much waste was being held in a number of waste storage and treatment sites by HES.
"While the waste was stored securely, it was not being processed and disposed of within the correct regulatory timescales. At no point has there been an impact on public health or any delay to the ability of the NHS to carry out operations," he said.
The Environment Agency said HES was in breach of its environmental permits at four of its six sites which deal with clinical waste – by having more waste on site than their permit allows and storing waste inappropriately.
The EA has launched a criminal investigation.
HES said it had highlighted a reduction in the UK's high-temperature incineration capacity for the last few years to the UK government, NHS bodies and the Environment Agency.
Mr Barclay insisted there was "significant" additional incineration capacity.
The waste disposal firm is to retain its NHS contracts in Scotland despite being stripped of them in England and Wales. The Scottish Government said the decision did not apply here and existing contracts were unaffected.
Are you a current or former employee of Healthcare Environmental Services? Please get in touch. Email email@example.com.
Please include a contact number if you are willing to speak to a BBC journalist. You can also contact us in the following ways:
- WhatsApp: +44 7555 173285
- Tweet: @BBC_HaveYourSay
- Text an SMS or MMS to 61124 or +44 7624 800 100