Ministers have accused Historic England of 'putting down' Britain's past after the public body linked villages, halls, churches and pubs to slavery in a sweeping 157-page report.
An audit of pre-existing work by Historic England, an arm of the Department for Culture which is given £88.5million to preserve buildings and monuments, tied rural communities to 'money made in transatlantic slavery'.
The body cited the patronage of figures including Edward Colston, William Gladstone and Francis Drake as part of the 'money trail' linking buildings to the wealth generated from slavery.
Its report also notes that less well-known families invested this wealth to improve infrastructure - causing villages to be included in the audit for funding for church repairs and road works in the 18th and 19th centuries.
Historic England say the information will be used to 'enrich' its records and provide topics for more in depth research in the future.
However MailOnline can reveal the project - which even audited chapels where slave profiteers and their relatives were buried - has been slammed by ministers who are 'frustrated' at the way public bodies focus on 'divisive parts of Britain's history rather than celebrating our shared heritage'.
A source in the Department for Culture told MailOnline: 'Ministers are increasingly frustrated with public bodies focusing on divisive parts of Britain's history rather than celebrating our shared heritage.
St Nicholas's Church in Deptford was listed due to its memorials of those involved in slavery, including the slave trader Edward Fenton and John Julius Angerstein, who co-owned plantations in Grenada and established Lloyds of London
Among the many sites which have been linked to the transatlantic slave trade include a school in Nunnington, North Yorkshire built by William Rutson, grandson of the cotton merchant and slave trader William Rutson of Liverpool
The statue of 17th-century slave trader Edward Colston being dumped in Bristol Harbour during a Black Lives Matter protest rally last summer
Ministers are 'frustrated' with public bodies including Historic England because they believe that they are taking partisan positions in a live debate on culture and history.
Culture Secretary Oliver Dowden wrote to the department's 'Arm's Length Bodies' - including Historic England - outlining how they are not allowed to engage in 'activism'.
In a letter sent on September 22, 2020, he wrote that as 'publicly funded bodies' the likes of Historic England should not 'erase' contentious parts of British history but instead 'seek to contextualise or reinterpret them in a way that enables the public to learn about them in their entirety'.
Mr Dowden called on Historic England to remain 'impartial' in the debate, largely fuelled by the sudden global Black Lives Matter protests.
He wrote: 'History is ridden with moral complexity. Statues and other historical objects were created by generations with different perspectives and understandings of right and wrong.
'Some represent figures who have said or done things which we may find deeply offensive and would not defend today. But though we may now disagree with those who created them or who they represent, they play an important role in teaching us about our past, with all its faults.'
The Culture Secretary continued: 'Historic England, as the Government's adviser on the historic environment, have said that removing difficult and contentious parts of it risks harming our understanding of our collective past. Rather than erasing these objects, we should seek to contextualise or reinterpret them in a way that enables the public to learn about them in their entirety, however challenging this may be.
'Our aim should be to use them to educate people about all aspects of Britain's complex past, both good and bad.
'As set out in your Management Agreements, I would expect Arm's Length Bodies' approach to issues of contested heritage to be consistent with the Government's position.
'Further, as publicly funded bodies, you should not be taking actions motivated by activism or politics. The significant support that you receive from the taxpayer is an acknowledgement of the important cultural role you play for the entire country.
'It is imperative that you continue to act impartially, in line with your publicly funded status, and not in a way that brings this into question.'
'We should face up to the challenging parts, but this needs to be done in a balanced way rather than constantly putting down our past.'
The audit was completed last summer, shortly after the toppling of Colston's statue and its dumping in Bristol Harbour by Black Lives Matter protesters.
The report, which is available online, states: 'The transatlantic slavery economy was invested in the built environment of the local area in housing, civic society organisations, churches, village halls, farms, shooting lodges, hotels.'
Among the many sites which have been linked to the transatlantic slave trade include a school in Nunnington, North Yorkshire built by William Rutson, grandson of the cotton merchant and slave trader William Rutson of Liverpool.
Rutson also funded a school, refurbishments to a church and for the rebuilding of houses in the quaint village, as well as buying Nunnington Hall in 1839 as a shooting lodge.
A small pub in Brockenhurst, Hampshire called the Morant Arms made the audit due to its connections to the Morant family, which owned extensive sugar plantations in Jamaica and invested in the village.
The report says Edward Morant moved to England from Jamaica and bought Brockenhurst House and estate in 1770, subsequently rebuilding the house as a large Georgian mansion with extensive grounds.
His son John Morant purchased the Manor of Ringwood in 1794. Brockenhurst House was greatly extended in 1865. Later demolished, a new house was built in 1960.
Historic England noted in its report, completed last year, that 'many English churches are the burial sites of local families with slavery connections' and sites of internment including rural places of worship and London chapels are included.
A gravestone in Dorset's Holnest has been reviewed because it commemorates slaver John Samuel Wanley Sawbridge, who married Jane Frances Grosvenor, a member of the slave-owning Drax family of Barbados.
The report states Sawbridge took possession of Charborough House and also owned an estate at Holnest, where he built an elaborate mausoleum besides the parish church. This was demolished in 1935 and replaced by a flat memorial stone.
And St Nicholas's Church in Deptford was listed due to its memorials of those involved in slavery, including the slave trader Edward Fenton and John Julius Angerstein, who co-owned plantations in Grenada and established Lloyds of London.
The report is more comprehensive than the National Trust review, which was limited to stately homes.
However, it does not address all 'tombs, monuments and memorials of individuals and families made wealthy from associations with the Atlantic slave economy'.
Historic England said the audit would: 'Identify significant gaps in knowledge that can be targeted through new collaborative research in order to produce a more complete picture of the impact of Atlantic slavery on the built environment in England.
'This new knowledge will over time facilitate Enriching the List entries and enhancements to the National Heritage List for England as well as providing a vehicle for greater engagement with heritage among under-represented audiences.'
Historic England told MailOnline that the audit 'will absolutely not be used to delist structures' in its drive to diversify.
BROCKENHURST: A small pub in Brockenhurst, Hampshire called the Morant Arms made the audit due to its connections to the Morant family, which owned extensive sugar plantations in Jamaica and invested in the village
HOLNEST: A gravestone in Dorset's Holnest has been reviewed because it commemorates slaver John Samuel Wanley Sawbridge, who married Jane Frances Grosvenor, a member of the slave-owning Drax family of Barbados
In a statement, a spokesperson said: 'In early 2020, we commissioned an audit which brings together previous research into the tangible traces of the transatlantic slave trade in England's built environment, mostly carried out over the last thirty years by universities and community groups.
'The audit has also identified gaps in knowledge and makes suggestions for future research.
'This knowledge will absolutely not be used to delist structures, but it will be used to enhance the National Heritage List for England and tell a fuller story of England's rich and complex history.
'As a separate piece of work in November we published our Inclusion, Diversity and Equality Strategy following two years of development and consultation.
'It reaffirms our commitment to delivering our work in a way that benefits a broader range of people, places and communities which better represent the diversity of England and our rich heritage.
'Heritage is for everyone and we want our work to ensure that a diverse range of people are able to connect with, participate in and enjoy the historic environment.'