They were horrified when consultant anaesthetist Manoj Srivastava had a new home built for his family on Portland Drive, Willen, several years ago.
The house was so big that it resembled a mini hotel, they said.
Dr Srivastava had planning permission to demolish the original bungalow on the site and replace it with bigger house. But, after certain breaches of the consent were discovered, he was forced to seek retrospective permission – which was refused by MK Council.
The council ordered enforcement action be taken to reduce the size of the building but Dr Srivastava appealed to the Planning Inspectorate.
A three-day planning inquiry was held last year, during which Dr Srivastava, a Hindu, claimed he faced racial discrimination and an “organised coup” from his neighbours and the council.
His barrister argued that the council's position was to make the situation acceptable, and demolition was not justified.
But this month the results of the appeal were published, showing Dr Srivastava lost his battle.
The enforcement notice has been varied and upheld and he now has 12 months to either; remove the entire roof of the house and rebuild it to a 30 degree pitch and lower height, or completely demolish and rebuild a large first floor wall at the rear.
Failure to comply with the enforcement notice would be a criminal offence and he could face being prosecuted and fined.
A spokesman for the Willen Residents’ Group, which was also represented at the appeal, said: “The planning inspector accepted our arguments that this unlawful house causes significant and unacceptable harm to the character and appearance of Willen.
“This has been a difficult and stressful experience for the residents of Willen over the last four years which has regrettably been compounded by a lot of mistakes made by Milton Keynes Council. We hope that the council will now meet with us and respond to our concerns about how they have handled matters.”
The group has thanked former former councillors Catriona Morris and John Bint for their support, as well as Campbell Park Parish Council.
In response, Dr Srivastava said just two words: “Satyameva Jayate” - a Hindu mantra meaning ‘truth alone triumphs'.