John Robinson, of Eaglestone, had shown signs of recovery after falling from the 12th storey, but died in Milton Keynes General Hospital on Friday evening.
Corporate manslaughter charges have not been ruled out although both police and the Health and Safety Executive (HSE) said it was too early in the investigations to speculate.
Mr Robinson was working on the scaffolding with his son Mark when it crashed down at 12.30pm last Tuesday.
It is understood Mr Robinson – who leaves three children and six grandchildren – suffered chest and leg injuries.
Mark, and another worker, remain in a serious but comfortable condition at hospital.
Martin Leekblade, commercial director of LSC Ltd, which employed Mr Robinson, said: "We're still trying to get to grips with it – we're all a bit shell shocked.
"On Thursday it sounded as though everything was going to be OK.
"Our thoughts and sympathies are with the family.
"At the end of the day he was trying to make a living – this sort of thing shouldn't happen nowadays."
He added the company will be meeting the HSE today (Tuesday).
A spokesman for the GMB union said: "The fact this accident has now led to the death of one of the workers while leaving two others seriously injured underlines how dangerous the construction industry actually is, and the need for extreme vigilance and adherence to a safe working regime."
A Thames Valley Police spokesman said: "We are working with the HSE on their
investigation, but as it stands we are not looking at corporate manslaughter charges."
No-one from the Robinson family was available to comment.
llee.macgregor@mkcitizen.co.uk