His fundraising efforts offered a glimmer of light in a world darkened by Covid. But now his family are portrayed as freeloaders and a garden building created in his name faces enforced demolition