Someone asked a question about how frequently young people have time to socialize and it made me think about what people do with their evenings. I recently asked my son to go to a concert (free ticket to see a band i know he likes) and he declined because it was an hour away on a weeknight. If we invite our kids or niece/nephew to dinner they always want to go at 6/630 which feels so early. Edit: Kids are 30ish.
Oh, they are full grown adults. During the week, they are grinding at work. Sometimes they just want to come home and reset with their own family (wife, kids, games, catch up on shows, whatever). Not sure what their commute or work is like, but I can definitely see dipping out on random family events early or just not going out completely. Especially during working days.
I think you just need to plan it better. Maybe schedule family dinner and events on the weekends.