Josh Carter‘s family is looking back on the lasting legacy of his late grandmother, Rosalynn Carter, and the joy she brought to her longtime Plains, Ga., home.
While on the red carpet at the “Jimmy Carter 100: A Celebration in Song” birthday concert on Sept. 17, Josh and his wife, Sarah, shared memories of the former first lady, who died last November at the age of 96.
“From the moment we actually laid her to rest there at the house and we walked into the building, her presence was missing and we felt that immediately,” Josh, 40, told PEOPLE. “It’s very bittersweet because she had such an amazing life, but the house was so much of her house.”
Sarah added, “She was the hostess. To walk in and have her greet you and offer to feed you and to take care of you, and she’s missing. We miss her.”
Married for 77 years, Rosalynn and Jimmy Carter had an unshakeable bond. Now, on the cusp of turning 100, the former president is missing his wife “a lot,” Josh said.
Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from celebrity news to compelling human interest stories.
Family members are also remembering the couple’s sense of duty and the impact they had on not only their family but also the greater community.
“My grandparents were very tight with wanting to know what was happening with their family,” Josh told PEOPLE, adding, “They have a whole life that they lived dedicated to helping people.”
As Josh and Sarah start their own foundation in honor of their youngest son, Jonathan, who was diagnosed with very early onset inflammatory bowel disease (VEO-IBD), they do so with the former first couple’s dedication to service in mind.
“We are able to kind of live that legacy ourselves and start this new foundation for parents and caregivers of children with VEO-IBD,” Josh said.
“Jonathan is doing great now,” Sarah told PEOPLE. “He just started kindergarten. He is healthy and happy.”