The royal’s appearance at the annual holiday party comes just weeks after his announcement that he is “stepping back from public duties” following his ties to disgraced financier Jeffrey Epstein.
The fallout from Andrew’s disastrous BBC interview about his friendship with convicted sex offender Jeffrey Epstein saw his official royal role collapse to almost nothing in a matter of days. He was removed from his hundreds of charitable patronages and his office was forced out of Buckingham Palace.
The decision “shows that the royals can move very swiftly and very ruthlessly,” royal historian Robert Lacey tells PEOPLE.
But the Queen, 93, has made sure that her personal support remains with her son. He accompanied her on horseback with two escorts on Nov. 22 — something she would know would likely be captured by photographers and send a signal that, despite her decision to ask him to withdrawn from public work on her behalf, she still stands by him.
While Prince Andrew will not perform any more public duties as a member of the royal family, not much else will change at this time.
“He remains a member of the royal family,” a royal source says. “As a royal colonel and a war veteran, you can expect to see him at Remembrance Sunday. You can expect to see him on the balcony at Trooping the Colour too.”
Can’t get enough of PEOPLE‘s Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!
The Queen’s holiday bash typically seats more than 50 royal family members.
Andrew is also joined by Prince William, Kate Middleton and their two older children, Prince George and Princess Charlotte. Princess Anne and Princess Beatrice were also spotted arriving to the lunch. Zara and Mike Tindall were also in attendance.
Prince Harry and Meghan Markle did not attend as they are currently on their six-week break from royal duties and reportedly abroad with baby Archie.