Gordon's 64th-minute goal was checked by VAR for over four minutes, with the ball having potentially gone out of play and checks also taking place for a foul and a marginal offside call.
Arsenal were furious at the decision to let the goal stand, as their unbeaten league ended at St James' Park and they lost ground to Manchester City and, come Monday night, potentially Tottenham.
Arteta risked a serious Football Association (FA) sanction as he delivered a brutal tirade towards the officials led by referee Stuart Attwell and VAR Andy Madley.
In a fiery post-match press conference, Arteta said: "We need to talk about the result because we need to talk about how the hell this goal could stand? It's incredible.
"I feel embarrassed but I have to stand here and try to defend the club and please ask for help because it is an absolute disgrace that this goal was allowed - an absolute disgrace because it is not a goal.
"I feel sick. That's how I feel. I feel sick to be part of this."
An ANGRY Mikel Arteta shares his thoughts on the controversial Newcastle goal that meant Arsenal suffered their first Premier League defeat of the season. 😳 pic.twitter.com/fYT5lyo6eF
— Sky Sports Premier League (@SkySportsPL) November 4, 2023
"For many reasons, it is not a goal. For more than at least one reason, it is not a goal. It is too much at stake here. We put in so much to compete at this level and what we have seen is an absolute disgrace.
"Again, I feel embarrassed. I have been in this country for more than 20 years and this is nowhere near the level of what is described as the best league in the world."
Asked whether the four minute and two second check meant officials took care of the decision, he said: "That makes it even worse - even worse. You just need to see one image and, if you have any doubt, you look at the second one. If this is a goal, then okay, that's fine.
"I don't care what they say, it is the outcome and it is too late, too late. I don't want to be in the hands of [these] people.
"It's simple, it is not a goal. There are certain things that happened for the goal that are not allowed in football, in football in Japan, China, Spain, in football they are not allowed but that's what happens."
He continued to explain he thought Eddie Nketiah should have had a penalty, saying: "There's a lot of things, we could continue."
He refused to criticise his players, who have now suffered back-to-back defeats in all competitions, adding: "I have praised my players for the way they played and competed in this stadium against this team."
Bruno Guimaraes could have also been sent off for an elbow on Jorginho late in the first-half, while Kai Havertz recklessly lunged into a challenge moments earlier but was lucky to avoid a red card.