During his recent appearance on the Radio Andy show Bevelations, Bevy Smith asked the actor about Osbourne's departure, as well as the recent exits of former hosts Carrie Ann Inaba and Elaine Welteroth. O'Connell, 47, currently co-hosts the series alongside Amanda Kloots, Sheryl Underwood and newly-announced panelist Akbar Gbajabiamila.
"For you, were you at all concerned about coming into a space after there had been that kind of, uh, a blow-up or kerfuffle or drama?" asked Smith.
"Yeah, a little bit," he admitted. "It's funny, I didn't know the exact details of everything that happened. I remember reading that Sharon Osborne was no longer on The Talk and then I was literally on there."
Cliff Lipson/CBS via Getty
He went on to compare his presence on The Talk to being the "rebound guy" when he first met his wife, Rebecca Romijn, after she had gone through a divorce.
"And I'll tell you, being that rebound guy is fun," he said. "My wife would get depressed and I'd be like, 'What are you depressed about? Like, let's go have fun. Who cares?'"
O'Connell continued, "Coming in here right after Sharon Osbourne left, I could tell there was trauma with this group, you know, they were shook, they were shooketh. And I came in and I was like, 'Hey guys, I don't know anything about that. Like, let's just have a good time.' And anytime it was even brought up, I was like, 'Hey guys, guys, guys, that wasn't me, please. Can we not talk about that today?'"
"I get to be like the hot new boyfriend," he concluded.
RELATED VIDEO: Sheryl Underwood Breaks Silence After Sharon Osbourne's The Talk Exit, Says They Haven't Spoken
CBS announced Osbourne's departure from the talk show on March 26, after the series had taken a hiatus while the network looked into Osbourne's behavior from a March 10 episode.
During the broadcast, Osbourne, 68, defended friend Piers Morgan following the backlash for his controversial remarks questioning the validity of Meghan Markle's revelations about her mental health during her interview with Oprah Winfrey.
Since the on-air dispute with Underwood, Osbourne has also been accused of additional instances of racism, homophobia and bullying on the set of The Talk, all of which she has denied.
Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer, from juicy celebrity news to compelling human interest stories.
While on Bevelations, O'Connell also addressed being the first male host of The Talk, sharing that it didn't "seem like that big of a deal" to him when he was first tapped to join the series.
"It wasn't until I was out to dinner with my wife and my wife was like 'You know you being on The Talk, this is kind of a big deal' and I didn't really think of it that way," he said. "And these shows come and go so fast and as I get older — I don't want to depress your listeners — but I get sort of dead in the eyes."
"This business and how tough it is has sort of messed me up in being able to celebrate anything," he continued, though he added that "it is an honor" to be a co-host.