Rob Lowe believes his father and Drew Barrymore’s mother once slept together.

Rob Lowe believes his father and Drew Barrymore’s mother dated years ago.

The actress will be the next guest on Lowe’s Literally! With Rob Lowe podcast on Thursday, and in an exclusive clip provided with PEOPLE, the actors recollect on memories from the 1980s Los Angeles hot spot Helena’s. Barrymore, 47, remembers going to the star-studded club with her mother, Jaid, now 76.

Lowe informs Barrymore that he has “suspicions” that his father Charles and Jaid “ended up together” one night at the club. “I wouldn’t doubt it,” she continues, to which Lowe agrees.

“I kind of like the thought of it, I have to say,” he continues.

According to Barrymore, “My mother was and still is a lot of fun. She was a party girl.” Lowe continues, “My sense is, as mothers go, she was in her wheelhouse at Helena’s.”

“She and I used to go to Helena’s many times a week, I believe. You know how folks go to the gym a couple of times a week? We went there a couple times a week.”

Barrymore reportedly recalls seeing Sean Penn and Madonna there “all the time,” as well as Jack Nicholson, with whom she would subsequently party outside of the nightclub years later. Lowe described Helena’s as “amazing” noting the “disparate, insane, high level of people that were there.”

“Jack Nicholson was a fixture [at Helena’s],” adds Barrymore. “He went to Helena’s on the same nights as my mother and I did. And, ironically, I ended up partying with him at his place years later. It was never offensive. We’d simply party, speak, and hang around. I adore Jack. But I feel like I made friends with him at Helena’s, which morphed into us, like, partying at his place a decade later.”

The podcast Literally! With Rob Lowe is part of the Team Coco podcast network.