Spencer Pratt Said David Foster Encouraged Him to ‘Be the Bad Guy’ on TV

Spencer Pratt never considered himself an on-camera personality until he got encouragement from an unlikely source.

Many who grew to know and love the 41-year-old as everyone’s favorite bad guy on The Hills may not know that the series wasn’t Pratt’s first foray into reality TV.

Appearing on the April 30 episode of SiriusXM’s The Julia Cunningham Show, Pratt recalled creating The Princes of Malibu, a look at the charmed life of Brody Jenner. Filming for the short-lived reality series took place at the time Jenner was living with his brother Brandon, plus his mom, Linda Thompson, and her then-husband, David Foster.

Pratt made small appearances as Jenner’s best friend/manager/publicist, but he wasn’t interested because he wanted to pursue acting and producing in a more serious capacity. Of that decision, he said, “I joke it’s like The Matrix — I didn’t realize I was taking the pill. And then once I was in, I was like, ‘Too late.’ ”

Never miss a story — sign up for PEOPLE’s free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

“I didn’t think about it, and that moment did change my life forever because nobody ever took me seriously again. Nowadays, there’s a Kim [Kardashian] who has been able to be a reality personality and an actor. Good for her, you know, but that was impossible [back then], so I kind of just accepted what I got into.”

His perspective changed when he was approached by Jenner’s stepfather.

“Then David Foster, he was like, ‘You need to be the Simon Cowell of reality TV.’ And I didn’t even watch American Idol. I didn’t even know who Simon Cowell was,” Pratt admitted.

“I was like, ‘What? What does that mean? What are you talking about?’ He’s like, “Be the bad guy. Be the villain.’ ”

Pratt grew into a “blunt” approach, but said he often ended up misunderstood by viewers as a result.

“The problem with me being blunt and doing whatever producers wanted is the audience doesn’t see [it as], ‘He’s just telling it how it is.’ It’s just, ‘He’s a monster,’ ” he explained. “So that’s what I didn’t see looking back. People, they took it very serious.”

Pratt appeared on The Hills and its recent reboot, The Hills: New Beginnings, in addition to other reality favorites like Celebrity Big Brother and I’m a Celebrity, Get Me Out of Here!. The multi-hyphenate, who is married to fellow reality star Heidi Montag, most recently starred on the Hulu reality show Got to Get Out.