Fans are always on the lookout for a new man in Taylor Swift's life, matching her to a number of ideal actors and singers and patiently waiting for the time when their love story will come out on her album. But while some men may be flattered to be linked to the country crooner, actor Zach Braff, 39, is not.

The Scrubs star spoke with Howard Stern on his radio talk show on Monday, and he vehemently denied the rumors that he and Taylor Swift was once an item. He also gave his thoughts on having a girlfriend that's well outside the industry.

Braff quickly retaliated when Stern joked that Zach and Taylor are "Very, very, very close friends."

"I've met Taylor a few times and she's a sweet girl...but that was all Internet bulls**t," the Garden State actor explained. "They [the tabloids] said I was dating her, I met her twice, she's a cool girl, but I was not dating her."

But while the rumors have now been clarified, Stern still pressed on, saying that Braff and Swift would be great together. Braff then went out to say that he doesn't want to date "the super-famous girls."

Braff, whose famous flames include Bonnie Sommerville and singer Mandy Moore, revealed, "Remember I did that once before when I dated Mandy Moore and I didn't like it. When you date another famous person that's when they really hound you. That's when the paparazzi are outside your house and that's when you can't ride your bike through Manhattan because they follow you."

He also mentioned an important relationship advice his publicist once shared with him, which he says helped him navigate the Hollywood dating scene.

"When my relationship with Mandy ended, she [my publicist] said, 'I'm going to give you a hint that you should know. If you don't date these 25 girls, your life will be so much easier.' And she was kind of exaggerating because there's more than those 25 girls, but in so many words, if you can, don't date other famous people because then all the tabloids pretty much leave you alone."

He also says that hooking up with a well-known celebrity may be "fun for your ego" but ultimately, the paps will end up following you everywhere you go.

Braff mentions his famous One Direction friend, Harry Styles as an example. "He can't walk two f**king feet or a crowd forms and a paparazzi forms."

"I don't want that existence," he finished.