Access Hollywood: Every week, millions of viewers tune in to "American Idol" to hear what Simon Cowell has to say -- but according to the Mirror, the no-nonsense judge may be planning on leaving the show.

"Most people do one show a year and I do three and they involve a lot of traveling. One may have to go," Cowell, 49, told the paper on Saturday. "I don't know which one. That is the problem, I like all of them. I can't imagine not doing 'Britain's Got Talent' and I can't imagine not doing 'X Factor.' But I guess America is more likely to go because I have got one year under contract. Maybe that will be the end."

According to the paper, Cowell may make four transatlantic flights a week during the summer to keep up with "American Idol" and his two British shows, "X Factor" and "Britain's Got Talent," in which he also serves as a judge.

"I don't want to come over as whiney because I am very grateful that I have got these jobs," he said. "But there is a point where I am not sure we can keep this schedule up. It takes longer and longer to make these shows because you have to put more into them, and I am not sure we can sustain me doing three shows a year, with the record label and the TV production company."

In addition, Cowell added that he's launching a "big new show next year," though he said details are top-secret.

As for a social life, the "Idol" judge said his busy schedule doesn't leave him with much time to date following his split with longtime girlfriend Terri Seymour.

"Have I got a black book?" he laughed. "Yes, it's called a mobile phone. I do get offers. There is no shortage of people if you want to go on dates -- working in TV, living in LA, it is there if you want it. But I am enjoying hanging out with my friends. I haven't had time to go on dates."

Read more hot stories Moms Are Talking About.