Simon Cowell surprisingly amused at The X Factor