Many Muslims may not seek to kill the infidel, but they don't want to condemn those carrying out the holy book command.
I must follow my Lord. No matter what. I must renew my allegiance every morning. It is His voice I must listen to, not the voices of those around me, however strident, however persuasive. It is His Word that must govern my life, not the words of others. God Himself has written a Book (think of it!) that must be the authority in my life.