ISIS claims that any Muslim who does not swear bay'ah (an oath of allegiance) to Baghdadi is an "infidel" or an "apostate".
Baghdadi is the leader of ISIL.
Is that true?
As explained in the article "What ISIS Really Wants" by Graeme Wood, ISIS supporters see Baghdadi as a legitimate caliph. This is a defining feature of ISIS. Muslims in general recognize that bay'ah to a legitimate caliph would be a fundamental duty. But only ISIS supporters (not even Al-Qaeda) believe that any legitimate caliphate has existed since (at least) the fall of the Ottoman sultanate.
According to Wood, referring to a particular sermon by Baghdadi:
the caliph commands obedience—and those who persist in supporting non-Muslim governments, after being duly warned and educated about their sin, are considered apostates.
Wood quotes directly from Baghdadi's sermon as follows, in reference to supporting the calpihate:
This is a duty upon the Muslims—a duty that has been lost for centuries … The Muslims sin by losing it, and they must always seek to establish it.
Note that in the available translation of the sermon itself the specific language of "infidels" or "apostates" is not used directly by Baghdadi himself, but arguably implied.
EDIT: ISIS directly discusses "apostasy" in various statements included as an appendix a Brookings report. In these statements they explicitly label the following groups as apostate: "[t]hose worshiping—or perceived to be worshiping—stones, saints, tombs, etc.", Shia muslims, "sorcerers", "whoever participates in the political process", "whoever extends to the occupier and his supporters any kind of assistance", "whoever disparages [the] honored stature and position [of the Prophet or the Rightly Guided Caliphs]", "all the rulers and armies of these states [of unbelief]", etc.
This falls short of literally labeling all Muslim non-supporters as apostates, but it comes fairly close because it is so open-ended. The only explicit limit they place is where they say "we do not believe in the unbelief of the generality of those entering [the political process], so long as legal proof has not [yet] been furnished for them."