Although it has never done so, the Electoral College could, indeed, do as you say, but to date there haven't even been a whole lot of instances of individual Electors bolting, let alone an en-masse event.
Depending on who runs, though, this year's election could conceivably inspire a few Electors to bolt and possibly swing a razor thin election.
I'm not ready to think about it yet.
Edit: It didn't even happen in '00, Bush v Gore when only 3 (self?)righteous Electors could have overturned the Supreme Court's decision...but Trump could be a-whole-nother matter.
