I've had this happen at least 5 or 6 times, usually don't push it down all the way after grabbing my glasses out of it, although I've never lost it, it always ends up in my lap and I'm able to put it back on on the fly.