You're right on both accounts. Some people who are on water a lot don't. They either never feel threatened, or figure they will die in the cold water anyway, and a life jacket prolongs the death.

But that may be the storyteller working overtime, there.