Cold water can be deadly for the unprepared. We often boat in water under 60° in temperature. In water between 40° and 50°, you have 30 - 60 minutes before you become unconscious. This might sound like a fairly decent amount of time on whitewater where you can quickly swim to shore. In large bodies of water, you can't swim to safety that fast. In whitewater we also have to consider entrapment possibilities. It takes time to setup an entrapment rescue. If you become unconscious, you can't help in your rescue and your face is likely to fall down in the water where you are likely to drown.
The best way to avoid this scenario is to dress appropriately. Many boaters paddle in cold weather with a dry top and not much below their waist. They feel warm since the boat provides significant insulating air. If they swim, they quickly lose body heat. In addition, they are useless for assisting in many rescues. Dress in layers that can be added or subtracted. Pack a hood or skull cap just in case. The hood doesn't take up much space and is amazing at providing quick warmth. The following article is excellent: Cold Water Survival. If you haven't done so, you should review the hypothermia article as well: Hypothermia & Hyperthermia.