The temperature range for consuming hot drinks includes temperatures that can damage cells on the tongue. We hypothesized that the consumption of very hot drinks can lead to a decrease in the ability to perceive low concentrations of tastants. We evaluated the ability to perceive low concentrations of five prototypical sapid compounds in 42 women and 40 men aged 18–65. A questionnaire made it possible to collect the usual frequencies (number of unit/day) and consumption temperature levels (medium hot/very hot) for four very common hot drinks (coffee, tea, herbal infusions, and hot chocolate). Our results showed that subjects who consumed very hot drinks (versus medium hot) were less sensitive to sweet (p = 0.020) and salty (p = 0.046) tastes. An aggravating effect of high consumption frequencies was only shown for sweet taste (p = 0.036). Moreover, our data also showed that women were more sensitive than men to sour, bitter, and umami tastes (p values < 0.05), as well as that taste sensitivity decreases with age, especially after 50 years old (all tastes; p values < 0.05). These findings strengthen our knowledge about the influence of sex and age on taste sensitivity, and they provide knowledge on the influence of consumption habits related to hot drinks on taste sensitivity.