Temporal Dominance of Sensations (TDS) and Temporal Check-All-That-Apply (TCATA) are the most used qualitative temporal sensory evaluation methods. Considered as rapid methods, they are more and more used with consumer panels. However, no study has investigated in detail the validity and reliability of these methods when they are used with consumers. This study aimed to fill in the gap by comparing the results obtained from temporal measurements to controlled temporal stimuli delivered using a gustometer. A total of 149 consumers were randomly assigned to one of three panels, each panel using a different temporal sensory evaluation method among TDS (n = 50), TCATA (n = 50) and AEF-RATA (n = 49). AEF-RATA is introduced as a new method allowing the consumers to retrospectively report the intensity of the applicable attributes in three defined periods. Using a gustometer, four single-compound solutions were delivered to the consumers to evaluate their recognition ability using Free Comment. Then, eighteen multiple-compound solutions (composed of two to five compounds varying in their sequence, concentration and duration of stimulation) were delivered to the consumers to evaluate their ability to use the three temporal evaluation methods. The compounds included sodium chloride (“salty”), saccharose (“sweet”), citric acid (“acid”), citral (“lemon”) and basil hydrosol (“basil”). The results show that consumers were able to report temporal sequences of perceived attributes congruent with the delivered sequence of compounds in the stimuli. Contrary to our hypotheses, despite different operational definitions, very few differences are observed between the TDS and TCATA measures, suggesting that most consumers interpreted applicability as dominance. Citation rates and durations of dominance/applicability are strongly correlated between them, but not with the concentration of the compounds. Individual repeatability is low, but does not impact reliability of the measures at the panel level. Consumers can be confused with attribute identification, which can make the interpretation of low citation rates tricky. TDS restores the signal the closest to the stimuli, except for durations for which TCATA performs better. AEF-RATA has no advantage over the other methods. These results open a discussion on what is possible to ask consumer panels and how not to overinterpret the results.