Chronic stress is encountered in our everyday life and is thought to contribute to a number of diseases. Many of these stress-related disorders display a sex bias. Because glucocorticoid hormones are the main biological mediator of chronic stress, researchers have been interested in understanding the sexual dimorphism in glucocorticoid stress response to better explain the sex bias in stress-related diseases. Although not yet demonstrated for glucocorticoid regulation, sex chromosomes do influence sex-specific biology as soon as conception. Then a transient rise in testosterone start to shape the male brain during the prenatal period differently to the female brain. These organizational effects are completed just before puberty. The cerebral regions implicated in glucocorticoid regulation at rest and after stress are thereby impacted in a sex-specific manner. After puberty, the high levels of all gonadal hormones will interact with glucocorticoid hormones in specific crosstalk through their respective nuclear receptors. In addition, stress occurring early in life, in particular during the prenatal period and in adolescence will prime in the long-term glucocorticoid stress response through epigenetic mechanisms, again in a sex-specific manner. Altogether, various molecular mechanisms explain sex-specific glucocorticoid stress responses that do not exclude important gender effects in humans.