In a stochastic controlled dynamical system, stochastic dynamic programming is used for either cost minimization or viability maximization. We propose a change in variables that enables the minimization of a linear combination of expected costs and exit probabilities, thus effectively combining the expected value and worst-case approaches to a problem. The resulting algorithm facilitates the discussion of the possible trade-offs between the two kind of objectives through an example of water supply from a reservoir.