Summary: An optimization problem of controlling a dam installed in a river is analyzed based on a stochastic control formalism of a diffusion process under model ambiguity: a new mathematical approach to this issue. The diffusion process is a pathwise unique solution to a water balance equation considering the inflow, outflow, water loss in the reservoir, and direct rainfall. Finding the optimal reservoir operation policy reduces to solving a degenerate parabolic partial differential equation: a Hamilton‐Jacobi‐Bellman‐Isaacs equation. A monotone finite difference scheme is constructed for discretization of the equation, successfully generating nonoscillatory and reasonably accurate numerical solutions. Stability analysis of the resulting water balance dynamics is finally carried out for both environmentally friendly and not friendly reservoir operations.