We introduce a notion of ternary distributive algebraic structure, give examples, and relate it to the notion of a quandle. Classification is given for low order structures of this type. Constructions of such structures from 3-Lie algebras are provided. We also describe ternary distributive algebraic structures coming from groups and give examples from vector spaces whose bases are elements of a finite ternary distributive set. We introduce a cohomology theory that is analogous to Hochschild cohomology and relate it to a formal deformation theory of these structures.