Kalmia latifolia has declined in southern New England and other parts of its range in recent decades. This long-term decline is generally attributed to abiotic forces (i.e., low light levels in maturing forests) with little attention to the possible role that top-down effects from ungulate herbivory may be playing. We examined the extent to which mature K. latifolia is capable of sprouting under a relatively undisturbed forest canopy—both after severe stem injury and when uninjured—and tested the hypothesis that, in areas with high deer densities, herbivory may exceed abiotic forces in controlling the dynamics of K. latifolia. A block design experiment with white-tailed deer (Odocoileus virginianus) exclusion and control as treatments and landscape position (hilltop and low slope) as block was established in 2008. Canopy openness was measured in the two treatments within each block using hemispherical canopy photos. Survival and sprouting vigor of cut and uncut K. latifolia stems were monitored over 4 years and analyzed using Bayesian Information Criteria model selection with deer herbivory, percent canopy openness, and slope position as predictor variables. Canopy openness and slope position were important drivers of adult K. latifolia survival and sprouting capacity, whereas deer herbivory and slope position were the most important drivers of sprouting vigor on cut stems. Our results suggest that in a relatively undisturbed forest with high deer densities, herbivory does not exceed abiotic factors in determining adult K. latifolia vigor over the short term, but herbivory and slope position are more important than light in determining sprouting vigor after stem cutting.