Anthropogenic impacts are typically detrimental to tropical coral reefs, but the effect of increasing environmental stress and variability on the size structure of coral communities remains poorly understood. This limits our ability to effectively conserve coral reef ecosystems because size specific dynamics are rarely incorporated. Our aim is to quantify variation in the size structure of coral populations across 20 sites along a tropical-to-subtropical environmental gradient on the east coast of Australia (~23°S to 30°S), to determine how size structure changes with a gradient of sea surface temperature, turbidity, productivity and light levels. We use two approaches: 1) linear regression with summary statistics (such as median size) as response variables, a method frequently favoured by ecologists; and 2) compositional functional regression, a novel method using entire size-frequency distributions as response variables. We then predict coral population size structure with increasing environmental stress and variability. Together, we find fewer but larger coral colonies in marginal reefs than in tropical reefs, where environmental conditions are more variable and stressful for tropical corals. Our model predicts that coral populations may become gradually dominated by larger colonies (> 148 cm2) with increasing environmental stress. Fewer but bigger corals suggest low survival of smaller corals, slow growth, and / or poor recruitment. This finding is concerning for the future of coral reefs as it implies populations may have low recovery potential from disturbances. We highlight the importance of continuously monitoring changes to population structure over biogeographic scales.