The remarkable variation in floral form and size is thought to have driven the evolutionary radiation of angiosperms and has traditionally been attributed to pollinator-mediated selection on floral displays, yet emerging evidence indicates that abiotic environmental factors can also generate and filter variation in flower size. Here, we investigate the geographic variation in reproductive organ growth in the self-fertilising species Arabidopsis thaliana, where the loss of pollinator dependence is predicted to favour reduced floral investment through resource reallocation. We find extensive variation in flower size underpinned by a polygenic architecture, in which derived alleles both increase or decrease petal size and show signatures of positive selection. Nevertheless, the evolutionary direction of flower size differs across geographic regions, reflecting environmentally mediated shifts in the relationship between flower size and seed production. Purifying selection favours small flowers at the climatic margins of the species, but is relaxed in more suitable habitats allowing the emergence and persistence of both small and large-flower variants. This biogeographic pattern extends to other life-history traits, highlighting how environmental heterogeneity and constraints shape evolutionary trajectories and maintain phenotypic diversity in selfing lineages.