Climate change is projected to sharply reduce agricultural productivity in hot developing countries and raise it in temperate regions. Reallocation of labor across sectors could temper the aggregate impacts of these changes if hotter regions shift toward importing food and specializing in manufacturing or exacerbate them if subsistence food requirements push labor toward agriculture where its productivity suffers most. I quantify these effects in two steps. First, I project changes in global comparative advantage by using firm-level micro-data from 17 countries covering over half the world’s population to estimate the heterogeneous effect of temperature on output per worker in manufacturing and services. I find large effects of extremely hot and cold temperatures on non-agricultural output per worker, but treatment effects diminish with income and expectations of temperature such that the projected impact of climate change is larger in agriculture than non-agriculture. Second, I embed my estimates in an open-economy model of structural transformation that matches moments on output-per-worker, sectoral specialization, and trade for 158 countries. Simulations suggest that subsistence food requirements dominate labor reallocation in response to climate change on average and the global decline in GDP is 12.0% larger, and 52.1% larger for the poorest quartile of the world, when accounting for sectoral reallocation than in the counterfactual with fixed sectoral shares. The aggregate willingness-to-pay to avoid climate change is 1.5-2.7% of annual GDP and 6.2-10.0% for the poorest quartile. Trade reduces the welfare costs of climate change relative to autarky by only 7.4% under existing policy, but by 30.7% overall and by 68.2% for the poorest quartile in an alternative scenario with reduced trade costs.