We study the problem of providing clean drinking water to the rural poor. We implement a cluster-randomized trial supplying bottled water in a direct delivery model to households in rural Odisha, India and test three different allocation mechanisms. We randomized 151 villages — over 60,000 households — into a control group and multiple allocation arms: (i) varying prices; (ii) a free entitlement; and (iii) an exchangeable entitlement, where households are paid cash in lieu of unused allocation. We randomize which households are treated within villages. We find that households value clean water highly but appear to face constraints preventing them from paying full market prices sufficient to buy their preferred amount of water. They also make lumpy consumption choices, either consuming no water or substantial amounts, and have an upper bound on demand that is consistent with reasonable drinking needs. Finally, we find that clean water has large, positive effects on health across a variety of different measures. Overall, our findings suggest that delivering clean water may be significantly more successful than point-of-use treatment approaches, that decentralized rural bottled water delivery is sustainable at scale, and that a free entitlement is an effective and fairly efficient allocation mechanism for water.