The amount of water a baby should drink varies depending on the feeding method. Generally, it is recommended that formula-fed infants drink a small amount of boiled water between feedings, about 20 milliliters each time.