Among those with HIV, anemia is a strong risk factor for disease progression and death independent of CD4 count and viral load. Understanding the role of anemia in HIV treatment is critical to developing strategies to reduce morbidity and mortality. We conducted a prospective analysis among 10,259 HIV-infected adults initiating first-line ART between April 2004 and August 2009 in Johannesburg, South Africa. The prevalence of anemia at ART initiation was 25.8%. Mean hemoglobin increased independent of baseline CD4. Females, lower BMI, WHO stage III/IV, lower CD4 count, and zidovudine use were associated with increased risk of developing anemia during follow-up. After initiation of ART, hemoglobin improved, regardless of regimen type and the degree of immunosuppression. Between 0 and 6 months on ART, the magnitude of hemoglobin increase was linearly related to CD4 count. However, between 6 and 24 months on ART, hemoglobin levels showed a sustained overall increase, the magnitude of which was similar regardless of baseline CD4 level. This increase in hemoglobin was seen even among patients on zidovudine containing regimens. Since low hemoglobin is an established adverse prognostic marker, prompt identification of anemia may result in improved morbidity and mortality of patients initiating ART.