Few investigators have studied cognition over time in adults with heart failure (HF). A battery of neuropsychological tests was administered to 279 adults with chronic systolic or diastolic HF at baseline, three and six months. Growth mixture modeling (GMM) was used to model the measure anticipated to be most sensitive, the digit symbol substitution task (DSST). We describe how and why the DSST patterns change over time. Other measures of cognition were examined to identify consistency with the DSST patterns. The sample was predominantly male (63.2%), Caucasian (62.7%), mean age 62 years. The best fit GMM revealed two trajectories of DSST scores: Average processing speed group (40.5%) and Below Average processing speed (59.9%). Neither group changed significantly over the six month study. Other measures of cognition were consistent with the DSST patterns. Factors significantly associated with increased odds of being in the Below Average processing speed group included older age, male gender, Non-Caucasian race, less education, higher ejection fraction, high comorbid burden, excessive daytime sleepiness, and higher BMI. As some of the factors related to cognitive impairment are modifiable, research is needed to identify interventions to preserve and improve cognition in these patients.