The language environment to which children are exposed has an impact on later language abilities as well as on brain development; however, it is unclear how early such impacts emerge. This study investigates the effects of children’s early language environment and socioeconomic status (SES) on brain structure in infancy at 6 and 30 months of age (both sexes included). We used magnetic resonance imaging to quantify concentrations of myelin in specific fiber tracts in the brain. Our central question was whether Language Environment Analysis (LENA) measures from in-home recording devices and SES measures of maternal education predicted myelin concentrations over the course of development. Results indicate that 30-month-old children exposed to larger amounts of in-home adult input showed more myelination in the white matter tracts most associated with language. Right hemisphere regions also show an association with SES, with older children from more highly educated mothers and exposed to more adult input, showing greater myelin concentrations in language-related areas. We discuss these results in relation to the current literature and implications for future research.