Cerebral white matter abnormalities relate to cognitive functioning in elders. We examine whether this association is (a) independent of mental ability in youth and (b) related to general and/or specific mental abilities. We retested 83 participants of the Scottish Mental Survey of 1932 on a battery of mental tests. Their brains were scanned by magnetic resonance imaging. Three independent ratings (Fazekas) were made of periventricular, and subcortical and deep white matter abnormalities. Structural equation models showed that, irrespective of brain location, white matter abnormalities contributed about 14% of cognitive function variance in old age. Some of this effect might be due to hypertension. This contribution is independent of mental function in early life and is associated with general cognitive ability.