Various layers of the retina are well known to alter the polarization state of light. Such changes in polarization may be a sensitive indicator of tissue structure and function, and as such have gained increased clinical attention. Here we demonstrate a polarization-sensitive optical coherence tomography (PS-OCT) system that incorporates adaptive optics (AO) in the sample arm and a single line scan camera in the detection arm. We quantify the benefit of AO for PS-OCT in terms of signal-to-noise, lateral resolution, and speckle size. Double pass phase retardation per unit depth values ranging from 0.25 degrees/microm to 0.65 degrees/microm were found in the birefringent nerve fiber layer at 6 degrees eccentricity, superior to the fovea, with the highest values being noticeably higher than previously reported with PS-OCT around the optic nerve head. Moreover, fast axis orientation and degree of polarization uniformity measurements made with AO-PS-OCT demonstrate polarization scrambling in the retinal pigment epithelium at the highest resolution reported to date.