We developed a high-speed polarization sensitive optical coherence tomography (PS-OCT) system for retinal imaging based on spectral domain OCT. The system uses two spectrometers, one for each polarization channel, that operate in parallel at 20000 A-lines/s each. It provides reflectivity, retardation, and cumulative optic axis orientation simultaneously. We present our instrument and discuss the requirements for the alignment of the two spectrometers specific for our setup. We show 2D spectral domain PS-OCT images and - to the best of our knowledge - the first 3D spectral domain PS-OCT data sets in form of fly-through movies and volume rendered data sets recorded in human retina in vivo.