As the size of a high contrast drifting Gabor patch increases, perceiving its direction of motion becomes harder. However, the same behavioral effect is not observed for a low contrast Gabor patch. Neuronal mechanisms underlying this size-contrast interaction are not well understood. Here using psychophysical methods and functional magnetic resonance imaging (fMRI), we investigated the neural correlates of this behavioral effect. In the behavioral experiments, motion direction discrimination thresholds were assessed for drifting Gabor patches with different sizes and contrasts. Thresholds increased significantly as the size of the stimulus increased for high contrast (65%) but did not change for low contrast (2%) stimuli. In the fMRI experiment, cortical activity was recorded while observers viewed drifting Gabor patches with different contrasts and sizes. We found that the activity in middle temporal (MT) area increased with size at low contrast, but did not change at high contrast. Taken together, our results show that MT activity reflects the size-contrast interaction in motion perception.