Automatically detecting and monitoring urban regions is an important problem in remote sensing. Very high resolution aerial and satellite images provide valuable information to solve this problem. However, they are not sufficient alone for two main reasons. First, a human expert should analyze these very large images. There may be some errors in the operation. Second, the urban area is dynamic. Therefore, detection should be done periodically, and this is time consuming. To handle these shortcomings, an automated system is needed to detect the urban area from aerial and satellite images. In this letter, we propose such a method based on local feature point extraction using Gabor filters. We use these local feature points to vote for the candidate urban areas. Then, we detect the urban area using an optimal decision-making approach on the vote distribution. We test our method on a diverse panchromatic aerial and Ikonos satellite image set. Our test results indicate the possible use of our method in practical applications.