The railway transportation plays an important role in the economic development since it has ability to transport large volumes of passengers and cargo through great distances. Therefore, monitoring the condition of railroad is essential to ensure train safety and has drawn much attention of researchers recently. With the development of computer vision the railway non-destructive inspection systems become possible. However, these methods have been still challenged by so much obstacles from ambient light on the rail surface and defects themselves. Defects appear on high-speed rail surface are variety in which corrugation type usually in heterogeneous shape and size. In essence, the most visible characteristic of this type of defect is that they are usually in bright color, with shadows cast along the length of the rail track. Thus the local gray level-based methods for detecting this type of defect perform most effectively. This paper proposes a detecting and evaluating corrugation defect method for high speed railway inspection system mainly based on histogram curves on both vertical and horizontal direction. The preliminary results achieved from the above analysis are combined in a single image to eliminate false positive defects and noises. The detected defects is last classified into two main categories to assess the level of danger of damage. Our experimentals achieved the precision of with-defect rail detection reaches 98.9% and the correct classification of defects is 97.5%.