Accidents frequently take place in the construction industry. Workers’ unsafe behaviour is one of the main causes of accidents. However, traditional measures for unsafe behaviour management mainly rely on manual methods and show limited effects. In recent years, computer vision is becoming a promising method for automatically recognising workers’ unsafe behaviour. This chapter presents a computer vision (image)–based approach to automatically recognising workers’ unsafe behaviour. Firstly, dynamic unsafe behaviour is simplified as a static leading posture. Then the posture is simplified as vital joint parameters according to the mathematical model of a human skeleton. Finally, unsafe behaviour is detected and identified if its parameter values match those in the database, which is established as criteria to describe unsafe behaviour with several parameters. As a result, complex behaviour is simplified as several parameter values, and the identification time is significantly shortened. Moreover, an experiment was executed to demonstrate the feasibility, efficiency, and accuracy of the proposed method. The result shows that the method is able to identify one of these unsafe behaviours and differentiate them in an efficient way, thus having the potential to support workers’ behaviour management on construction sites.