Many construction scenes feature low-light work, such as nighttime construction and tunnel construction. Poor lighting and low visibility will increase the risk of site accidents. One of the leading causes of construction accidents is unsafe worker behavior, which can be predicted via worker posture estimation. Therefore, this study proposes an Unsupervised Illumination Reflectance Estimation (UIRE-Net) framework for estimating the dark worker pose. On the basis of lightness-color consistency, in spite of ungratified illumination conditions, the “true color” of objects depends on the illumination reflectance only. The illumination reflectance estimation is monotonous to neighboring pixel differences, making the extracted features robust for worker pose estimation. In addition, the proposed UIRE-Net restores image brightness without relying on image pairs. A testing experiment based on nighttime construction workers is conducted to validate the veracity.