This dataset contains all the training and testing data generated from 90 videos (over seven sets of welding material stack-ups). A new method is developed to assemble sufficient datasets from these videos for neural network training. These data also contains the ground truth on the weld nuggets, derived from the post-weld measurement and video conversion ratios. More specific technical details can be found in the manuscript: Jian Zhou, Dali Wang, Jian Chen, Zhili Feng. (2019), Autonomous non-destructive evaluation of resistance Spot Welded Joints.