There is tremendous interest in understanding the role that secondary structure plays in the rigidity and dynamics of proteins. In this work we analyze nanomechanical properties of proteins chosen to represent different secondary structures: α-helices (myoglobin and bovine serum albumin), β-barrels (green fluorescent protein), and α + β + loop structures (lysozyme). Our experimental results show that in these model proteins, the β motif is a stiffer structural unit than the α-helix in both dry and hydrated states. This difference appears not only in the rigidity of the protein, but also in the amplitude of fast picosecond fluctuations. Moreover, we show that for these examples the secondary structure correlates with the temperature- and hydration-induced changes in the protein dynamics and rigidity. Analysis also suggests a connection between the length of the secondary structure (α-helices) and the low-frequency vibrational mode, the so-called boson peak. The presented results suggest an intimate connection of dynamics and rigidity with the protein secondary structure.