As you age, collagen and elastin production in the skin reduces, resulting in the skin losing its elasticity. This is the major factor which contributes to sagging skin. Other external factors are smoking, constant exposure to the sun and rapid weight loss which can age your skin prematurely.