Even though men in their 50s don't have the same rapid bone loss women do, that changes around 65. Then, men lose bone mass at the same rate, and recovery from fracture is riskier for men. Most surprising? The risk of an osteoporotic fracture in men is higher than the risk of prostate cancer.