They never apologized for what they did and told him to "let go of the past to move forward in a positive way"