We all remember Elf for its holly jolly hilarity, but there's one moment that stands out to Ferrell for another reason