After sitting through the ceremony in the third row, the Duke of Sussex swiftly left Westminster Abbey to fly back to the US