Failure to pay taxes? IRS consequences of filing late or never filing at all To often, taxpayers owe taxes at the end of the year and decide not to file.