You have no income tax liability, but depending upon the size of the remaining balance you may face federal gift tax issues. Your son has no tax liability; your generosity in paying off his mortgage is a gift to him.
Payments made to a separated or divorced spouse as required by a decree or agreement. Qualifying payments are deductible by the payor and taxable to the payee.