Exactly what maxgoof said. We got around having PMI by getting the rest of our mortgage in a line of credit, knowing we could pay it off later, sooner. That unfortunately had a fluctuating APR, and it kept going up, slowly, over time. I got an offer to change that to a secured line of credit card debt, but at a much lower fixed rate. We took a hit to the amount we deducted, yes - but the interest per year was more than halved. The tax hit on the interest was maybe $50, but the interest was 20 times that.

Paying off debt is *always* good, unless the rate is incredibly low, or you haven't put away for retirement. By the way - put stuff away for retirement if you haven't yet, you'll want that to pay for real estate taxes when you DO retire :)