The central point is the hypocrisy of the Pharisees, not duties to government, though the latter is mentioned.
Christ is intentionally being unhelpful and saying less than more, since He knows it's a trap question. Kind of like I've jokingly said to an temp employee at work when helping him work through an issue, "The problem is that you're doing it wrong. You need to start doing it right." Yeah, like he didn't know that already.
Of course, the coin had Caesar's image on it, and even called him Pontifex. So if it's wrong to pay taxes to Caesar, why do they take the benefit of his currency anyway? Pretty cheap of them. So, let them stop paying taxes and see how well they can live with that. And if it's right to pay taxes, well, they have the benefit of his governance, so of course it's right to pay those taxes. The proof is that they have Caesar's money in their hand... and probably in the Temple as well. Either way, they are hypocrites for asking the question.
It's Christ the Lamb being tested, and springing the trap.