This paper aims at clearing up some misunderstandings about the social impacts of carbon taxes that proved to be a decisive obstacle to their further consideration in public debates. It highlights the gap between the cost of a carbon tax reform as it is spontaneously perceived by the taxpayers and the reality of its ultimate consequences: the real impact on households' poverty and inequalities is not mechanically determined by the initial burden of energy on consumption budgets and by the capacity of households to alleviate it, but also depends upon the use made of the tax proceeds and its general macroeconomic impacts. The comparison of five taxrecycling schemes highlights the existence of trade-offs between maximising total consumption, reducing unemployment, maximising the consumption of the lowincome classes and reducing income inequality.