Rinse the jasmine rice in a fine-mesh strainer until the water runs mostly clear, about 30 seconds, to remove excess starch.
In a heavy-bottomed pot, combine rinsed rice, coconut milk, whole milk, sugar, and salt. Bring to a gentle simmer over medium heat, stirring frequently to prevent sticking.
Once simmering, reduce heat to low and cook, covered, for 15 minutes, stirring every 5 minutes. The rice will absorb the liquid and become creamy.
After 15 minutes, remove the lid and stir constantly as the mixture thickens, about 10-15 more minutes, until the rice is tender and the pudding is creamy but still a bit loose.
Remove from heat and stir in butter, vanilla extract, and cardamom for added silkiness.
Let the pudding cool for 10 minutes. Dice the mangos into 1/2-inch cubes during this time.
Once cooled slightly, fold in three-quarters of the diced mango, saving the rest for topping.
Spoon pudding into bowls while warm, topping with the remaining mango and toasted coconut flakes if desired.