there should be a 'badge trophy case' so when a badge is removed (for badges that are temporary) it'll say somewhere on the profile all the previous badges the user had.
Thanks for sharing your idea! It was forwarded to related team.