With Google, Apple and Amazon racing to embed voice services across our devices, conversational user interfaces will soon become ubiquitous. But the shift from screens to voice unlocks a new set of concerns for personalization, distinct from Netflix's recommendations or Google's filter bubbles. As advances in natural language understanding and affective computing take hold, assistants will rely on a vast, textured data set of choices, memories and desires in the past to condition our lives in the future. How can we navigate this tradeoff between personalization and emotional growth, and design successful personal assistants that still leave space for the user to grow, change and forget?