A new neuro-symbolic framework, LGM, is proposed for disentangling long-term memory in personalized agents. It uses a latent graph construction with a sparse autoencoder to map historical interactions into a continuous latent space, enabling effective activations and outperforming state-of-the-art baselines in capturing explicit and implicit preferences.