CRFU: Compressive Representation Forgetting Against Privacy Leakage on Machine Unlearning

Assuming $D$ represents the original training dataset and $e$ is the sample a user requests to unlearn, we denote $D_{r}=D\backslash\{e\}$ as the remaining dataset. If there are several erased samples, we refer to the erased dataset as $D_{e}$. Machine unlearning aims to erase the influence of the specified dataset $D_{e}$ from a trained ML model $\mathcal{M}(D)$, where the model is trained based on the full training dataset $D$. The unlearning update is performed by executing an unlearning algorithm $\mathcal{U}$ on the current trained model using the erased dataset. More formally, given a erased dataset $D_{e}$, a trained ML model $\mathcal{M}$, and the unlearning algorithm $\mathcal{U}$, the unlearning process can be defined as $\mathcal{U}:D_{e},\mathcal{M}(D)\to\mathcal{M}_{u}(D\backslash D_{e})$, where $\mathcal{M}_{u}$ is the unlearned version model $\mathcal{M}$.

The goal of privacy leakage attacks on machine unlearning is to infer the privacy of the erased samples $D_{e}$ of an unlearning update. We assume the adversary can only have black-box access to the target model. This limitation means that the adversary can interact with the model solely through queries using a specific set of data samples, known as the probing set, and subsequently receive the corresponding outputs. Furthermore, it is presumed that the adversary’s local probing dataset is sourced from the same distribution as that of the target model’s training dataset. Moreover, we consider that the adversary has knowledge of both the unlearning and original learning algorithms and has the capability to establish the same learning and unlearning training as the target model. This can be achieved by performing model hyperparameter stealing attacks [26]. Finally, and most importantly, we assume that the target model effectively removes the information of the erased data, and the erased dataset and the remaining dataset are disjoint.

In privacy inference attacks, the adversary’s initial step involves collecting varied outputs from their probe data $X_{probe}$. This includes obtaining the original model outputs $\hat{Y}$ before unlearning, as well as the outputs $\hat{Y}_{-(X_{e},Y_{e})}$ after unlearning. The key to the attack lies in the difference $\delta=\hat{Y}_{-(X_{e},Y_{e})}-\hat{Y}$, which is used to train the attack model. The structure of the attack model typically includes an encoder and a decoder, resembling the architecture of Variational Autoencoders (VAEs) [27]. We replicate the privacy inference attacks following the methodology outlined by [10], and the detailed attack process is depicted in Figure 2. The direct victims are those users who request for unlearning. Their unlearning requests prompt the ML server to execute an unlearning update, which is leveraged by this kind of attack to infer the privacy of the erased data $(X_{e},Y_{e})$.

We begin by briefly introducing the core concept of the proposed CRFU, which is illustrated in Figure 3. CRFU is tailored to unlearn models that were trained using the IB method (shown in the upper half in Figure 3). An IB method mainly learns a representation $Z$ that compresses information of inputs while retaining sufficient information for the targets. The CRFU, the lower half in Figure 3, aims to erase the influence of $D_{e}=(X_{e},Y_{e})$ from $Z$. To achieve this goal, we minimize the mutual information $I(X_{e};Z)$ and $I(Y_{e};Z)$; however, directly minimizing them could result in the representation $Z$ catastrophically forgetting everything, decreasing model utility. To mitigate the degradation of model utility, we introduce a constraint ensuring that the unlearned representation $Z$, derived from the distribution $p(Z|X_{-X_{e}})$, should be close to the representation $Z$ obtained from the original-fixed distribution $p(Z|X)$. We deal with the predicting approximation $\hat{Y}$ in the same way. Finally, combining the minimizing mutual information and a constraint, we formalize the losses $\mathcal{L}^{u}_{rep}$ and $\mathcal{L}^{u}_{app}$ to unlearn the trained IB model.

In an IB model, the objective is to learn a representation $Z$ by minimizing the mutual information $I(X;Z)$, while simultaneously maximizing $I(Y;Z)$. In our CRFU approach, we specifically focus on minimizing $I(X_{e};Z)$ for the inputs $X_{e}$ designated for erasure. This is aimed at effectively eliminating the information of $X_{e}$ from the previously learned representation $Z$. Similarly, we work on minimizing $I(Y_{e};Z)$, the mutual information between the targets $Y_{e}$ and the representation $Z$, thereby removing the information of $Y_{e}$ from $Z$. This dual minimization ensures a comprehensive unlearning of both input and label information from the representation.

The ideal unlearned representation should contain no information about $(X_{e},Y_{e})$, optimally satisfying the conditions where $I(X_{e};Z)=0$ and $I(Y_{e};Z)=0$, indicating zero mutual information between the erased data and the representation $Z$. However, directly minimizing $I(X_{e};Z)$ and $I(Y_{e};Z)$ poses a significant risk: it could lead the representation $Z$ to inadvertently “forget” everything, including the valuable knowledge it had previously learned. Therefore, we must keep the unlearned representation $Z$ and approximation $\hat{Y}$ remembering the knowledge learned in the former training.

To maintain the learned knowledge, we propose a solution that minimizes the Kullback–Leibler divergence (KLD) [33] between the unlearned and originally learned representations. This involves minimizing both $\text{KL}[p(Z|X)||p(Z|X_{-X_{e}})]$ and $\text{KL}[p(\hat{Y}|Z)||p(\hat{Y}_{-(X_{e},Y_{e})}|Z)]$, ensuring that the unlearned representations remain closely aligned with the original ones. During unlearning, $p(Z|X)$ and $p(\hat{Y}|Z)$ are calculated using the trained IB model as a temporary reference model. Combining the above solutions, we formalize the unlearned representation loss as

and the unlearned approximation loss as

Integrating these two loss functions, we formulate the optimization of CRFU as the proposition outlined below.

Since CRFU unlearns an IB model trained through Eq. 2, we can combine the original IB loss and CRFU loss to prove that minimizing Eq. 7 based on a trained IB model is equivalent to minimizing the loss of retraining an IB model on the remaining dataset. We first prove the representation loss. Given that $X=X_{r}\cup X_{e}$ and $X_{r}\cap X_{e}=\emptyset$, and considering that the data are IID, along with the conditional independence of $X_{r}$ and $X_{e}$ given $Z$, it follows that $I(X;Z)=I(X_{r};Z)+I(X_{e};Z)$. Since $I(X_{e};Z)\geq 0$ and $\text{KL}[p(Z|X)||p(Z|X_{-X_{e}})]\geq 0$, we can expand original and unlearned representation loss $\mathcal{L}_{rep}+\mathcal{L}_{rep}^{u}$ function as

It is clear that the sum of the representation loss $\mathcal{L}_{rep}$ and the unlearned representation loss $\mathcal{L}_{rep}^{u}$ serves as an upper bound for the mutual information $I(X_{r};Z)$. Minimizing this upper bound during both original training and unlearning is equivalent to minimizing $I(X_{r};Z)$ itself.

Similarly, for the original and unlearned approximation loss function $\mathcal{L}_{app}+\mathcal{L}_{app}^{u}$, we can obtain the expanded approximation as

Since $Y=Y_{r}\cup Y_{e}$ and $Y_{r}\cap Y_{e}=\emptyset$ and the data are IID, it follows that $I(Y_{r};Y_{e})=0$ and $H(Y)=H(Y_{r},Y_{e})=H(Y_{r})+H(Y_{e})-I(Y_{r};Y_{e})=H(Y_{r})+H(Y_{e})$. We can similarly achieve $H(Y_{r}|Z)$. And since $I(Y_{r};Z)=H(Y_{r})-H(Y_{r}|Z)$, thus Eq. 9 holds. Minimizing the original and unlearned approximation as Eq. 9 is the upper bound of $-I(Y_{r};Z)$ in retraining. Therefore, minimizing the CRFU loss function based on a trained IB model is equivalent to minimizing the loss function of retraining an IB model based on the remaining dataset.

Optimizing the CRFU loss function Eq. 7 in a deep learning scenario with a huge amount of training data is challenging. To address this issue, we present a variational method to make the CRFU loss function calculatable in deep learning. The unlearned representation loss can be expanded as

Generally, minimizing this loss is intractable due to the complexity involved in computing the KL term. This computation requires knowledge of the marginal distribution $Z$ and $p_{\theta^{r}}(Z)=\int dx\ p(z|x)p(x)$, which is not easily obtainable. To address this issue, we introduce $q(Z)$ as a variational approximation to this marginal as the variational IB learning in [30, 31]. Since $\text{KL}[p(Z)||q(Z)]\geq 0\Longrightarrow\int dzp(z)\log p(z)\geq\int dzp(z)\log q(z)$, we have the following upper bound of Eq. 10:

This unlearned representation loss can be explained as forgetting the impact of the erased data’s inputs (first term) and remembering the knowledge of original full training inputs (second term). The proof of the correctness of Eq. 11 is similar to the proof of Eq. (14) in [30]. Hence, we omit the detailed proof here.

Following the solutions to $\mathcal{L}_{app}$ in [30, 31], the unlearned approximation loss function can be optimized in a similar way as

The computable optimization loss Eq. 12 represents a balance between completely unlearning the information of the erased labels $Y_{e}$ (first term) and not entirely forgetting the originally learned information of $Y$ in representation $Z$ (second term). Combining Eqs. 12 and 11 and optimizing them together is the CRFU loss Eq. 7 that we proposed in proposition 1. When minimizing this loss, we should notice that the second term in Eqs. 12 and 11 is calculated based on the original full training dataset; however, storing all original datasets and training it again is impractical. A simple way we used is fixing the trained model (i.e., the model before unlearning) as a temp model that is specially used to calculate the first term but only based on the erased dataset.

To make the unlearning method adaptive to different tasks, we introduced an unlearning rate parameter, $\beta_{u}$, which serves to regulate the extent of unlearning during both the representation and approximation unlearning process. Specifically, we add an unlearning rate parameter $\beta_{u}$ before the forgetting terms in Eqs. 11 and 12 to optimize the balance between forgetting the information of erased samples and remembering the representation learned before. Adjusting the unlearning rate $\beta_{u}$ will also impact the unlearning speed. The final equation can be described as

We present a pseudocode of CRFU in Algorithm 1. At the beginning of executing representation forgetting unlearning, we first prepare a trained IB model $\mathcal{M}(\theta^{r},\theta^{a})$, the erased dataset $D_{e}=(X_{e},Y_{e})$, and training epochs $E$. Then, we fix the original optimal model $\mathcal{M}_{fix}(\theta^{r}_{fix},\theta^{a}_{fix})\leftarrow\mathcal{M}(\theta^{r},\theta^{a})$ as a temp model for the later unlearned representation and approximation loss calculation, as shown in Line 1 of Algorithm 1. When executing unlearning training, lines 2 to 7, we draw a minibatch of $m$ data samples $\{(x_{i},y_{i})\}_{i=1}^{m}$ from $D_{e}$. Using the representer $\theta^{r}$, we generate the corresponding representations $z_{i}$ for these samples. The unlearning loss function is then calculated as per Eqs. 11 and 12. Following this, we update the parameters of the representer and approximator $(\theta^{r},\theta^{a})$ in the IB model in accordance with Eq. 13 at line 7.