Differential Privacy
Background
Differential privacy (DP) [1] is a powerful theoretical criterion metric for privacy preserving in database. Specifically, a randomized mechanism satisfying $(\epsilon, \delta)$-DP promises the privacy loss of all neighboring datasets is bounded by $\epsilon$ with the probability at least $1-\delta$[1].
Differential Privacy:
A randomized algorithm $\mathcal{M}$ satisfies $(\epsilon, \delta)$-DP if for all $S \subseteq{Range(\mathcal{M})}$ and all neighboring datasets $(x, y)$:
$Pr( \mathcal{M} (x) \in{S}) \leq{exp(\epsilon) Pr(\mathcal{M} (y) \in{S})} + \delta$.
Support of DP
In federated learning scenario, noise injection and gradient clipping are foundational tools for differential privcay. In FederatedScope, DP algorithms are supported by preseting APIs in the core library, including
- noise injection in download channel (server)
- noise injection in unload channel (client)
- gradient clipping before upload
Noise Injection in Download
In server, a protected attribute _noise_injector is preset in the server class for noise injection with default value is None.
class Server(Worker):
def __init__(**kwargs):
...
# inject noise before broadcast
self._noise_injector = None
...
Developers can achieve noise injection by calling register_noise_injector. Then the function self._noise_injectorwill be called before the server broadcasts parameters.
class Server(Worker):
...
def register_noise_injector(self, func):
self._noise_injector = func
...
Noise Injection in Upload
For clients, the noise injection can be done by registering hook function at the end of training (before uploading parameters).
def noise_injection_function(ctx):
pass
trainer.register_hook_in_train(new_hook=noise_injection_function,
trigger='on_fit_end',
insert_pos=-1)
Gradient Clipping
Gradient clipping is preset in flapackage/core/trainers/trainer.py. When the function _hook_on_batch_backwardis called, the gradient will be clipped by the parameter cfg.optimizer.grad_clipin the config.
...
def _hook_on_batch_backward(self, ctx):
ctx.optimizer.zero_grad()
ctx.loss_task.backward()
if ctx.grad_clip > 0:
torch.nn.utils.clip_grad_norm_(ctx.model.parameters(),
ctx.grad_clip)
ctx.optimizer.step()
...
Threshold of gradient clipping in federatedscope/config.py.
# ------------------------------------------------------------------------ #
# Optimizer related options
# ------------------------------------------------------------------------ #
cfg.optimizer = CN()
cfg.optimizer.type = 'SGD'
cfg.optimizer.lr = 0.1
cfg.optimizer.weight_decay = .0
cfg.optimizer.grad_clip = -1.0 # negative numbers indicate we do not clip grad
Implementation of DP
NbAFL [2] is a DP algorithm designed for federated learning, which protects both the upload and download channels with $(\epsilon, \delta)$-DP. Taking NbAFL as an example, we show how to implement DP algorithm in FederatedScope.
Prepare DP Parameters
Add parameters into federatedscope/config.py. Note FederatedScope supports at most two levels of config, e.g., cfg.data.type.
# ------------------------------------------------------------------------ #
# nbafl(dp) related options
# ------------------------------------------------------------------------ #
cfg.nbafl = CN()
# Params
cfg.nbafl.use = False
cfg.nbafl.mu = 0.
cfg.nbafl.epsilon = 100.
cfg.nbafl.w_clip = 1.
cfg.nbafl.constant = 30.
Prepare DP Functions
Then developers should design their own DP functions. For NbAFL, the following three hook functions are required for clientTrainer. The functionsrecord_initializationand del_initializationmaintain the initialization received from the server, and inject_noise_in_uploadinjects noise into the model before upload.
def record_initialization(ctx):
ctx.weight_init = deepcopy(
[_.data.detach() for _ in ctx.model.parameters()])
def del_initialization(ctx):
ctx.weight_init = None
def inject_noise_in_upload(ctx):
"""Inject noise into weights before the client upload them to server
"""
scale_u = ctx.nbafl_w_clip * ctx.nbafl_total_round_num * 2 * ctx.nbafl_constant / ctx.num_train_data / ctx.nbafl_epsilon
# logging.info({"Role": "Client", "Noise": {"mean": 0, "scale": scale_u}})
for p in ctx.model.parameters():
noise = get_random("Normal", p.shape, {
"loc": 0,
"scale": scale_u
}, p.device)
p.data += noise
For the server, the following function is created for noise injection.
def inject_noise_in_broadcast(cfg, sample_client_num, model):
"""Inject noise into weights before the server broadcasts them
"""
if len(sample_client_num) == 0:
return
# Clip weight
for p in model.parameters():
p.data = p.data / torch.max(torch.ones(size=p.shape),
torch.abs(p.data) / cfg.nbafl.w_clip)
if len(sample_client_num) > 0:
# Inject noise
L = cfg.federate.sample_client_num if cfg.federate.sample_client_num > 0 else cfg.federate.client_num
if cfg.federate.total_round_num > np.sqrt(cfg.federate.client_num) * L:
scale_d = 2 * cfg.nbafl.w_clip * cfg.nbafl.constant * np.sqrt(
np.power(cfg.federate.total_round_num, 2) -
np.power(L, 2) * cfg.federate.client_num) / (
min(sample_client_num.values()) * cfg.federate.client_num *
cfg.nbafl.epsilon)
for p in model.parameters():
p.data += get_random("Normal", p.shape, {
"loc": 0,
"scale": scale_d
}, p.device)
Register DP Functions
The wrap function wrap_nbafl_trainer initializes parameters related to NbAFL and registers the above hook functions.
def wrap_nbafl_trainer(
base_trainer: Type[GeneralTrainer]) -> Type[GeneralTrainer]:
"""Implementation of NbAFL refer to `Federated Learning with Differential Privacy: Algorithms and Performance Analysis` [et al., 2020]
(https://ieeexplore.ieee.org/abstract/document/9069945/)
Arguments:
mu: the factor of the regularizer
epsilon: the distinguishable bound
w_clip: the threshold to clip weights
"""
# ---------------- attribute-level plug-in -----------------------
init_nbafl_ctx(base_trainer)
# ---------------- action-level plug-in -----------------------
base_trainer.register_hook_in_train(new_hook=record_initialization,
trigger='on_fit_start',
insert_pos=-1)
base_trainer.register_hook_in_eval(new_hook=record_initialization,
trigger='on_fit_start',
insert_pos=-1)
base_trainer.register_hook_in_train(new_hook=del_initialization,
trigger='on_fit_end',
insert_pos=-1)
base_trainer.register_hook_in_eval(new_hook=del_initialization,
trigger='on_fit_end',
insert_pos=-1)
base_trainer.register_hook_in_train(new_hook=inject_noise_in_upload,
trigger='on_fit_end',
insert_pos=-1)
return base_trainer
Finally, in federatedscope/core/auxiliaries/trainer_builder.py, the function get_trainerwraps the basic trainer with NbAFL variables and functions.
def get_trainer(model=None,
data=None,
device=None,
config=None,
only_for_eval=False,
is_attacker=False):
...
# differential privacy plug-in
if config.nbafl.use:
from federatedscope.core.trainers.trainer_nbafl import wrap_nbafl_trainer
trainer = wrap_nbafl_trainer(trainer)
...
Run an Example
Run the following command to call NbAFL on the dataset Femnist.
python federatedscope/main.py --cfg federatedscope/cv/baseline/fedavg_convnet2_on_femnist.yaml \
nbafl.mu 0.01 \
nbafl.constant 1 \
nbafl.w_clip 0.1 \
nbafl.epsilon 10
Evaluation
Take the dataset Femnist as an example, the accuracy with different $(\epsilon, \delta)$-DP is shown as follows.
| Task | $\epsilon$ | $\delta$ | Accuracy(%) |
|---|---|---|---|
| FEMNIST | 10 | 0.01 | 11.73 |
| 10 | 0.17 | 24.82 | |
| 10 | 0.76 | 41.71 | |
| 50 | 0.01 | 54.85 | |
| 50 | 0.17 | 67.98 | |
| 50 | 0.76 | 80.58 | |
| 100 | 0.01 | 74.80 | |
| 100 | 0.17 | 80.39 | |
| 100 | 0.76 | 80.58 |
References
[1] Dwork C, Roth A. “The Algorithmic Foundations of Differential Privacy”. Foundations and Trends in Theoretical Computer Science, 2014, 9(3-4): 211-407.
[2] Wei K, Li J, Ding M, et al. “Federated Learning With Differential Privacy: Algorithms and Performance Analysis”. IEEE Transactions on Information Forensics and Security, 2020, 15: 3454-3469.