Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

AAAI 2021 Workshop,Chen Hang - Department of Computer and Information Sciences, University of Delaware, USA

Code:https://github.com/hanglearning/VBFL

VBFL Talk Public Presentation

Youtube:https://www.youtube.com/watch?v=LMseEXEITvw

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Issue in BlockFL

  • 没有验证本地模型更新
  • PoW

    • 实践中严重的延迟
    • 能量低效的
    • 在选择一个合法设备与恶意设备作为winning miner之间 无所谓

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Introduction to VBFL

  • 向系统中添加了 validators
  • 验证机制

    • validators为本地模型更新投票
    • miners 为聚合结果投票
    • 将连续识别为潜在恶意的设备加入黑名单

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Overall operations of VBFL

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<u>Attributes of device</u>

以下划线_ 开头的属性是 私有的(每个设备私有),其他属性对网络是公共的

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Process of VBFL

  • 1.1.1Workers’ local model updates
  • 1.1.2Workers calculate expected rewards for learning L

$$ r^w = \sum ^{le^w}_{i=1}|train^w| $$

  • 1.2 W send worker-transactions to V ($wtx^w = {L^w, r^w, device\_id^w, signature}$)
  • 2.1 Receive and broadcast worker-transactions to other validators
  • 2.2.1 Verify signature of worker-transaction

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

ValidateByVotingMechanism($L_w$) 在下一部分阐述

PS:对于$is\_verified is False$的情况,本文为了简单起见,假设所有的交易签名都是有效的

  • 2.2.2 Vote for local model update
  • 2.2.3 Validators forms validator-transaction
  • 2.2.4 Validators send transactions to miners
  • 3.1 Miners broadcast validator-transactions
  • 3.2 Miners verify and aggregate voting results
  • 3.3 Miners create candidate block and mine the block
  • 3.4 Miners propagates mined blocks
  • 3.5 Miner picks legitimate block
  • 3.6 Miners append legitimate block to their own blockchain
  • 3.7 Miners requests associated devices to download its appended block
  • last phase appended block processing

ValidateByVotingMechanism($L_w$)

Vote for local model updates by the validators

验证机制(如Introduction to VBFL 部分所述)

  • <u>validators为本地模型更新投票</u> (Vote for local model updates by the validators)
  • miners 为聚合结果投票(Voting results aggregation by the miners)
  • 连续识别为潜在恶意的设备加入黑名单

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

2.2.2 Vote for local model update

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

PS:==vad==是validation accuracy difference 的缩写。$L^{w2}$存储在交易$wtx^{w2}$中,$L^{w2}$是w2做本地更新后的模型

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

VBFL validator验证目标是:检验$L^{w2}$是否受到攻击,转化为==$A^{w2}(last_G)-A^{w2}(L^{w2})与vh^{v1}$的大小关系==(而v1也可能受到攻击或者是一个不合法的学习者,这里是则是decentralized validation mechanism发挥作用)。 但是v1只有$last_G与L^{w2}$,所以v1能得到$A^{v1}(last_G)与A^{v1}(L^{w2})$ 作为它们的替代值,进而用$A^{v1}(last_G)-A^{v1}(L^{w2})$ 近似 $A^{w2}(last_G)-A^{w2}(L^{w2})$

如果 $A^{v1}(last_G)-A^{v1}(L^{w2})>vh^{v1}$(其中$vh^{v1}$表示验证精度阈值,$vh^{v1}\in[0,1]$),v1将$L^{w2}$视作不合法的学习 并 投反对票(Negetive vote

这里的假设条件:==v1和w2的数据集服从相似的分布==

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

但是因为随着一轮轮的训练,准确率差值呈上升趋势,所以很难用固定的阈值vh来 评估它 (failed)

Why this fail?

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

MNIST_CNN 有两个卷积层,每个层后面有最大池化层、 然后是全连接层 和 soft层

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<center> 对应论文的fig 3,Vanilla FL一台经典设备的连续学习曲线(前10个回合),横州gx表示第x回合的全局模型,纵轴表示对应的准确率。标有le的所有其他模型代表本地更新的模型, xle1, xle5分别表示设备使用第x-1个全局模型经过1个, 5个训练回合后对应的本地模型。测试数据集是整个MNIST dataset。 图中显示的设备学习准确率变化是合法的,即全局模型与本地模型对应的准确率的差值是小于某个阈值的(但是 基于全局模型训练的第一个回合会出现准确率下降【a sharp accuracy drop】,如下图的紫色线条)。
</center>

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

从上图的绿色圈起的部分观察得知,本地更新后的模型准确率 可能出现提升 、持平以及下降 这些情况。

An alternative attempt

通过上面的观察,一种可能的解决方法是 避免使用$A^v(G_{last})$,而是采用 $A^v(L^v(1))$ 即 validator v在使用全局模型$G_{last}$做一轮合法本地训练后的本地模型$L^v(1)$的准确率来代替。此处假设w也是合法的学习$L^w(1)$

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • 2.2.2 Vote for local model update (使用替代方法

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Generic implementation and An example

核心思想:比较$vad=A^{v1}(L^{v1}(1))-A^{v1}(L^{w2})$ 与 $vh^{v1}$,如果$vad>vh^{v1}$,则认为worker是恶意的,投票为 Negative,否则投票为 Positive

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Determination of $vh^v$?($vh^v$的确定)

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

角色分配:12 workers,5 validators,3 miners。这与论文图1相关。15%的malicious devices就足以使得Vanilla FL的准确率下降很多

role-switching policy实现如下:

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Plot of vad

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

从上图发现 $vh^v=0.08$时,可以筛选出大部分 malicious workers(并丢弃对应的本地模型更新)

$vh^v=0.08$

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Limitation

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Voting results aggregation by the miners

验证机制(如Introduction to VBFL 部分所述)

  • <u>validators为本地模型更新投票</u> (Vote for local model updates by the validators)
  • <u>miners 为聚合结果投票</u>(Voting results aggregation by the miners)
  • 连续识别为潜在恶意的设备加入黑名单

2.2.2 Validators' Voting

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

2.2.3 Validators forms validator-transaction

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

2.2.4 Validators send transactions to miners

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

3.1 Miners broadcast validator-transactions

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

3.2 Miners verify and aggregate voting results

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Strong assumption:Assume links are stable, and transactions can always arrive in time. (假设 任意一对设备(w-v, v-w)是足够稳定的,且不会丢包。在实践中,我们设置超时机制来等待所有交易都能及时到达。因此从上图可以看出,对于每个本地模型的投票总数量为4,即为validators的数量)

3.3 Miners create candidate block and mine the block

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

3.4 Miners propagates mined blocks

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

3.5 Miner picks legitimate block

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

VBFL-PoS 根据矿工stake 来选择candidate block,即选择stake最大的。这里则用到了Stake_tracker

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

上图显示,m2的stake最大

3.6 Miners append legitimate block to their own blockchain

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

3.7 Miners requests associated devices to download its appended block

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Forking Events

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  1. Dishonest miner

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  1. Same highest stake

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  1. Previous chain fork

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  1. Timeout (Latency)

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

虚线连接不是很稳定,沿着这三个链接的广播区块则永远跑到红色,所以尽管它们有相同的stake_tracker,但最后三个矿工最终有不同的候选区块集

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Solution to Forking

chain-resync

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

实现了chain-resync,但就以上四种情况,本文作者没有测试

last phase appended block processing

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

More to do

<u>VBFL Global Model Update</u>

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<u>Update stake tracker</u>

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<u>VBFL-PoS Rewarding Mechanism (Worker)</u>

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

VBFL-PoS Rewarding Mechanism (Validator)

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<u>VBFL-PoS Rewarding Mechanism (Miner)</u>

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<u>VBFL-PoS ==Mainly== Rewards Legitimate Workers</u>

Incentive to honest workers!

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Blacklisting

验证机制(如Introduction to VBFL 部分所述)

  • <u>validators为本地模型更新投票</u> (Vote for local model updates by the validators)
  • <u>miners 为聚合结果投票</u>(Voting results aggregation by the miners)
  • <u>连续识别为潜在恶意的设备加入黑名单</u>

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<u>Communication round ends</u>

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Experimental Results

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Conclusion

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Abstract

VBFL:解决FL的健壮性问题

在区块链框架中利用两种机制:

  1. 去中心化验证机制:各个验证者 验证本地模型更新的合法性
  2. 专用的PoS共识机制:权益更频繁地奖励给诚实的设备,通过增加它们指示(dictating)附加到区块链的块的机会 来保护合法的本地模型更新

Introduction

FL在现实中的两个健壮性问题

  1. 基于区块链的FL是一个很有前途的去中心化解决方案,但由于有大量的设备,其工作证明(PoW)共识延迟是显著的
  2. FL不能过滤掉全局模型构建过程中的故障恶意攻击。(如图1)

==>> 直接的解决方案:比较每个局部模型与其他模型的精度《Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. 2020》。这种方法在分散的体系结构下是不可行的,因为缺少可供比较的模型,因为不是所有的局部模型都能在有限的时间内在一个地点很容易地收集到

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

VBFL引入 1)以去中心化方式检查和投票本地模型更新的验证机制;2)基于PoS共识机制,允许做出最多学习贡献的设备创建合法块,用它们对应的投票结果记录本地模型更新

每个通信轮中,每个拥有本地数据集的设备都是随机指定的:

  1. 一个ML工人 worker 更新他的 本地模型
  2. 一个模型验证者 validator 对接收到的本地模型更新的合法性进行检查和投票
  3. 区块链 miner 尝试将 投票结果存储在下一个协商一致的块中相应的本地模型 合并

鉴于这种角色转换策略,VBFL旨在通过以下补充机制在合法设备中促进更多的联邦

  • Decentralized Model Validation
  • PoS-inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

<center>图2. 一个通信回合内的VBFL操作</center>

Experimental

20 devices,Vanilla FL和VBFL 每台设备的训练集时 完整MNIST数据集相同大小的分片 且 无重叠

KickR: 6; r:1

每轮通信5个本地训练回合, 学习率0.01,batch size: 10

  1. VFL_0/20: 带有20台合法学习设备的Vanilla FL
  2. VFL_3/20: 3/20的恶意设备的 Vanilla FL
  3. VBFL_PoS_0/20_vh1.00: 20台合法设备,validator_threshold: 1.0的 VBFL-PoS (-nd 20 -max_ncomm 100 -ha 12,5,3 -aio 1 -pow 0 -ko 6 -nm 0 -vh 0.08 -cs 0 -B 10 -mn mnist_cnn -iid 0 -lr 0.01 -dtx 1)
  4. VBFL_PoS_3/20_vh0.08: 3/20台恶意设备 validator_threshold: 0.08的 VBFL-PoS (-nd 20 -max_ncomm 100 -ha 12,5,3 -aio 1 -pow 0 -ko 6 -nm 3 -vh 0.08 -cs 0 -B 10 -mn mnist_cnn -iid 0 -lr 0.01 -dtx 1)
  5. VBFL_PoW_3/20_vh0.08:3/20台恶意设备 validator_threshold: 0.08的 VBFL-PoW (-nd 20 -max_ncomm 100 -ha 12,5,3 -aio 1 -pow 0 -ko 6 -nm 3 -vh 0.08 -cs 0 -B 10 -mn mnist_cnn -iid 0 -lr 0.01 -dtx 1) pow的值非0,则表示PoW协议的挖掘难度

Figures

  • fig 1.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • fig3.[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus
  • fig 4.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • fig 5.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • fig 6.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • fig 7.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • fig 8.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • fig 9.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

  • fig 10.

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

[PaperNotes]2021.Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Experimental Arguments

debug attributes note
-g--gpugpu id,默认为0
-v--verbose打印verbose 调试日志
-sn--save_network_snapshots设置为1时 只保存网络快照,
-dtx--destroy_tx_in_block
-rp--resume_path
-sf--save_freq保存网络快照的频率,默认为5
-sm--save_most_recent为了节省空间,只保存最近特定数量的快照,0表示保存所有
FL attributes
-B--batchsize本地训练batch size
-mn--model_name训练的模型名称
-lr--learning_rate学习率,用原论文的数值作为默认值,0.01
-op--optimizer使用的优化器,默认使用SGD
-iid--IID设备数据的分布
-max_ncomm--max_num_comm通信的最大轮数, 如果收敛则提前结束
-nd--num_devices仿真网络中 设备的数量
-st--shard_test_data当测试数据集没有分片时,容易看到设备间的全局模型是一致的
-nm--num_malicious网络中恶意节点的数量, 恶意节点的数据集引入高斯噪声
-nv--noise_variance影响的高斯噪声的 噪声variance等级
-le--default_local_epochs本地训练回合。如果-mt没有指定,那么训练本地模型与每个工人的回合数相同
blockchain system consensus attributes
-ur--unit_reward提供数据、签名验证(verification) 以及 有效(validation)及其他的单元奖励, 默认为1
-ko--knock_out_rounds如果在这个轮数内被识别为恶意的。则 worker 或 validator设备被踢出设备列表(放在黑名单中)
-lo--lazy_worker_knock_out_rounds如果工人设备没有提供这轮训练的更新(因为太慢或 懒得做更新而只接受模型更新),工人将被踢出设备列表
-pow--pow_difficulty如果设置为0,意味miners使用PoS,默认值为0
blockchain FL validator/miner restriction tuning parameters
-mt--miner_acception_wait_time矿工接受交易的默认时间窗口。0表示不限时间,每台设备每轮执行相同回合数(-le), 如FedAvg paper中的
-ml--miner_accepted_transactions_size_limit在这个限制后 将不会接受更多的交易 (这个参数决定了最终的区块大小 block_size)
-mp--miner_pos_propagated_block_wait_time等待时间是从通信轮开始时 计数,用于模拟PoS中分叉事件
-vh--validator_threshold判断恶意工人的误差阈值
-md--malicious_updates_discount不完全丢弃投票为 负面的工人交易, 因为
-mv--malicious_validator_on让恶意validator 翻转投票结果
distributed system attributes
-ns--network_stability设备在线的概率
-els--even_link_speed_strength此变量用于模拟传输延迟。默认值为1,表示每个设备被分配相同的链路强度 -dts bytes/sec。如果设置为0,链路强度随机初始化[0,1],意味实验过程中一台设备将传输 -els*-dts bytes/sec, 一个交易大约35k 字节
-dts--base_data_transmission_speed当-els=1时 每秒可以传输的数据量。设置这个变量来决定传输速度(带宽),这进一步决定传输延迟
-ecp--even_computation_power用于模拟硬件设备的强度。计算时间将按此值缩短。默认值1表示平均分配计算能力为1。如果设置为0,power被随机初始化为[0,4]之间的整数
simulation attributes
-ha--hard_assign网络中角色的困难的设计数量,三个值的顺序为worker,validator,miner
-aio--all_in_one在注册时 让网络中的节点相互注意
-cs--check_signature如果设置为0,假设所有签名被验证来保存执行时间

References