---
# System prepended metadata

title: Bibiliography on federated learning
tags: [bibliography]

---

# Bibiliography on federated learning

###### tags: `bibliography`

[toc]

## Most relevant papers

### Jinhyun So, et al. Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning. IEEE JOURNAL ON SELECTED AREAS IN INFORMATION THEORY, VOL. 2, NO. 1, MARCH 2021.
 * Reduce $O(n^2)$ to $O(n log n)$ by partitioning users into groups, but requires more rounds.
 * Honest-but-curious model.
 * Pairwise keys based on DH

### Zheng, Wenting, et al. "Cerebro: A platform for multi-party cryptographic collaborative learning." USENIX Seccurity, 2021.

 * Free access [paper](https://www.usenix.org/system/files/sec21-zheng.pdf)
 * Use same SPDZ as in Helen
 * Tested 2-12 parties

### Poddar, R., Kalra, S., Yanai, A., Deng, R., Popa, R. A., & Hellerstein, J. M. (2021). Senate: A Maliciously-Secure {MPC} Platform for Collaborative Analytics. USENIX Securiy 2021.
 

 * [USENIX page](https://www.usenix.org/conference/usenixsecurity21/presentation/poddar) with page and slides
 * Use a MPC technique due to [WRK](https://eprint.iacr.org/2017/189.pdf). WRK needs $O(1)$ rounds, but requires pairwise private and authenticated channels.
 * It says SPDZ and WRK are the state-of-the-art MPCs for arithmetic and boolean settings respectively.
 * Tested against up to 16 parties. 

### Kandhe et al. FASTSECAGG: Scalable Secure Aggregation for Privacy-Preserving Federated Learning, FL-ICML, 2020.
 
 * Proposed an aggregation method called FastSecAgg.
 * Reduced computation costs of SegAgg from $O(N^2)$ to $O(N log N) by using a technique to compress high-dimensional model updates, but the communication cost remains $O(N^2)$. It needs 3 rounds.
 * Used honest-but-curious model.
 * Assume pairwise secure channels

### Zheng, Wenting, et al. "Helen: Maliciously secure coopetitive learning for linear models." IEEE S&P, 2019.

* They represent the dataset using fixed point integer representation to suit cryptographic primitivates that use groups and fields.
 * Used $n$/$n$ threshold control
 * Target a few large organizations (the paper says around 10); tested only 4 parties
 * Testing used two real world datasets (gas sensor data and the million song dataset) from UCI 
 * SGD (stochastic gradient descent) scales quadratic in the number of players $m$ because it first scales linearly in $m$ due to the behavior of the MPC protocol.
 * Used SPDZ which requires $O(n)$ rounds. Lindell, Pinkas, Smart and Yanai (Crypto'15) tried to improve round efficiency of SPDZ round efficiency to $O(1)$ by combining BMR and SPDZ, but depend on pairwise secret/authenticated channels (required by BMR). 

### Luca Melis, et al. Exploiting Unintended Feature Leakage in Collaborative Learning. IEEE Symposium on Security and Privacy 2019.
 * It show the updates can leak unintende information about the local training data.

### Hard et al. FEDERATED LEARNING FOR MOBILE KEYBOARD PREDICTION, arxiv, 2019
 * Used FederatedAveranging algorithm for G-Keyboard which provides autocorrection, word completion, and next-word prediction.
 * Users are required to trust the aggregation server not to scrutinize individual weight uploads.

### Xu et al. verifynet: secure and verifiable federated learning, IEEE TIFS'20
 * Assume a trustworthy authority (TA) who plays the roles as a CA
 * Assume an honest-but-curious threat model
 * Use Diffie-Hellman to set up pairwise secret channels
 * Used secret sharing techniques to mask input data
 * 4 rounds
 * Tested 600 mobile devices and up to 30% drop-out rate

### Guo et al. VeriFL: Communication-Efficient and Fast Verifiable Aggregation for Federated Learning. IEEE TIFS'20
 * Assume an honest-but-curious attacker
 * Use DH to establish pairwise secret channels
 * Use the double double-masking method as in VerifyNet
 * 3 Rounds
 * 30% drop-out

### McMahan et al, Communication-efficient learning of deep networks from decentralized data. AISTATS, 2017. 

 * Proposed *FederatedAvareging*, which computes the average of gradients based on Stochastic Graident Descent (SGD)

### Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., ... & Seth, K. (2017, October). Practical secure aggregation for privacy-preserving machine learning. CCS, 2017.

  * Free access paper at [IACR](https://eprint.iacr.org/2017/281.pdf)
  * Describe the problem of *Secure Agggregatioon*: the problem of computing a multiparty sum where no party reveals its update in the clear - even to the aggregator.
  * Computing a secure weighted average is straightforward given a secure sum ooperation.
  * In some circumstances, it is possible to learn indiviudal words tht a user has typed by inspecting thhat user's most recent update. 
  * Use PKI and DH to established pairwise private, authenticated channels.
  * To address random dropouot, they use a threshold secret sharing scheme.
  * The protocol is run in 4 rounds, and tested with up to 1000 clients.
  * Doesn't verify that users' inputs are well-formed, hence, don't prevent out-of-range attacks.

## Other papers

### Yin, Xuefei, Yanming Zhu, and Jiankun Hu. "A Comprehensive Survey of Privacy-preserving Federated Learning: A Taxonomy, Review, and Future Directions." ACM Computing Surveys (CSUR) 54, no. 6 (2021): 1-36.

* Three types of data partitioning: horizontal, vertical and hybrid
* During training, the exchanged model parameters are of two types: a model weight or gradient.

### Fereidooni, et al. SAFELearn: Secure Aggregation for private FEderated Learning. IEEE Security and Privacy Workshops (SPW). 2021.
 * A generic design. 2 rounds based on secure two-party computation. 
 * Semi-honest aggregator. Tested 500 clients. 
 * Doesn't look relevant as the design is generic.


### Culea et al. A Survey of Two Open Problems of Privacy-Preserving Federated Learning: Vertically Partitioned Data and Verifiability. 

 * Free access [page](https://repository.tudelft.nl/islandora/object/uuid%3Ab8d7817d-3cb9-4e22-9b1e-9db7f599aed8)
 * Two open problems: vertically partitioned data and verifiability
 * "GDPR regulations made the classical centralized training not only unfeasible but also illegal"
 * "Most of the existing FL frameworks only work with horizontally partiiotoned data. Unlike horizental FL, vertifical FL requires a more complex mechansism to decompose a the loss function at each party."

### Rathee, D., Rathee, M., Kumar, N., Chandran, N., Gupta, D., Rastogi, A., & Sharma, R. (2020, October). CrypTFlow2: Practical 2-party secure inference. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (pp. 325-342).

 * Free access [paper](https://eprint.iacr.org/2020/1002.pdf)
 * Two-party computation. Doesn't seem relevant.

### Chen, H., Kim, M., Razenshteyn, I., Rotaru, D., Song, Y., & Wagh, S. (2020, December). Maliciously secure matrix multiplication with applications to private deep learning. AsiaCrypt 2020.

 * Free access [paper](https://eprint.iacr.org/2020/451.pdf)
 * Applied SPDZ to matrix multiplication. Doesn't seem relevant to our work.

## Notes

 * Secure aggregation. 


| Column 1 |  Bonawitz et al.'17 | Helen'19 | VerifyNet'20 | VERIFL'20 |
| -------- | --------             | -------- |  -------- | -------- | 
| Channels | Pairwise (ECDH)      |  Public auth    | Pairwise (DH) | Pairwise (DH)
| Rnds     | 4                    | $O(n)$     | 4 | 3 | 
| Security     | HBC/Mal          | Mal     | HBC | HBC |
| Out-of-range     | No protection | Yes (Boudot proof)     | No | No | 
| drop-outs     | Up to 30% (SS)    | No     | up to 30% (SS) | Up to 30% |
| No of clients     | 1000     | Up to 10   | 600 | 500 |
| Computation     | C $O(n^2)$, S $O(n^2)$     | $O(n^2)$ | Not clear | Not clear | 
| Communication     | C $O(n)$, S $O(n^2)$     | Not clear | Note clear | Not clear |
| Notes |  |  | |

