# Bibiliography on federated learning

###### tags: `bibliography`

[toc]

## Most relevant papers

### Jinhyun So, et al. Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning. IEEE JOURNAL ON SELECTED AREAS IN INFORMATION THEORY, VOL. 2, NO. 1, MARCH 2021.
* Reduce $O(n^2)$ to $O(n log n)$ by partitioning users into groups, but requires more rounds.
* Honest-but-curious model.
* Pairwise keys based on DH

### Zheng, Wenting, et al. "Cerebro: A platform for multi-party cryptographic collaborative learning." USENIX Seccurity, 2021.

* Free access [paper](https://www.usenix.org/system/files/sec21-zheng.pdf)
* Use same SPDZ as in Helen
* Tested 2-12 parties

### Poddar, R., Kalra, S., Yanai, A., Deng, R., Popa, R. A., & Hellerstein, J. M. (2021). Senate: A Maliciously-Secure {MPC} Platform for Collaborative Analytics. USENIX Securiy 2021.

* [USENIX page](https://www.usenix.org/conference/usenixsecurity21/presentation/poddar) with page and slides
* Use a MPC technique due to [WRK](https://eprint.iacr.org/2017/189.pdf). WRK needs $O(1)$ rounds, but requires pairwise private and authenticated channels.
* It says SPDZ and WRK are the state-of-the-art MPCs for arithmetic and boolean settings respectively.
* Tested against up to 16 parties.

### Kandhe et al. FASTSECAGG: Scalable Secure Aggregation for Privacy-Preserving Federated Learning, FL-ICML, 2020.

* Proposed an aggregation method called FastSecAgg.
* Reduced computation costs of SegAgg from $O(N^2)$ to $O(N log N) by using a technique to compress high-dimensional model updates, but the communication cost remains $O(N^2)$. It needs 3 rounds.
* Used honest-but-curious model.
* Assume pairwise secure channels

### Zheng, Wenting, et al. "Helen: Maliciously secure coopetitive learning for linear models." IEEE S&P, 2019.

* They represent the dataset using fixed point integer representation to suit cryptographic primitivates that use groups and fields.
* Used $n$/$n$ threshold control
* Target a few large organizations (the paper says around 10); tested only 4 parties
* Testing used two real world datasets (gas sensor data and the million song dataset) from UCI
* SGD (stochastic gradient descent) scales quadratic in the number of players $m$ because it first scales linearly in $m$ due to the behavior of the MPC protocol.
* Used SPDZ which requires $O(n)$ rounds. Lindell, Pinkas, Smart and Yanai (Crypto'15) tried to improve round efficiency of SPDZ round efficiency to $O(1)$ by combining BMR and SPDZ, but depend on pairwise secret/authenticated channels (required by BMR).

### Luca Melis, et al. Exploiting Unintended Feature Leakage in Collaborative Learning. IEEE Symposium on Security and Privacy 2019.
* It show the updates can leak unintende information about the local training data.

### Hard et al. FEDERATED LEARNING FOR MOBILE KEYBOARD PREDICTION, arxiv, 2019
* Used FederatedAveranging algorithm for G-Keyboard which provides autocorrection, word completion, and next-word prediction.
* Users are required to trust the aggregation server not to scrutinize individual weight uploads.

### Xu et al. verifynet: secure and verifiable federated learning, IEEE TIFS'20
* Assume a trustworthy authority (TA) who plays the roles as a CA
* Assume an honest-but-curious threat model
* Use Diffie-Hellman to set up pairwise secret channels
* Used secret sharing techniques to mask input data
* 4 rounds
* Tested 600 mobile devices and up to 30% drop-out rate

### Guo et al. VeriFL: Communication-Efficient and Fast Verifiable Aggregation for Federated Learning. IEEE TIFS'20
* Assume an honest-but-curious attacker
* Use DH to establish pairwise secret channels
* Use the double double-masking method as in VerifyNet
* 3 Rounds
* 30% drop-out

### McMahan et al, Communication-efficient learning of deep networks from decentralized data. AISTATS, 2017.

* Proposed *FederatedAvareging*, which computes the average of gradients based on Stochastic Graident Descent (SGD)

### Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., ... & Seth, K. (2017, October). Practical secure aggregation for privacy-preserving machine learning. CCS, 2017.

* Free access paper at [IACR](https://eprint.iacr.org/2017/281.pdf)
* Describe the problem of *Secure Agggregatioon*: the problem of computing a multiparty sum where no party reveals its update in the clear - even to the aggregator.
* Computing a secure weighted average is straightforward given a secure sum ooperation.
* In some circumstances, it is possible to learn indiviudal words tht a user has typed by inspecting thhat user's most recent update.
* Use PKI and DH to established pairwise private, authenticated channels.
* To address random dropouot, they use a threshold secret sharing scheme.
* The protocol is run in 4 rounds, and tested with up to 1000 clients.
* Doesn't verify that users' inputs are well-formed, hence, don't prevent out-of-range attacks.

## Other papers

### Yin, Xuefei, Yanming Zhu, and Jiankun Hu. "A Comprehensive Survey of Privacy-preserving Federated Learning: A Taxonomy, Review, and Future Directions." ACM Computing Surveys (CSUR) 54, no. 6 (2021): 1-36.

* Three types of data partitioning: horizontal, vertical and hybrid
* During training, the exchanged model parameters are of two types: a model weight or gradient.

### Fereidooni, et al. SAFELearn: Secure Aggregation for private FEderated Learning. IEEE Security and Privacy Workshops (SPW). 2021.
* A generic design. 2 rounds based on secure two-party computation.
* Semi-honest aggregator. Tested 500 clients.
* Doesn't look relevant as the design is generic.

### Culea et al. A Survey of Two Open Problems of Privacy-Preserving Federated Learning: Vertically Partitioned Data and Verifiability.

* Free access [page](https://repository.tudelft.nl/islandora/object/uuid%3Ab8d7817d-3cb9-4e22-9b1e-9db7f599aed8)
* Two open problems: vertically partitioned data and verifiability
* "GDPR regulations made the classical centralized training not only unfeasible but also illegal"
* "Most of the existing FL frameworks only work with horizontally partiiotoned data. Unlike horizental FL, vertifical FL requires a more complex mechansism to decompose a the loss function at each party."

### Rathee, D., Rathee, M., Kumar, N., Chandran, N., Gupta, D., Rastogi, A., & Sharma, R. (2020, October). CrypTFlow2: Practical 2-party secure inference. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (pp. 325-342).

* Free access [paper](https://eprint.iacr.org/2020/1002.pdf)
* Two-party computation. Doesn't seem relevant.

### Chen, H., Kim, M., Razenshteyn, I., Rotaru, D., Song, Y., & Wagh, S. (2020, December). Maliciously secure matrix multiplication with applications to private deep learning. AsiaCrypt 2020.

* Free access [paper](https://eprint.iacr.org/2020/451.pdf)
* Applied SPDZ to matrix multiplication. Doesn't seem relevant to our work.

## Notes

* Secure aggregation.

| Column 1 | Bonawitz et al.'17 | Helen'19 | VerifyNet'20 | VERIFL'20 |
| -------- | -------- | -------- | -------- | -------- |
| Channels | Pairwise (ECDH) | Public auth | Pairwise (DH) | Pairwise (DH)
| Rnds | 4 | $O(n)$ | 4 | 3 |
| Security | HBC/Mal | Mal | HBC | HBC |
| Out-of-range | No protection | Yes (Boudot proof) | No | No |
| drop-outs | Up to 30% (SS) | No | up to 30% (SS) | Up to 30% |
| No of clients | 1000 | Up to 10 | 600 | 500 |
| Computation | C $O(n^2)$, S $O(n^2)$ | $O(n^2)$ | Not clear | Not clear |
| Communication | C $O(n)$, S $O(n^2)$ | Not clear | Note clear | Not clear |
| Notes | | | |