The High-Level Practical Overview of Open-Source Privacy-Preserving Machine Learning Solutions
Abstract
This paper aims to provide a high-level overview of practical approaches to machine-learning respecting the privacy and confidentiality of customer information, which is called Privacy-Preserving Machine Learning. First, the security approaches in offline-learning privacy methods are assessed. Those focused on modern cryptographic methods, such as Homomorphic Encryption and Secure Multi-Party Computation, as well as on dedicated combined hardware and software platforms like Trusted Execution Environment - Intel® Software Guard Extensions (Intel® SGX). Combining the security approaches with different machine learning architectures leads to our Proof of Concept in which the accuracy and speed of the security solutions will be examined. The next step was exploring and comparing the Open-Source Python-based solutions for PPML. Four solutions were selected from almost 40 separate, state-of-the-art systems: SyMPC, TF-Encrypted, TenSEAL, and Gramine. Three different Neural Network architectures were designed to show different libraries’ capabilities. The POC solves the image classification problem based on the MNIST dataset. As the computational results show, the accuracy of all considered secure approaches is similar. The maximum difference between non-secure and secure flow does not exceed 1.2%. In terms of secure computations, the most effective Privacy-Preserving Machine Learning library is based on Trusted Execution Environment, followed by Secure Multi-Party Computation and Homomorphic Encryption. However, most of those are at least 1000 times slower than the non-secure evaluation. Unfortunately, it is not acceptable for a real-world scenario. Future work could combine different security approaches, explore other new and existing state-of-the-art libraries or implement support for hardware-accelerated secure computation.
References
F. Newsroom. (2018) Fda permits marketing of artificial intelligence-
based device to detect certain diabetes-related eye problems. [Online].
Available: https://www.fda.gov/news-events/press-announcements/fda-
permits-marketing-artificial-intelligence-based-device-detect-certain-
diabetes-related-eye
FDA. Artificial intelligence and machine learning (ai/ml)-enabled
medical devices. [Online]. Available: https://www.fda.gov/medical-
devices/software-medical-device-samd/artificial-intelligence-and-
machine-learning-aiml-enabled-medical-devices
C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. J.
Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” in
nd International Conference on Learning Representations, ICLR 2014,
Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings,
Y. Bengio and Y. LeCun, Eds., 2014. [Online]. Available: http:
//arxiv.org/abs/1312.6199
A. Shafahi, W. R. Huang, M. Najibi, O. Suciu, C. Studer,
T. Dumitras, and T. Goldstein, “Poison frogs! targeted clean-
label poisoning attacks on neural networks,” in Advances
in Neural Information Processing Systems 31: Annual Conference on
Neural Information Processing Systems 2018, NeurIPS 2018, December
-8, 2018, Montr ́eal, Canada, S. Bengio, H. M. Wallach, H. Larochelle,
K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., 2018, pp.
–6116. [Online]. Available: https://proceedings.neurips.cc/paper/
/hash/22722a343513ed45f14905eb07621686-Abstract.html
X. Yuan, P. He, Q. Zhu, and X. Li, “Adversarial examples: Attacks
and defenses for deep learning,” IEEE Trans. Neural Networks Learn.
Syst., vol. 30, no. 9, pp. 2805–2824, 2019. [Online]. Available:
https://doi.org/10.1109/TNNLS.2018.2886017
D. M. Bamasoud, A. S. Al-Dossary, N. M. Al-Harthy, R. A.
Al-Shomrany, G. S. Alghamdi, and R. O. Algahmdi, “Privacy and
security issues in cloud computing: A survey paper,” in International
Conference on Information Technology, ICIT 2021, Amman, Jordan, July
-15, 2021. IEEE, 2021, pp. 387–392. [Online]. Available: https:
//doi.org/10.1109/ICIT52682.2021.9491632
Y. Zhang and R. Sion, “Speculative execution attacks and cloud
security,” in Proceedings of the 2019 ACM SIGSAC Conference on
Cloud Computing Security Workshop, CCSW@CCS 2019, London, UK,
November 11, 2019, R. Sion and C. Papamanthou, Eds. ACM, 2019,
p. 201. [Online]. Available: https://doi.org/10.1145/3338466.3360287
Y. Alghofaili, A. Albattah, N. Alrajeh, M. A. Rassam, and B. A. S.
Al-rimy, “Secure cloud infrastructure: A survey on issues, current
solutions, and open challenges,” Applied Sciences, vol. 11, no. 19, 2021.
[Online]. Available: https://www.mdpi.com/2076-3417/11/19/9005
N. Dowlin, R. Gilad-Bachrach, K. Laine, K. Lauter, M. Naehrig, and
J. Wernsing, “Cryptonets: Applying neural networks to encrypted data
with high throughput and accuracy,” Tech. Rep. MSR-TR-2016-3,
[Online]. Available: https://www.microsoft.com/en-us/research/
publication/cryptonets-applying-neural-networks-to-encrypted-data-
with-high-throughput-and-accuracy/
J. Alvarez-Valle, P. Bhatu, N. Chandran, D. Gupta, A. Nori, A. Rastogi,
M. Rathee, R. Sharma, and S. Ugare, “Secure medical image analysis
with cryptflow,” 2020.
A. Soin, P. Bhatu, R. Takhar, N. Chandran, D. Gupta, J. Alvarez-Valle,
R. Sharma, V. Mahajan, and M. P. Lungren, “Multi-institution encrypted
medical imaging ai validation without data sharing,” 2021.
M. H. M. Elham Tabassi (NIST), Kevin Burns (MITRE). A taxonomy
and terminology of adversarial machine learning. [Online]. Available:
https://csrc.nist.gov/publications/detail/nistir/8269/draft
Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010.
[Online]. Available: http://yann.lecun.com/exdb/mnist/
F. Boemer, A. Costache, R. Cammarota, and C. Wierzynski, “ngraph-
he2: A high-throughput framework for neural network inference on
encrypted data,” 2019.
F. Boemer, Y. Lao, R. Cammarota, and C. Wierzynski, “ngraph-he: A
graph compiler for deep learning on homomorphically encrypted data,”
A. Benaissa, B. Retiat, B. Cebere, and A. E. Belfedhal, “Tenseal: A
library for encrypted tensor operations using homomorphic encryption,”
S. Carpov, P. Dubrulle, and R. Sirdey, “Armadillo: A compilation
chain for privacy preserving applications,” in Proceedings of the 3rd
International Workshop on Security in Cloud Computing, ser. SCC ’15.
Association for Computing Machinery, 2015, p. 13–19. [Online].
Available: https://doi.org/10.1145/2732516.2732520
I. Chillotti, N. Gama, M. Georgieva, and M. Izabach`ene, “Faster fully
homomorphic encryption: Bootstrapping in less than 0.1 seconds,” Cryp-
tology ePrint Archive, Report 2016/870, 2016, https://ia.cr/2016/870.
S. S. Magara, C. Yildirim, F. Yaman, B. Dilekoglu, F. R. Tutas,
E. ̈Ozt ̈urk, K. Kaya, ̈O. Tastan, and E. Savas, “Ml with he: Privacy
preserving machine learning inferences for genome studies,” 2021.
R. Dathathri, O. Saarikivi, H. Chen, K. Laine, K. Lauter, S. Maleki,
M. Musuvathi, and T. Mytkowicz, “Chet: an optimizing compiler for
fully-homomorphic neural-network inferencing,” in Proceedings of the
th ACM SIGPLAN Conference on Programming Language Design and
Implementation, 2019, pp. 142–156.
E. Hesamifard, H. Takabi, M. Ghasemi, and R. N. Wright, “Privacy-
preserving machine learning as a service,” Proc. Priv. Enhancing
Technol., vol. 2018, no. 3, pp. 123–142, 2018. [Online]. Available:
https://doi.org/10.1515/popets-2018-0024
C. Boura, N. Gama, M. Georgieva, and D. Jetchev, “Chimera: Combin-
ing ring-lwe-based fully homomorphic encryption schemes,” Cryptology
ePrint Archive, Report 2018/758, 2018, https://ia.cr/2018/758.
Q. Lou, B. Feng, G. C. Fox, and L. Jiang, “Glyph: Fast and accurately
training deep neural networks on encrypted data,” 2020.
OpenMined. (2021) Tenseal library. [Online]. Available: https:
//github.com/OpenMined/TenSEAL
J. H. Cheon, D. Kim, D. Kim, H. H. Lee, and K. Lee, “Numerical
method for comparison on homomorphically encrypted numbers,” Cryp-
tology ePrint Archive, Report 2019/417, 2019, https://ia.cr/2019/417.
J. H. Cheon, D. Kim, and D. Kim, “Efficient homomorphic comparison
methods with optimal complexity,” Cryptology ePrint Archive, Report
/1234, 2019, https://ia.cr/2019/1234.
U. Michelucci, Advanced applied deep learning : convolutional neural
networks and object detection. Apress, 2019.
A. Dalskov, D. Escudero, and M. Keller, “Secure evaluation of quantized
neural networks,” Cryptology ePrint Archive, Report 2019/131, 2019,
OpenMined. (2021) Sympc library. [Online]. Available: https://
github.com/OpenMined/SyMPC
N. Kumar, M. Rathee, N. Chandran, D. Gupta, A. Rastogi, and
R. Sharma, “Cryptflow: Secure tensorflow inference,” 2020.
EXAMPLE OF ARTICLE FOR INTERNATIONAL JOURNALS OF ELECTRONICS AND TELECOMMUNICATIONS 7
D. Rathee, M. Rathee, N. Kumar, N. Chandran, D. Gupta, A. Rastogi,
and R. Sharma, “Cryptflow2: Practical 2-party secure inference,”
Proceedings of the 2020 ACM SIGSAC Conference on Computer and
Communications Security, 2020. [Online]. Available: http://dx.doi.org/
1145/3372297.3417274
D. Rathee, M. Rathee, R. K. K. Goli, D. Gupta, R. Sharma, N. Chandran,
and A. Rastogi, “Sirnn: A math library for secure rnn inference,” Cryp-
tology ePrint Archive, Report 2021/459, 2021, https://ia.cr/2021/459.
B. Knott, S. Venkataraman, A. Hannun, S. Sengupta, M. Ibrahim, and
L. van der Maaten, “Crypten: Secure multi-party computation meets
machine learning,” in arXiv 2109.00984, 2021.
M. Dahl, J. Mancuso, Y. Dupis, B. Decoste, M. Giraud, I. Livingstone,
J. Patriquin, and G. Uhma, “Private machine learning in tensorflow using
secure computation,” 2018.
W. Henecka, S. K ̈ogl, A.-R. Sadeghi, T. Schneider, and I. Wehrenberg,
“Tasty: Tool for automating secure two-party computations,” Cryptology
ePrint Archive, Report 2010/365, 2010, https://ia.cr/2010/365.
P. Mohassel and P. Rindal, “Aby¡sup¿3¡/sup¿: A mixed protocol
framework for machine learning,” in Proceedings of the 2018 ACM
SIGSAC Conference on Computer and Communications Security, ser.
CCS ’18. Association for Computing Machinery, 2018, p. 35–52.
[Online]. Available: https://doi.org/10.1145/3243734.3243760
S. Wagh, D. Gupta, and N. Chandran, “Securenn: Efficient and private
neural network training,” Cryptology ePrint Archive, Report 2018/442,
W. Zheng, R. Deng, W. Chen, R. A. Popa, A. Panda, and I. Stoica,
“Cerebro: A platform for Multi-Party cryptographic collaborative
learning,” in 30th USENIX Security Symposium (USENIX Security 21).
USENIX Association, 2021, pp. 2723–2740. [Online]. Available:
https://www.usenix.org/conference/usenixsecurity21/presentation/zheng
S. Wagh, S. Tople, F. Benhamouda, E. Kushilevitz, P. Mittal, and
T. Rabin, “Falcon: Honest-majority maliciously secure framework for
private deep learning,” 2020.
M. S. Riazi, M. Samragh, H. Chen, K. Laine, K. Lauter, and F. Koushan-
far, “Xonn: Xnor-based oblivious deep neural network inference,” 2019.
M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schneider,
and F. Koushanfar, “Chameleon: A hybrid secure computation frame-
work for machine learning applications,” 2018.
A.-R. Sadeghi and T. Schneider, “Generalized universal circuits for
secure evaluation of private functions with application to data clas-
sification,” Cryptology ePrint Archive, Report 2008/453, 2008, https:
//ia.cr/2008/453.
M. Barni, P. Failla, R. Lazzeretti, A.-R. Sadeghi, and T. Schneider,
“Privacy-preserving ecg classification with branching programs and
neural networks,” IEEE Transactions on Information Forensics and
Security, vol. 6, no. 2, pp. 452–468, 2011.
P. Mohassel and Y. Zhang, “Secureml: A system for scalable privacy-
preserving machine learning,” in 2017 IEEE Symposium on Security and
Privacy (SP), 2017, pp. 19–38.
N. Koti, A. Patra, R. Rachuri, and A. Suresh, “Tetrad: Actively secure
pc for secure training and inference,” Cryptology ePrint Archive,
Report 2021/755, 2021, https://ia.cr/2021/755.
A. Patra and A. Suresh, “Blaze: Blazing fast privacy-preserving machine
learning,” Proceedings 2020 Network and Distributed System Security
Symposium, 2020. [Online]. Available: http://dx.doi.org/10.14722/
ndss.2020.24202
N. Koti, M. Pancholi, A. Patra, and A. Suresh, “Swift: Super-fast and
robust privacy-preserving machine learning,” 2021.
EzPC. (2021) Ezpc. [Online]. Available: https://github.com/mpc-msri/
EzPC
PySyft. (2021) Pysyft. [Online]. Available: https://github.com/
OpenMined/PySyft
T. Ryffel, P. Tholoniat, D. Pointcheval, and F. Bach, “Ariann: Low-
interaction privacy-preserving deep learning via function secret sharing,”
D. Labs. (2021) tf-encrypted library. [Online]. Available: https:
//github.com/tf-encrypted/tf-encrypted
gramine. (2021) gramine, library. [Online]. Available: https:
//github.com/gramineproject/gramine
D. Labs. (2021) tf-trusted, library. [Online]. Available: https://
github.com/capeprivacy/tf-trusted
F. Tram`er and D. Boneh, “Slalom: Fast, verifiable and private execution
of neural networks in trusted hardware,” 2019.
F. Mo, H. Haddadi, K. Katevas, E. Marin, D. Perino, and N. Kourtel-
lis, “Ppfl: Privacy-preserving federated learning with trusted execution
environments,” 2021.
F. Mo, A. S. Shamsabadi, K. Katevas, S. Demetriou, I. Leontiadis,
A. Cavallaro, and H. Haddadi, “Darknetz,” Proceedings of the 18th
International Conference on Mobile Systems, Applications, and Services,
[Online]. Available: http://dx.doi.org/10.1145/3386901.3388946
J. J. Dai, Y. Wang, X. Qiu, D. Ding, Y. Zhang, Y. Wang, X. Jia,
L. C. Zhang, Y. Wan, Z. Li, J. Wang, S. Huang, Z. Wu, Y. Wang,
Y. Yang, B. She, D. Shi, Q. Lu, K. Huang, and G. Song, “Bigdl: A
distributed deep learning framework for big data,” in Proceedings of
the ACM Symposium on Cloud Computing, ser. SoCC’19. Association
for Computing Machinery, 2019, pp. 50–60. [Online]. Available:
https://arxiv.org/pdf/1804.05839.pdf
M. Orenbach, P. Lifshits, M. Minkin, and M. Silberstein, “Eleos: Exitless
os services for sgx enclaves,” in Proceedings of the Twelfth European
Conference on Computer Systems, ser. EuroSys ’17. Association
for Computing Machinery, 2017, p. 238–253. [Online]. Available:
https://doi.org/10.1145/3064176.3064219
R. Kunkel, D. L. Quoc, F. Gregor, S. Arnautov, P. Bhatotia, and
C. Fetzer, “Tensorscone: A secure tensorflow framework using intel sgx,”
W. Ozga, D. L. Quoc, and C. Fetzer, “Perun: Secure multi-stakeholder
machine learning framework with gpu support,” 2021.
A. Mondal, Y. More, R. H. Rooparaghunath, and D. Gupta, “Flatee:
Federated learning across trusted execution environments,” 2021.
LeNET. (2021) Lenet. [Online]. Available: https://en.wikipedia.org/
wiki/LeNet
Downloads
Published
Issue
Section
License
Copyright (c) 2022 International Journal of Electronics and Telecommunications
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
1. License
The non-commercial use of the article will be governed by the Creative Commons Attribution license as currently displayed on https://creativecommons.org/licenses/by/4.0/.
2. Author’s Warranties
The author warrants that the article is original, written by stated author/s, has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author/s. The undersigned also warrants that the manuscript (or its essential substance) has not been published other than as an abstract or doctorate thesis and has not been submitted for consideration elsewhere, for print, electronic or digital publication.
3. User Rights
Under the Creative Commons Attribution license, the author(s) and users are free to share (copy, distribute and transmit the contribution) under the following conditions: 1. they must attribute the contribution in the manner specified by the author or licensor, 2. they may alter, transform, or build upon this work, 3. they may use this contribution for commercial purposes.
4. Rights of Authors
Authors retain the following rights:
- copyright, and other proprietary rights relating to the article, such as patent rights,
- the right to use the substance of the article in own future works, including lectures and books,
- the right to reproduce the article for own purposes, provided the copies are not offered for sale,
- the right to self-archive the article
- the right to supervision over the integrity of the content of the work and its fair use.
5. Co-Authorship
If the article was prepared jointly with other authors, the signatory of this form warrants that he/she has been authorized by all co-authors to sign this agreement on their behalf, and agrees to inform his/her co-authors of the terms of this agreement.
6. Termination
This agreement can be terminated by the author or the Journal Owner upon two months’ notice where the other party has materially breached this agreement and failed to remedy such breach within a month of being given the terminating party’s notice requesting such breach to be remedied. No breach or violation of this agreement will cause this agreement or any license granted in it to terminate automatically or affect the definition of the Journal Owner. The author and the Journal Owner may agree to terminate this agreement at any time. This agreement or any license granted in it cannot be terminated otherwise than in accordance with this section 6. This License shall remain in effect throughout the term of copyright in the Work and may not be revoked without the express written consent of both parties.
7. Royalties
This agreement entitles the author to no royalties or other fees. To such extent as legally permissible, the author waives his or her right to collect royalties relative to the article in respect of any use of the article by the Journal Owner or its sublicensee.
8. Miscellaneous
The Journal Owner will publish the article (or have it published) in the Journal if the article’s editorial process is successfully completed and the Journal Owner or its sublicensee has become obligated to have the article published. Where such obligation depends on the payment of a fee, it shall not be deemed to exist until such time as that fee is paid. The Journal Owner may conform the article to a style of punctuation, spelling, capitalization and usage that it deems appropriate. The Journal Owner will be allowed to sublicense the rights that are licensed to it under this agreement. This agreement will be governed by the laws of Poland.
By signing this License, Author(s) warrant(s) that they have the full power to enter into this agreement. This License shall remain in effect throughout the term of copyright in the Work and may not be revoked without the express written consent of both parties.
