What is Federated Learning?
It is a very interesting topic since the dawn of 2017 when Google introduces this particular term.
So what is federated learning?
- FL is a distributed learning which will allow you to train multiple models on multiple devices. But what does that mean? It means that the user data never leaves from their devices. User privacy right? This is very important that you are protecting your users' information. Federated learning is a very effective way where you can protect user information.
However, you might be wondering where did the FL come from?
- Basically federated learning comes from Machine Learning and Distributed Learning. Well is there any difference between machine learning and distributed learning? No, there is no difference. Distributed learning comes from machine learning. Let’s talk about that a little.
Machine Learning?
- ML is a way of teaching a machine to do some tasks for us which are complicated and tough to do manually. For example, you want to teach a machine detecting a specific fingerprint. Can you think of an example? Smartphone fingerprint sensor right? So, your device is intelligent enough to detect your fingerprint but denying others. How do you think the device is doing that? Well, the answer is machine learning. You give your device a specific template to detect your fingerprint and denying others. The device learns using your given template and based on datasets. And the dataset? The dataset is your multiple fingerprints. Each time you are trying to log into your device using fingerprint, the device keeps storing every record and learns automatically to improve its’ performance.
Okay, we have understood about machine learning. But what is “Distributed Learning”?
- The word distribute gives you some clue about it. Machine learning is to train on some datasets with a given template and produce model output. However, the model is your given template. And the model could be predicting the stock market, recognizing fingerprints, etc. Distributed learning is also about machine learning with the difference of training on multiple data sources such as servers. Let’s create an example. Think that you are a data scientist and you are given a task by Walmart to produce a common goal from 5 different Walmarts data.
To complete the job, you are collecting the data from five different locations. So, basically you are using a central server to collect all the data from different Walmart server and apply machine learning to produce the output as Walmart instructed you to. Therefore, do you think there is something with that approach? I do!!!!!!
Data Privacy?
- User data privacy is important, right? Think of another evil company out there who exploits your data transfers from Walmart servers to your central server. That is not what you want. So, you applied multiple differential privacy to protect the data. However, it is not guranted that the evil company won’t be able to exploit the data. Because, they might have smart programmer than you. 😔 It is pretty important topic to think about, right? There comes federated learing!!!
Federated Learning:
- FL allows the user to keep their data in their server. You, on the other hand, will send a global model to the users' server to train the model.
The central server and local servers are the common terms in federated learning. The central server is provided by you. And all the local servers are the user devices. In the above example, the central servers are smartphone devices. Since Google introduced this topic, let’s consider their example. They have implemented this technique in google's keyboard. If you are using Google keyboard, you will see the word suggestions based on your usage. Well, how do you think the automatic suggestions are coming? Federated learning!! Google has a central server and sends a dummy model to your device and all other users’ who are using Google keyboard. The model then will train your data. The data is your daily usage. When the model is training on the data? When you are not using your device and it is connected to WIFI and on the charge. Therefore, the model started its’ training phase when the device is IDLE condition. However, if a user interrupts during the training, then clients dropout also handled by Google. After completing the training, the model will come back to the central server. Google then aggregates all the models together to make a global model with modifications of course. However, in the central server, the global is tested by the users as well. So which users are used to testing the model? The users that couldn’t take part in the training phase or dropouts during the training phase. Google then distributes the global model to the users if it is better than the previous local model after the testing phase. This is a continuous process. Every smartphone is getting the model update if it is using the Google keyboard.
So, what federated learning ensures?
- First of all, user data never leaves from their devices but only the model. So if you can keep the model protected then user privacy is increased a lot.
- Scalability is increased since hundred of users can participate in this learning.
Sounds interesting right? I think so. That’s it for today! What do you think of Federated Learning? is it the next big thing in Deep Learning? Let me know in the comment section. Please if you find any mistake, do not hesitate to let me know.
Cheers 😆
