Artificial Intelligence in Education and Student Data Privacy: What Educators Need to Know

One of the most critical questions that any education leaders must ask themselves before implementing an education technology product that utilizes artificial intelligence (AI) is: “Will my educator and student data be kept private and secure?” While data privacy is a deep and complex topic, when it comes to AI, there are a few critical areas to pay attention to. 

Machine learning and AI are at the heart of many of the data-privacy issues today. Technology companies that are guilty of misusing consumer data often do so through these emerging technologies. However, it should be pointed out this is the exception and not the rule. When a consumer pays a company for its products, the consumer’s data should not be used in a way other than to make the product function as expected or to improve the product experience for that consumer.

Personally Identifiable Information

With so many news stories and articles discussing the questionable ways that technology firms use and sell consumer data, a nonnegotiable factor in deciding which edtech company to partner with is to make sure that it is up front and clear about never selling your data for marketing, commercial, or political purposes (for instance, see EdSurge’s “What Every Edtech Company Needs to Know About Schools and Data Privacy”).

Personally identifiable information (PII) is any piece of information that can be used to identify or reidentify a user. There are numerous laws and regulations in place that protect student and teacher data, but given the complexity of data privacy, many companies tend to operate in the gray area of PII data use. However, under no circumstances, should PII be shared with any entity other than the edtech provider.

Let’s assume that your edtech provider has assured you and proven that it will be the only third party with access to PII. What are the expectations you must place on it for use of said PII data? Focus on how long the company has access to your data and how it uses the data.

Historical Data Retention Policy

It is common that an edtech company will need to store product-usage data to provide an optimal user experience for educators and students, and that should be expected. Historical data can be critical for providing student performance trends and forecasts and for providing educators with more automation and personalization. However, there must be data-retention policies in place to ensure that only the necessary historical data will be stored to provide the end user with the agreed-upon experience. Clear justification should be provided for any data storage beyond that, and any excess data should be purged permanently.

It’s important to remember that data belong to consumers, not the edtech provider. This is not to say that historical data shouldn’t be stored; in fact, as a data-science leader, I know that it is critical to provide the best possible products and solutions and to continuously improve and innovate. However, the collections of data don’t belong to me or my company, and they need to be treated with great care and privacy—the exact way that I would want my own data treated.

Machine Learning Algorithm Training

Considering that collecting consumer data is necessary to develop AI and machine-learning algorithms, how can edtech companies that are building these products and product features ethically use consumer data to improve the consumer experience? By far, the most important step is to never use PII to train the models. While usage patterns and macro-level demographic characteristics are often important, using specific PII data is never needed to build accurate and powerful AI.

Ultimately, those who are training and building the machine-learning algorithms have no reason to even have access to PII. Maintaining this as a critical best practice will go a very long way in ensuring data privacy for consumers.

Edmentum, artificial intelligence, and the future of education technology

Ready to bring AI into your edtech implementation? Be on the lookout for what Edmentum has in store for summer 2020 to make setting up and implementing a new program quicker and easier than ever before.

For more information on the impact AI is having on education, check out the blog posts How Artificial Intelligence Can Improve the Success of your Edtech Implementation and  Two Ways Artificial Intelligence in Online Learning Can Save Teachers Time.