Some Additional Questions On Data Sharing and Collection

2 min read

Senator Al Franken has sent a letter to Google CEO Sundar Pichai asking some questions about Google's data collection practices. His letter raises some excellent questions, but there are two specific use cases that are left out that also need to be addressed. In a post from May 2015, I described some of the issues with how Google structures their terms for Google Apps for Education; these questions build on that post.

  • If a student is logged into their GAFE account and accesses a non-GAFE service provided by Google, can data collected via the non-GAFE service be shared with ad and data brokers, like Doubleclick or Quantcast?
  • If a student is logged into their GAFE account and accesses a 3rd party service that is integrated with GAFE, what student data can be accessed by that 3rd party vendor?

The focus on targeted advertising is important, but ultimately is too narrow. Targeted advertising is a visible manifestation of data collection and user profiling. While the presence of targeted ads let us know that some services are tracking users, larger potential harms accrue when people are invisibly profiled, with their data being shared with different brokers and combined with data sets from different sources. This should not happen within an educational context where students are legally required to attend.

It's also worth noting that while these specific questions focus on Google, comparable abuses of student data occur in many ecosystems that have a business plan based on 3rd party integrations.

Surveillance, Worst Case Scenarios, and the Winceable Moment

4 min read

In discussing issues related to privacy, people often devolve to trying to identify and define immediate harm and/or a worst case scenario. Both of these lenses are reductive and incomplete. Because data analysis often occurs invisibly to us, via proprietary algorithms that we don't even know are in play, assigning harm can be a matter of informed guesswork and inference. As one example, try explaining how and why your credit score is determined - this algorithmically defined number determines many opportunities we receive or don't receive, yet few of us can say with any certainty how this number is derived. Algorithms aren't neutral - they are a series of human judgments automated in a formula. There isn't any single worst case scenario, and discussions of worst case scenarios risk creating a false vision that there is a single spectrum with "privacy" at one one and some vague "worst case scenario" at the other - and this is not how it works.

The reason privacy matters - and the reason that profiling matters - is that we are seeing increasingly experimental and untested uses of data, especially in the realm of predictive analytics. Products using new statistical methods are used in hiring decisions, lending, mortgage decisions, finance, search, and personalization. The hype is that these new - or "innovative" or "disruptive" - uses of data will help us get more efficient, and push pass the biases of the past. However, this fails in at least two ways: first, algorithms contain the biases of their creators. Second, the performance of these products fails to live up to the hype, which in turn doesn't justify the risk.

Data collected in an educational setting - by definition - is data collected on people in the midst of enormous development, questioning, and growth. If people are doing adolescence right, they will make mistakes, ask questions, break things - all in the name of growth and learning. In the context of, for example, an eighth grade classroom, it all makes sense. But outside that context, it's very different. One of the promises of Big Data and Learning Analytics is that the data sets will be large enough to allow researchers to distill signal from noise, but as noted earlier, the reality fails to live up to the hype.

How many of us have memories of our behavior from high school, middle school, and elementary school that make us wince? Those winceable moments are our data trail. I mentioned earlier that talking about worst case scenarios is an inaccurate frame, and this is why: there is no single data point that, if undone, can "fix" our past. However, data collected from our adolescence is bound to contain things that are inaccurate, temporary, flawed, or confusing - for us, and for people attempting to find patterns.

If people are aware of surveillance, it shifts the way we act. When students are habituated to surveillance from an early age, it has the potential to shift the way we develop. If this data is shared outside of an educational context, it creates the potential that every person attending a public school is fully profiled before they graduate. A commonly overlooked element of this conversation is that profiles never come from a single source - they are assembled and combined from multiple sources. When data collected within an educational context gets combined with data sets collected from social media or our personal browsing history, different stories emerge.

For most people over 30 reading this post, our detailed records begin early in the 21st century, when we were 15 or older. For some kids in school now, their data profile begins when their parents posted their ultrasound on Facebook. While targeted advertising to kids is an immediate concern, at least targeted advertising is visible. Profiling by alogorithm is invisible, and is forever. Requiring students to pay for their public education with the data that will then be used to judge them sells out our kids. We can use data intelligently, but we need to have a candid conversation about what that means.

You Look Unprofessional When You Copy Your Policies

2 min read

Examples like this abound, but here's one. Searching for the exact phrase "resources we make available on our Site. Users may visit our Site anonymously" returns over 15,000 hits.

You can do this yourself by going to the policies on any site, and selecting a random phrase. I recommend selecting a the second half of one sentence and the beginning of the next, to include about 10-15 words total. If you find a typo or a misspelled work, use that word on the off-chance that the mistake has been faithfully plagiarized. If you are searching using Google, be sure to enclose the phrase in quotation marks to only get results using the exact string.

This is a simple, fast test to do on any site you think about using. It takes all of about 30 seconds. While plagiarized terms don't necessarily mean that there are issues with the site, it does indicate that the people behind the site have cut corners and taken shortcuts around privacy issues. Given that many breaches occur due to human error, and that problems we can observe often indicate issues that are hidden from view, plagiarized terms should at least give us pause.

If you are a vendor, do this test on your own terms. Pull ten excerpts at random, and see what comes up. If you discover that, for whatever reason, your terms have been plagiarized and you don't know why, you can then begin to fix the issue.

FERPA Directory Information as Anchor Data

3 min read

I'm currently working on a longer piece about education technology, the "school official" designation, parental consent, and how these legal definitions get frayed in practice. There has been a lot of attention paid to the school official designation in recent weeks, which is good. However, there have also been some inaccuracies and overstatements, and an unrealistic focus on one company - Google - in these conversations. While the actions of Google are interesting and very relevant because of their size and reach, we should not focus exclusively here on one company: the practices Google uses are widespread, and as we pursue better privacy practice we are better off identifying practices and habvits that need to improve, rather than playing whackamole with companies.

But, the process of writing this longer (not yet finished) piece has also spurred me on to finally get some thoughts out on directory information. As described in this FERPA directory information model form, "Directory information, which is information that is generally not considered harmful or an invasion of privacy if released, can also be disclosed to outside organizations without a parent’s prior written consent."

The list of information included as part of directory information - or "information that is generally not considered harmful or an invasion of privacy if released" - is pretty complete:

  • Student's name
  • Address
  • Telephone listing
  • Electronic mail address
  • Photograph
  • Date and place of birth
  • Major field of study
  • Dates of attendance
  • Grade level
  • Participation in officially recognized activities and sports
  • Weight and height of members of athletic teams
  • Degrees, honors, and awards received
  • The most recent educational agency or institution attended
  • Student ID number, user ID, or other unique personal identifier used to communicate in electronic systems
  • A student ID number or other unique personal identifier that is displayed on a student ID badge

If this information was compromised as part of a data breach, it would be considered substantial - yet, this information about children can be shared without parental consent, for their entire K12 experience.

Directory information data is accurate, high quality data. It places a person in a school. It ties them to a location. It provides a list of friends or acquaintances (kids in the same year at the same school). It provides a date and place of birth. It can provide a unique ID that could appear in other data systems - a boon for data enhancement via recombination. It includes a photo, and can include their height and weight. It's also worth noting that each school district identifies directory information differently, so some schools might share less information than what is listed in the model letter.

This data is updated annually, so that over the thirteen years of a child's K12 education, the data trail of directory information provides a comprehensive snapshot of a child's life from age 5 to 18.

Or, to put it in the words of Experian, directory information can be used to presort our high school students into those more likely to be "American Royalty" or those who are "Small Town Shallow Pockets".

While FERPA does allow parents to opt out of this data sharing, parental rights are often poorly understood.

As it currently stands, directory information creates an accurate, high quality dataset that provides the foundation for uses ranging from identity theft to profiling. While the standard argument in favor of easy sharing of directory information generally center around creating yearbooks, school play brochures, and media coverage of student athletes, we need to celebrate student accomplishments without compromising student privacy. To achieve that, we should reconsider how we handle directory information.

 

I'd Like Some "Smart" in Smart Toys, aka Hello Barbie Is Extra Chatty

2 min read

As reported in The Guardian, Hello Barbie had multiple security issues that would have allowed hackers to reroute the doll's communication to a different server, and override the doll's privacy settings. This initial breach would allow an attacker to access account information that would allow a hacker to know a home address, listen in to the doll's immediate surroundings, and even take over the responses that the doll said.

But the fun doesn't end there. Because the doll connects to the internet via a home wireless connection, hackers could use that information to take over the home internet connection, monitor traffic on the connection, and compromise devices and steal information within the home network. A compromised Hello Barbie doll would also be a great tool for people looking to see if and when a family is home. No need to place a microphone inside the house, because Jimmy is playing with it!

It's worth noting that these security issues were discovered by a researcher and disclosed to the manufacturer responsibly. As reported in the Guardian, ToyTalk, the company that partners with Mattel to make Hello Barbie, downplayed the multiple security holes.

An enthusiastic researcher has reported finding some device data and called that a hack. While the path that researcher used to find that data is not obvious and not user-friendly, it important to note that all that information was already directly available to Hello Barbie customers through the Hello Barbie Companion App. No user data, no Barbie content, and no major security nor privacy protections has been compromised to our knowledge.

If the manufacturer of a toy can't understand how an attack that compromises an entire home network isn't a "major security or privacy" issue, they need to get into another line of work. I don't think we're ready for smart toys until manufacturers can behave responsibly.

Privacy, Parenting, and the VTech Breach

1 min read

I put a post out on the Common Sense Media blog about the VTech breach:

Late last week, news broke about a data breach at the popular game and toy manufacturer VTech. The initial description of the breach sounded fairly large: Nearly five million parent accounts and 200,000 student accounts were compromised. If we make the assumption that one account equals one person, that's the equivalent of a data breach that covers every person living in Wyoming, Vermont, the District of Columbia, Alaska, North Dakota, South Dakota, and Delaware combined.

Read the whole post over at Common Sense.

VTech Data Breach - Some Steps To Take

3 min read

Edit, 11/27/2015: Troy Hunt - who was consulted on the Vice story - has a comprehensive review of the breach that is a great read. End edit.

Earlier today, Vice published a story on a data breach at VTech, a popular toy and game manufacturer. The bad news is that the breach was very extensive, with nearly 5 million unique emaila and personal info on approximately 200,000 children. The good news - if there is ever good news when reporting a data breach - is that the person who compromised the data isn't planning on releasing it.

However, from reading the description of the breach, it is possible that other people have already accessed the data stored by VTech. Data that could be accessed included email addresses, children's first name, gender, and date of birth, passwords, and password recovery questions and answers. According to the hacker (as quoted in the article linked above):

The hacker said that while he doesn’t intend to publish the data publicly, it’s possible others exfiltrated it before him.
"It was pretty easy to dump, so someone with darker motives could easily get it".

What To Do Now

First, don't panic. If you have bought a VTech product in the past, these steps can help minimize any risk. And, even if you haven't bought a VTech product, these steps are good practice. Nothing listed here is particularly novel or earth shattering, but these steps can help protect you over time.

  • On sites that use password recovery questions, go back and update your password and change the recovery questions and answers. This can be tedious, but it's a lot less work than recovering an account after it has been compromised.
  • Use a password manager (something like LastPass or 1Password) to store your passwords. This will allow you to store and use more complex passwords.
  • Establish a fraud alert. This guide from the Privacy Rights Clearinghouse contains clear instructions on how to do that, as well as other useful information on steps to take if and when your personal data gets compromised.
  • Visit Have I been pwned to see if your data was breached in any of the high profile breaches. While this site is not 100% comprehensive, it's a useful resource to see if you have been affected by recent data breaches. Because the data from the VTech breach has not been released, this site will not include people affected there (the list from the VTech breach has been added to "Have I Been pwned"), but it is still a useful resource in monitoring whether your information has been breached anywhere.

And, to state the obvious: now is not a good time to buy any new VTech games.

Tracking We Can't Hear

4 min read

The Center for Democracy and Technology recently filed comments to the FTC on cross device tracking. Their report is a good summary of current practices in cross device tracking, and worth a read. In this report, however, they highlight the use of high frequency tones embedded in television and online video ads to connect people to devices in order to allow marketers to connect a single individual to multiple devices. Because each device holds information about how a person interacts online, the ability to combine these separate views into a single profile allows marketers to develop more comprehensive (and more invasive) profiles of people. Ars Technica has a solid writeup that summarizes these points. The FTC is holding an event dedicated to cross device tracking tomorrow (on November 16, 2015).

As noted in the Ars writeup and the CDT document, multiple companies use high frequency pitches to track users. In September, 2015, one of these companies announced additional VC funding - while the quotations below are specifically about this one company, they are generally accurate about this tracking practice.

With such data related to TV commercials, companies can come up with targeted mobile ads. The technology essentially consists of an audio beacon signal embedded into tv commercials which are picked up silently by an app installed on a user phone (unknown to a user).

A rough profile of user (sic) is then created, containing information about where the ad was watched, for how long did the user watch that commercial before changing the channel, which kind of mobile device is user using and so on.

Just to highlight: the app that picks up the audio signal that cannot be detected by human hearing needs to be installed on a person's phone. That would seem to be a pretty significant barrier, as very few people would willingly install software on their phone for the expressed purpose of tracking them.

However, affiliate deals sidestep this barrier:

The company reportedly has agreements with about 6-7 apps to incorporate this technology in their app to catch signals from TV and claims to have data of 18 million smartphones already. It has already created mobile ads for over 50 brands in six countries including Google, Dominos, Samsung, Candy Crush, Airtel, P&G, Kabam and Myntra.

Based on this report, it sounds like the tracking technology is embedded within other apps. So, when you download an app from the Play Store of the Apple Store, it could have this tracking software silently embedded in it, with no notice to end users. Both Google and Apple could play a positive role here by requiring apps that embedded this tracking software to display a prominent notice to end users on their app pages.

It's also worth noting that, while the stated use is for advertisers to connect multiple devices to a single user, this technology could also be used to track multiple people to a single location. For example, high frequency pitches could be sent out in a mall (tracking people through a store), at a concert, or via any televised display. This would allow a specific device (and the person carrying it) to be tracked to a precise location, even if that person has their location services fully disabled on their phone. This technology would also allow marketers or observers to identify people who were in the same place at the same time.

Intrusive practices like this move marketing solidly into the realm of profiling and surveillance. Technologies like this also make the case for requiring a hardware switch on mobile devices to disable microphones and cameras. At the very least, these intrusive practices by marketers show us that browsing the web with the volume turned off, and muting the television when any commercials play, are best practice. It also highlights how privacy weaknesses in the Internet of Things (most recently seen in Vizio's sloppy business and privacy practice) can be compounded into greater intrusions into our privacy. These intrusions - committed by marketers in their self-described mission to deliver more relevant content - cross the line from marketing research into tracking and surveillance. Many of these practices are invisible, and offer no option to opt out, let alone to review or correct the full profiles amassed on us.

Now, it turns out that in addition to being invisible, the tracking is inaudible as well.

Privacy Policies and Machine Learning

3 min read

Today, Google announced the release of their v2 machine learning system under an open source license. This is a big deal for a few reasons. First, to understate things, Google understands machine learning. The opportunity to see how Google works on addressing machine learning will save huge numbers of people huge amounts of time. Second, this lets us take a look inside what is generally a black box. We don't often get to see how ratings, reviews, recommendations, etc are made at scale. This release peels back one piece of one curtain and lets us look inside.

Before we go any further, it's worth highlighting that machine learning - even with a solid codebase - is incredibly complex. Doing it well involves a range of work on infrastructure, data structure, training the algorithm, and ongoing, constant monitoring for accuracy - and even then, there is still a lot of confusion and misconception about what machine learning does, and what machine learning should do. So, before we proceed any further it needs to be highlighted that doing machine learning well requires (at the very least) clearly defined goals, a reliable dataset, and months of dedicated, focused work training the algorithm with representative data. The codebase can jumpstart the process, but it is only the beginning.

As part of the work we're doing at Common Sense Media, Jeff Graham and I are working with a large number of school districts on a system that streamlines the process of evaluating the legal policies and terms of a range of education technology applications.

The first part of this work involves tracking policies and terms so we can (among other things) track changes to policies to alert us when we need to update an evaluation. There are a range of other observations this will allow - and we have started talking about some of them already.

The second part of this work involves mapping specific paragraphs in privacy policies to specific privacy concerns. When it comes to evaluating policies, this analysis is the most time consuming. Doing it well requires reading and re-reading the policies to pull relevant sections together. While there are ways to simplify this, these methods are more useful for a general triage than a comprehensive review.

However, Jeff has been looking at machine learning as a way to simplify the initial triage for a while. To understate things, it's complicated. Doing it right - and training and adjusting the algorithm - is no small feat. Implementing machine learning as part of the privacy work is a distant speck on a very crowded roadmap. It's incredibly complicated, and we have a lot of work to do before it makes sense to begin looking at implementing machine learning to do the initial categorization. But, announcements like the one from Google today get us closer.

Is This Part Of Your Social Media Training For Kids?

2 min read

While the fact that a variety of data brokers engage in digital redlining is old news, and Facebook's patent on assessing credit via friends could theoretically be dismissed as a future plan, we now have reports that credit rating agencies are using social media to assess credit ratings:

FICO is working with credit card companies to use several different methods for deciding what size loans people can handle, and using non-traditional sources like social media allows them to collect information on people who don't have an in-depth credit history

As educators, if you steer kids towards social media use, how do you prepare them for the reality that their posts are being archived by companies that will use their interactions to judge and sort them for the indeterminate future? Does your social media training and digital citizenship for kids cover how to create an online persona that is as creditworthy as possible? And how do we reconcile the needs for "authentic conversation" against a backdrop where for-profit companies invisibly mine these "authentic" interactions looking for predictors of future behavior? How many of us could withstand the actions of our youth weighted alongside our adult choices? How many parents consider these realities when they share information about their kids online?