I’ve been working in Silicon Valley since 2000, at first as an engineer and then as a …. well, that’s what we’re here to talk about.
Over the last decade I’ve seen increasing use of the title “Scientist” for a certain kind of employee who works on large consumer Internet services (like Google, Facebook, Twitter, Yahoo, Amazon, etc.). The profile of these people is that they typically (but not always) have a Ph.D. in Computer Science or a related discipline (Statistics, Physics, Math), they are mathematically facile, they are well-versed in machine learning and other statistical techniques, and they are comfortable with the analysis of really large data sets. (I’m talking here about the “applied scientists” who work on product, not the small number of people in research labs at these companies who really are judged by a “publish or perish” standard).
They are also distinguished from “engineers” by what they don’t (usually) do. They may or may not write production-ready code that is expected to be polished, and bug-free, and performant, and ready to run on many machines at once without causing problems. The output of “scientists” can be anything from proposed algorithms (to be implemented for real by “engineers”), to prototypes, to analyses of data, to metrics code, and (especially) to carefully-crafted machine-learned models that are applied at run-time by production code (and which engineers are not trained or qualified to build).
The “Scientist” title has always bothered me a little, both because it feels like an inaccurate use of a perfectly fine word, and because it feels like grade inflation.
It feels inaccurate because I think of scientists as people who figure out new facts and principles about how the the natural world works, and I think of engineers as people who apply that scientific learning to create useful artifacts. By those definitions, anyone working on, say, making Google Search better is clearly an engineer, not a scientist. As my friend Sameer put it: “They’re not looking for no Higgs Boson”.
It feels like grade inflation because I can see how it happened: you had half a generation of very smart people graduating with Ph.D’s, many of whom had always planned to be scientists, and then found themselves in a world with a few low-paid postdoc positions and great job openings in a booming Internet sector. The only hitch? “But I’m a scientist!”. Hiring manager: “No problem, we need scientists too!” (editing the job title as he speaks).
(The somewhat more recent term “Data Scientist” bothers me a little less – more specific, more modified – but it’s only very recently indeed that you see that as a literal job title. Previously you might have someone informally described as a “data scientist” but the literal title would be “Scientist” still.)
If I had my druthers I would call all of us Internet tech types “engineers” and then divide further into “software engineers” and “data engineers”. (This is where the “real engineers” (civil, mechanical, aeronautical) pop up and sniff “Software engineers aren’t even engineers! They don’t even have to be certified!” OK, OK – I’m sure there was grade inflation in that shift too, but I’m not willing to go with “code monkey” and “data monkey”.)
Now, have I convinced you that a (Data) Scientist isn’t really a scientist? Good, because now I am going to change my mind and argue in the other direction.
The reason is that most data scientists employ a very particular kind of Method in their work. Can you guess what kind of Method? (Sure you can.)
A lot of data scientists end up improving product by going through the following cycle (and here I am not giving away any trade secrets particular to my company – I have seen this done in enough different bigcorps that I’m sure it’s ubiquitous):
1. Develop a theory about what kind of change might improve the service.
2. Test the theory as best you can with offline data.
3. If offline tests look good, implement a version of the service that implements the change
4. Expose the new version to a slice (or bucket, or shard) of the service’s users. This is usually done by mapping user IDs to that slice persistently, so that a given user will see the new version with every request, over the space of a week or a month.
5. Keep a control slice with a disjoint set of users that see the old version of the service.
6. Now, compare the two slices using whatever metrics you have defined for “improvement” of the service. It might be number of clicks, or time on the site, or transactions performed, or money spent – it depends on the particular application and business.
7. Ask: is there improvement? And is the improvement statistically significant? If so, take it that the “theory” in step #1 is provisionally validated.
Now let’s call this cycle by its right name: this is a “double-blind controlled experiment”, not that dissimilar from the experiments that pharmaceutical companies use to study new medicines. (Do people who conduct such drug studies call themselves scientists? I’m guessing that they do.)
It may not be obvious at first that it is blind in both directions – after all, the scientist could in principle know all the affected user IDs – but in practice nothing can be done to treat the control population differently from the experimental population aside from the change they are trying to study.
So are these folks engineers or scientists? They mostly have Ph.D.’s, and they use the scientific method (double-blind controlled study) in their work. But what they are studying are artifacts (not the natural world) and they are all about making the artifacts better, not about enhancing the store of human knowledge. To improve the artifacts they use all the fruits of scientific research, one of which is … the Method itself.
So I say “Engineer”, but if you like the S-word better then that’s OK with me – you’ll get the same number of free lunches either way, right?
Tim Converse 