Some people may be confused about Microsoft Research. What is it that your
Well, Microsoft Research is really the basic research part of Microsoft,
so what we do is long term basic research. Think of it like what you are
doing in a computer science department in a university type of
environment. Now we’re doing the context of the company Microsoft where
we can work directly with product group technology a lot faster.
2011 is your twentieth anniversary at Microsoft Research, what do you feel
is your biggest accomplishment?
That’s a pretty long list of accomplishments. I think if you look at the
technical side of things. We’ve made enormous strides in areas that are
directly related in the creation of software. So if you look at the work
we’ve done with program analysis, program proof technology, etc. - we’ve
really gotten to a point now where we can prove significant properties
of programs of hundreds of thousands of lines of code. Its changing the
way people think of about software and the way they think of managing
and creating software - not only inside Microsoft, but outside of
Microsoft as well.
Just today, I got the news that Microsoft Research has won a significant
award from the PLDI conference,
now it goes back more than 10 years. It’s a tremendous amount of
research value. I think more broadly there are so many parts of
Microsoft, there are so many products that have depended on the work
that has come out of Microsoft Research. Obviously the most recent
example is Kinect - it’s the fastest selling consumer electronics
product in history. It was a huge effort by Microsoft Research teams
working with the Xbox team, that really made that happen.
Was there an "aha" moment when you
realized the potential that you had with Kinect?
For me, the “aha” moment was when I
saw my two youngest children, 10 & 12, (my family was on the beta) they
used it for the first time and what was interesting they weren’t just
having fun playing the game and winning. They were having fun
celebrating. They were jumping up and down and their character on the
screen was doing exactly what they were doing. It was that realization,
it was the thing that was driving the experience for them that they were
physically in the game, they were able to project themselves in a very
real way, and to make a connection that people have never made with
computers before. And seeing that, and seeing the faces of these kids
and that joy that I brought them. I thought “ok, I think we are really
going to win with this.”
How do you think the upcoming release of the SDK will drive this further?
What I’m hoping is, the SDK that we
are releasing, in just a bit now is really intended for research and for
experimentation. It is not a product SDK, it’s a research SDK. My hope
is what it will do is get people to really leverage work and technology
with capabilities that we have in Kinect and to do more interesting
projects and experiments. People have been really excited about Kinect
because it’s It really shows that computers can do something, that
people didn’t think computers could do before. Researchers and people
that who like experimenting with computers have been jumping on that,
and saying here’s a very inexpensive piece of technology I can do
experimentation on. What SDK will do is open up a significant set of new
capabilities to those people who have not had access to before, in order
to be able to try and do new things.
So as you said Microsoft Research is the “little r” in Microsoft’s huge R&D
total budget, as the total budget is nearly 10 billion dollars. Still being
the “little r” what value do you think Microsoft has received after their
continued ongoing investment in research?
Well our research is used every
product we own, and look how Microsoft has just been, look back at the
last 10 years, enormously dynamic in terms of growing its businesses.
We’ve gone from 2001 with 25 billion in revenue, now we are at 68
billion. That’s incredible growth over a relatively small time. And
I’ve always been drive by the technology and the kind of research that
has helped us build and make popular things like Xbox. Xbox Live has
the capabilities - that no one else still has to be able to do the
things like Xbox Live does. Look at the technology that we have put into
SharePoint and has driven that product (a multibillion dollar product),
you look at the technology that has gone into Windows and Office and the
things that has allowed them to do what they are able to do and allows
us to build them more reliably and more consistently over time. A lot of
our technology has gone into to helping the company do its job better,
as well as help the company make the products better. So there has been
a huge amount of value, take things like Bing, because we have Microsoft
Research, when Microsoft as a company decided to compete with Google in
that way in order get the competitive edge in that market very quickly,
more quickly than our competitor really expected us to. As we introduced
Bing as a product and a vehicle its really allowed us to differentiate
and to give us value that again, Google doesn’t have. There is a steady
market acceptance of Bing. I think again that’s a testament to the fact
we have the world’s best information retrieval group that we can put on
problems like that.
I’m glad you brought up Google, Steve Jobs has said “our friends up north -
all they are doing is copying Google & Apple.“ What would you say to that?
Well I think if you actually look at
the business we are in and the things we can do, really there is no
evidence of that. If you look we’ve gone back to fifteen years ago and
Microsoft has very small presence in the enterprise space. And today
people look at Microsoft with look of hate, that we are a large
enterprise company. Well its only been because we have been able to
build so many businesses in that space. Over the last 10 years we have
created over 8 new billion dollar businesses. That’s a lot of
innovation that’s a lot of change over a relativity small period of
time. If you look at Kinect for example of something when it was in the
consumer stage, that came out of nowhere I think for everybody. This is
significant example of real time computer vision applied in a
significant, in this case consumer setting. And again, no one had any
notion you could do it. One of the things that is great about Kinect is
it was so surprising. People stood in front of it – played with it –
and suddenly realized computers could do something previously not
What is interesting about Kinect, again, it is not just a great product
in its own sense; it is literally opened up a whole new area thinking
about what computers can do. It has excited the research world, the
business community. Everyone is thinking, if computers can see, and
react in real time to the actions of the user – what does that allow us
to do now? People have talked about natural user interfaces for some
time – this is probably the first palpable step towards a really natural
interface where the computer can track you and understand what you are
doing in an environment.
Being the lead of over 850 researchers across six worldwide labs – what is
your typical day like? What is the best part of your job?
The absolute best part of my job is
spending time with the researchers that I have. There are so many cool
ideas in technology that we are working on. The way I manage my
researchers – is I try to hire the best possible people and I give them
a great environment to work in. We don’t tell anyone what to do, we do
not have any mechanisms for controlling or pushing people in one
direction or another. We are relying on the fact that we hired the best
possible people and that they are going to find the right areas to work
on. They just surprise me. I can remember the time I had a lunch with
one of my researchers a number of years ago and he said “oh by the way,
we have been working with a cancer center on an AIDS vaccine” I was like
“OK - that seems a little far afield for us”, then he started to
explaining that one technology we created for Spam detection that was
being used in Exchange and Outlook is equally applicable for this space.
He was basically saying – here is this virus that is constantly changing
and attacking you, right – how do I recognize the similarities between
the things that are attacking and the things that are normal. Which is
exactly what a Spam problem is….at that point it began to make a lot of
sense and I could see the relationship. I still couldn’t see how that
would this would impact Microsoft businesses. But, as time goes on, as
things usually happens; now Microsoft has a healthcare group. And they
are very much using these kinds of technologies as part of their
business intelligence part of healthcare solutions.
How has Microsoft Research contributed to the development of Microsoft’s
cash cows – Windows and Office? How did you save Windows/Office in 1995
during the transition between 16-bit and 32-bit software? Who told you the
changes would be “over my dead body”?
Well, we had this issue as we were
moving from 16 to 32-bit that the 32-bit code pretty much is twice the
size as 16-bit. Normally, there would have been a nice memory price
curve headed downward. In fact, memory prices kind of spiked right
before 1995. So, as a company we were put in a situation where we had to
try to sell Windows 95 and Office 95 products into business PCs that
only have either 4 or 8 MB of memory, and that hadn’t changed much from
the days of Windows 3.1 or Windows 3.11. That was a big problem for us.
Luckily, going back to 1992 we had developed the technology that
basically optimized the working size of 32-bit programs and effectively
got that factor of 2 back. That was critical in being able to release
Did you know you were going to need that?
You could argue the fact or not
those researchers thought it would eventually be needed or not. We were
working on it because it was an interesting problem to solve. We tried
to get our product groups interested in it - Dave Cutler of the
Windows NT group said “over my dead body will this be included in our
product.” But in reality, it wasn’t unreasonable for him to say that
because the reality is whenever you are introducing a new technology
into an environment it produces a risk. The question is always is the
risk worth it or not? Because of the set of circumstances, which no one
had necessarily predicted, the risk is worth it. Suddenly you are in a
situation because of external forces you have to solve the problem,
luckily, because you have been in basic research – we had a solution and
that gave us the opportunity to solve the problem for the product group
at a critical time.
That in general is something you
always see of research. That same thing happened with Kinect. You can’t
do research based on what a product team needs right now. Because it
won’t be done soon enough, it takes too long. You need to have hired
great people, build the capacity, created the IP, created the artifacts
– solved some of the problems. So when someone comes to you, you can
either adapt something you have already done or you can use what you
already know to solve this new element of the problem. But now you can
do it fast enough so the product teams can actually use it. That’s
something people usually forget about it. That’s the reason why you
don’t want to direct fundamental basic research – it is solving problems
you do not know you have.
This is a philosophical point, but
if you go back to the writings of Vannevar Bush, whose writings and
efforts in part helped created the National Science Foundation. His
arguments for making investments in basic research were like “look, we
just went through a war, a lot of the reasons we were able to survive
that war were due to technology and bring technology to bear, and
investments in previously made research. We don’t know when there will
be a new war or famine or disease, we don’t know when that will happen
but we want to make sure when the time arises we will have the
technology, know-how, and people that can help us survive.
At a company like Microsoft, the
same thing applies. Look at the history of the company’s technology.
How much 20 years ago is still here – how much from 20 years ago are
still here in a recognizable form? There is so much change, the market
is constantly is adjusting to ideas, approaches and market conditions,
new technologies. You need to have that upfront set of investments, so
when the time comes and you have to change – you have the technology,
know-how and people to do it. It has been something at Microsoft that
has been a huge advantage to us over the years, it has allowed us to
survive situations that were difficult, and allowed us to be involved in
new businesses that we could not be in any other way.
We are working with people on
Malaria, Hepatitis, Memory Loss, Asthma, and others...You’d be surprised
using our research involvement in Computer Science to tackle some of
these hard problems in other areas. Increasingly a lot of biology looks
like information processing, and people realize genetics has a lot of
string/variation processing, so there is an opportunity for us there.
And it goes both ways – we have some researchers in our Cambridge,
England lab who, for example, have been learning how cells function and
adapting to programming languages and how to take our expertise in
software and how to program cells, for example to fight cancer, etc.
You’ve written some 600,000 lines of
code during your career. What is still in use? You've mentioned Allegiance
– which we of course covered on our site many years ago.
It is still being used – there are a
bunch of people on the Internet still using it. At the time, it was one
of the best reviewed products Microsoft had put out. The sales were
disappointing, I think in part, because it was Microsoft’s first online
only product and no one knew how to sell it. It was an experiment for
It is fascinating how that
particular game came to be, there was a point in time after Windows 95
was released that the guy who ran the DirectX team reported to me. That
was due because there was a lot of research involved in it, I was always
running research but this was a side job for me. In the 1970s, I created
one of the first networked computer games – called Alto Trek. I decided
I wanted to learn the DirectX API my team was using, and wanted to
experiment with it. So I took that old game and ported it to the DirectX
interface. It was fun, but I just ended up with the old game – which was
kind of boring. So I decided to make it in color, 3D, etc…more modern
looking, but still wasn’t very good because my ability to do artwork is
close to zero. Ed Fries, who at the time was running the Games Division,
I knew him pretty well – he had seen it. He said it was a interesting
idea – so I got this designer that would be great working on it, so the
two of us started going at it, and a few years later we shipped the
game. The game was a little ahead of its time, maybe.
Does the Microsoft Research group ever interact with the Microsoft Most
Valuable Professional Program?
Not formally, but certainly our
people are out interacting with community users, doing presentations and
events, etc. We also put out a lot of code which a lot of the MVPs test
and provide feedback on, which is very helpful.
ActiveWin.com (Bob Stein):
Nathan Myhrvold, the man who hired you, went on to write the Modernist
Cuisine. Do you see cookbooks in your future? What do you see next for
My joke with Nathan, when I used to
report to him, was that he always wanted to go to the gourmet restaurant
while I was happy eating at McDonalds. So, I don’t think a cookbook is
in my future (laughing), but I do make a good Barbeque. I don’t see us
getting into Modernist Cuisine though.
Discovery News published this week news of Microsoft Research’s
development of a human antenna. This story highlights some of the very
exciting (and interesting) projects Microsoft Research is developing. What
other exciting projects come to mind?
We have experimenting with the whole
area of natural user interface. There are a lot of opportunities to
explore how users can interact more naturally with people, how to be
able to communicate more. So, there is some of our work of looking at
micro-movement of muscles and the electrical signals in your body and
use that control interfaces or interact with things of that kind – we
can do some mean air guitar that way, referring to one of the demos we
have done in the past.
We are at a point of time where
people are experimenting with new forms of interaction, and obviously we
are doing a lot of the work ourselves.
In the past 20 years, is there any research that you would have liked to
bring to market but for some reason it didn’t come to fruition?
The reality is that almost
everything gets used somewhere, somehow, some day. Circumstances often
dictate when a particular technology becomes suddenly important. Take
for example, something like real time 3D computer vision. We have been
working on that since the 1990s. We certainly tried to get people
interested along the way, but then circumstances suddenly come up. The
incredibly talented Xbox product team came to us to tackle this space,
then all of the work we did became relevant. They were doing great work,
and our people were working extra hard because we had such great
partners. It went back and forth, a lot of great opportunity for us.
That is the way a lot of things
work. You come up with an idea, time is not right, later on though the
idea might have real value. In technology in general, there are many
cases where technology ideas are discounted at certain point in time,
then a number of years later will suddenly become important. You always
have to be thoughtful of fact that just because it was a bad idea once,
doesn’t mean it is still a bad idea. Things change. Underlying
technology, circumstances change.
I remember for a long time people
said “we’ll just never make these multiprocessors work, always will be a
problems, code is an issue, etc.” Circumstances conspired to make it an
absolute requirement – not is no longer optional. We build these
operating systems, databases, products like Exchange, etc. they have to
be multi-threaded, they have to work in these environments. Otherwise,
you are dead. Clock speeds have not kept over, but the number of cores
have. Parallelism is an absolute requirement now. Suddenly, all of the
work from twenty years ago that was sitting on the shelf because clock
speeds were going up and everything was great – that’s not happening and
more and now all that work is important.
I think it’s a treasure trove of
intellectual property, ideas, people, etc. that you build up over time.
Sometimes it is a war chest,
sometimes it is your first aid kit. It can be any of those things.
Sometimes its your new set of tools to tackle a problem, which is new,
and no one worked on it before.
The key thing is being ready. And being receptive to change. Change is
the key element in our industry. I look back – you wouldn’t know, my PhD
thesis was actually in 3D computer vision . I look back to what we we
were doing in the 1970s in that space, and then you look at Kinect. Then
I say “we were really dumb”, but maybe we just didn’t have the tools –
yea, probably didn’t have the tools. We weren’t as smart as the people
are now. I am impressed with the quality of people graduating out of our
schools. I was talking with some older computer scientists, on the
advisory board for the Engineering school at Stanford. I don’t think we
could compete today. We look at how smart these young people are, the
knowledge they have, how much they know, it is a very different world
today. That’s the future – in the kids.
Speaking of universities, congratulations on your honorary doctorate degree
you received this past weekend at CMU.
How does it feel to be recognized at such an accredited university?
For good or bad, I am at the point
in my life where I am starting to accumulate a large number of honors.
It may be a little bit of a negative. But I think it is great, I have a
strong personal connection with Carnegie Mellon. I was a professor there
for twelve years, it was a very formative point in my career. The
university, not as an entity, but the people, really gave me a great
opportunity to succeed and supported me. It feels great to be honored by
this particular university because of that connection.
Please Comment On This Interview