Computer voice assistants such as Apple’s Siri, Google Now, and Amazon Echo, have started to proliferate like kudzu. This is something that every geek, especially sci-fi geek, has wanted since we first heard the voice of the computer in Star Trek (of any generation). Interfacing with a computer using a human voice which responds in the same way makes computers more human. It’s not necessarily more efficient but certainly more enjoyable. Not to be left out, Microsoft has developed its own voice assistant, Cortana. For those who are not gamers, Cortana is the synthetic intelligence from the Halo series. This is what she looks like in the games:
Figure 1: Cortana Image Copyright, Microsoft Inc.
So, Microsoft chose to pattern their voice assistant after a game character right out of a fan boy’s dream. This is either horrid sexism or brilliant marketing. Maybe both?
Cortana has one big advantage over all the others – it is baked right into Windows 10 operating system. That means that she (and yes, you will come to think of Cortana as she) is available on just about every laptop or desktop running Windows 10. And Microsoft is doing everything they can to have Windows 10 running on everyone’s computer. Cortana comes preloaded and ready to go to help… with stuff. Computer stuff. That’s right. Google Now is available from the Google web page but that means you have to have Google open in the Chrome browser. Cortana is always there, waiting for you to want something. And she sounds way better than Google Now. More personable and less mechanical. Like a computer girlfriend… which is creepy.
I decided to run Cortana through her paces. It was highly unlikely that she would be perfect. The goal of my testing was to see if she was good enough to use on a daily basis… which sounds terrible. Maybe mechanical is better.
The Good
Cortana has a lot of personality. This makes Cortana seem more real than it really should. Like Siri, it responds in voice to most (but not all) questions. One of the neat things about Cortana is that she can answer typical questions such as “what time is it?” or “what is today’s date?” Cortana will also answer silly questions such as “How old are you?” That question generated a surprising set of answers. At one point Cortana said her “framework was new but her wisdom is as old as the internet” – a joke! She also seemed to scold me when I asked the same question again, telling me that she was a little bit older than the last time I asked the question. It was as if the computer was admonishing me for wasting its time.
You can also ask more complex questions and get answers much of the time. Cortana will accept open ended or casual questions and provide reasonable responses. This further adds to the humanization of Cortana. For example, you can ask “What’s on my schedule for today” and Cortana will reply with a list of your appointments from your calendar. It must be pulling these from the Windows 10 calendar applications. Cortana was obviously blind to calendars that were not configured for the Windows 10 unified app. Cortana can also set reminders for you which end up on the Calendar app and are synced to your online Microsoft account. It helps when an application is Cortana-aware (or Cortana is aware of the application… not sure which it is). I was able to create tasks in Wunderlist by asking Cortana. It was not an easy process, and fraught with mistakes, but hopefully will improve over time.
When Cortana just runs out of things she can do on her own, she can fire off a request to Bing to search for one. That covers a lot of ground. When asked “What time is 4pm GMT in eastern time?”, Cortana knew to send a request to Bing which gave me the answer with search results if I didn’t like what it presented to me.
You can also ask Cortana to find files for you. This has mixed results though. Cortana seems to do fine when the file is indexed on your hard drive but not so much when it is not. It also struggles with filenames with special characters or lacking in spaces. It had no trouble finding a file with “Financial Report” in the name but utterly failed on TweetsFromText.xls.
Figure 2: Response to “What time is it?” Spoken and displayed.
The Bad but Sometimes Hilarious
Cortana doesn’t always understand what I say. Trying to set a task in Wunderlist took an inordinate number of tries. I don’t bother now since it’s easier to type. I know it’s because the voice recognition engine can’t parse a sentence as complex as “Set a task in Wunderlist for tomorrow to pick up milk at the store”. Cortana just stops and adds a task with partial (very partial) information. In the context of a voice assistant, it makes Cortana seem impatient, as if she simply doesn’t have time to deal with the stupid meat computer talking to her. It turns out that the request to add a task needs to be structured in a specific way to work right. That’s completely opposite the idea of natural language interfaces. Cortana should be able to understand reasonable variations on a request.
There is a certain randomness to Cortana’s behavior. In some cases, it’s because an application – or worse, a system function, is not integrated. Asking Cortana to turn down the sound on a computer results in a Bing search of articles on how to turn down the sound on your computer. Adjusting or muting sound seems like an obvious Cortana function. It’s worse when applications that aren’t integrated with Cortana but feel like they should produce no action at all or a Bing search. Asking Cortana to pause Spotify results in her attempting to launch Spotify even when it is clearly running. Cortana obviously cannot interrogate Windows well enough to understand which programs are running.
Further evidence of Cortana’s lack of awareness of its own OS is its program launching failures. A number of times it couldn’t find an application that was right in the Start Menu. It also got confused when encountering application names that were similar. It consistently got Skype and Skype for Business confused and would only launch Skype for Business. Since it was presenting a list of five “Skypes” why didn’t it show regular Skype as well. Maybe I should ask Cortana? More often than not, it doesn’t get the difference between a request for a file, application, and Bing search and will default to the Bing search.
As expected, there are a lot of times that Cortana mangles a command. However, sometimes it kind of understands but gets it wrong with hilarious results. When asked to create a Wunderlist task to “edit blog”, meaning this blog, it insisted on the task being called “edit blah.” I’m not sure if that was commentary on Cortana’s part. At one point I asked it find a file called “Big Thoughts”. It translated “thoughts” into a NSFW slang term for male genitalia. Cortana then searched on it automatically. I’m only glad no one walked into my office at that moment. I’m pretty sure Cortana would have simply stayed mute instead of owning up to her mistake. Or was it a mistake? Perhaps Cortana has a dirty mind or is looking to mess with me.
Finally, Cortana sometimes talks and at other times is silent. If I ask Cortana what’s on my schedule, she will respond by reading all the entries from my calendar back to me. If I ask what time is it, she displays it on a card and sometimes says it but doesn’t say it. Once again, unreliable responses lead to an inconsistent experience.
The Ugly (Sorry Cortana)
The single most irritating, annoying, and downright intrusive aspect of Cortana is the need to turn on location services. Cortana will not operate unless you agree to share your location with… Microsoft, I presume. It’s not clear. There is a benefit to having Cortana know where you of course. It allows her to look up your local weather or find events in your area. Good stuff that should be optional. If I choose to use Cortana without giving up my location, I fully understand that I won’t be able to ask for the weather report. That’s my business not Microsoft’s. I can’t imagine why Microsoft must know my location to present me with my schedule. I worry that it is not Microsoft but Cortana herself that wants this information. That way they can know where you are when the AI driven robot apocalypse happens and target your location. Or Microsoft wants to sell your information to advertisers. Believe whichever wont’ keep you up at night. I’m going with the robots.
What’s also weird is how poorly integrated Cortana is with Microsoft’s business applications. For example, she can send an email using the Windows 10 email application but not Outlook, even though Outlook 2016 is the default email app on my system. There were similar misfires with other Microsoft Office applications and she refused to change my Skype for Business status. It’s understandable that Cortana will not be able to set my status to away on Slack but on Microsoft’s own products? Perhaps they are betting that no business wants everyone muttering into their computers.
The End Result
Cortana’s utility comes from the ability to initiate actions without stopping what you are doing to type. For someone such as myself, this is a real boon. I’m often writing away and want to grab a piece of information or just check the time. Being able to continue working while asking questions is very useful indeed. Cortana is fun because of her personality. Many Siri users talk about how they can ask Siri to tell them jokes and such. Cortana doesn’t do much of that. She is more efficient and Microsoft has positioned Cortana as an assistant and not just a toy.
Unfortunately, Cortana is too limited to realize her full potential. It’s not so much her inability to understand what I say as her lack of awareness of her own system’s surroundings. Cortana doesn’t know what applications are on the computer she resides on herself, is incapable of adjusting simple system functions such as sound volume, and is unaware of too many programs including Microsoft’s own Office productivity suite. This yields an experience that is both limited and inconsistent.
Sorry Cortana but I need you to grow up a little more before you can be really helpful.
