Gamasutra is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
September 21, 2020
arrowPress Releases







If you enjoy reading this site, you might also want to check out these UBM Tech sites:


Sponsored Content
Sponsored: Integrating Cortana with Unity in Windows Universal games (Unite 2015)

Sponsored: Integrating Cortana with Unity in Windows Universal games (Unite 2015)

August 10, 2015 | By Kevin Ashley, Microsoft Game Evangelist

August 10, 2015 | By Kevin Ashley, Microsoft Game Evangelist
Comments
    1 comments
More: Programming, Sponsored Article



This article brought to you by Microsoft

IMPORTANT: This demo shows a Windows 10 Universal app built with Unity and demonstrating Cortana integration. At Unite 2015, Unity made available its public roadmap which also includes Windows Universal apps support here. Making this demo was so much fun, because Cortana naturally blends with games, and in some scenarios you can significantly improve the user experience with your game by adding intelligent voice features.

This is the sample I presented at Unite Europe 2015 event to demonstrate how to integrate Unity with Cortana in Windows 10, among other cool things, such as using Unity mecanim system for animation. Fortunately, it’s very easy to do: if you are a Unity developer, integrating Unity with Cortana can certainly add flavor to your Windows version of the app. And for users it just means more fun! So let’s get started…

The Idea: Mecanim and Cortana

I’ve been thinking of combining Unity mecanim animation with Cortana to make my animated character react to human voice commands. Cortana, besides being a personal assistant and just a friendly artificial intelligence inside my Windows devices can also help with basic speech things: parsing human voice commands and as it turns out, with Unity games. Here’s what my app looks like:

Cortana integrated app built with Unity

Cortana integrated app built with Unity

I’m using a free asset from Mixamo for my animation character (in fact you can use any character with animations from Unity Asset Store). I want to activate Cortana with standard Windows “Hey, Cortana” and then make my character dance, walk, run etc, by speaking a voice command. Easy?

Dancing avatar

Dancing avatar

You can find the code and also instructions for this example at the bottom of this post. Instructions:

1. After loading the Unity project, open the MainScene scene.

2. Now let’s build a Windows specific project. Open Build Settings. Since we will use Cortana in our project, click Player Settings and make sure Microphone is checked.

3. Now, select Windows Store output, then choose Universal 10 as a target. If you do not see this target, then you don’t have a Windows 10 integrated Unity. Make sure you select XAML as an output project type.

4. Click Build…, Unity will prompt you for output folder, this is where a native Visual Studio project will be generated.

5. Make sure to make the following changes to App.xaml.cs

Make AppCallbacks public instead of private:

public AppCallbacks appCallbacks;

Change OnLaunched event as follows:

protectedasyncoverridevoid OnLaunched(LaunchActivatedEventArgs args)

{

splashScreen = args.SplashScreen;

try

{

StorageFile vcdStorageFile = awaitPackage.Current.InstalledLocation.GetFileAsync(@”CortanaCommands.xml”);

await Windows.ApplicationModel.VoiceCommands.VoiceCommandDefinitionManager.InstallCommandDefinitionsFromStorageFileAsync(vcdStorageFile);

awaitSpeechHelper.InitialiseSpeechRecognition();

}

catch (Exception ex)

{

System.Diagnostics.Debug.WriteLine(“Installing Voice Commands Failed: “ + ex.ToString());

}

InitializeUnity(args.Arguments);

}

Then, update OnActivated event to include voice command activation:

protectedoverridevoid OnActivated(IActivatedEventArgs args)

{

string appArgs = “”;

switch (args.Kind)

{

caseActivationKind.Protocol:

ProtocolActivatedEventArgs eventArgs = args asProtocolActivatedEventArgs;

splashScreen = eventArgs.SplashScreen;

appArgs += string.Format(“Uri={0}”, eventArgs.Uri.AbsoluteUri);

break;

//Add VoiceCommand start detection and use the SpeechHelper handler

caseActivationKind.VoiceCommand:

SpeechHelper.HandleSpeechCommand(args);

break;

}

InitializeUnity(appArgs);

}

6. Now, drop CortanaCommands.xml, SpeechHelper.cs, ToastNotificationHelper.cs from WindowsSpecificCode folder to your project, build and run it. For your convenience I also added App.xaml.cs file.

7. When the project starts, you can use Windows 10 Cortana to give it commands such as “Avatar dance” or “Avatar run”.

Next steps:  download the code, install the Windows 10 SDK and check out our updated Windows 10 DevCenter for Game Developers.

And if you are a Unity beginner, then check out this Microsoft Virtual Academy course done by some of my friends at Microsoft for some starter files and another neat tutorial.



Related Jobs

Deep Silver Volition
Deep Silver Volition — Champaign, Illinois, United States
[09.17.20]

Senior Engine Programmer
Deep Silver Volition
Deep Silver Volition — Champaign, Illinois, United States
[09.17.20]

Senior Technical Designer
Random42
Random42 — London, England, United Kingdom
[09.17.20]

UE4 Technical Artist
Evil Empire
Evil Empire — Bordeaux, France
[09.17.20]

Senior Technical Developer









Loading Comments

loader image