//Build 2018 – Conversational AI: Best Practices for Building Bots

Conversational AI: Best Practices for Building Bots
Elaine Chang, Vishwac Sena Kannan

This session focused on the best practices to build bots at every step of the development cycle.
And breaking news, Bot development follows the same best practices as any other development!

You should define what you want to achieve with the bot. It must add value, be accessible and be effective.

20180509_215219577_iOS.jpg

You should include domain experts when designing your bot and iterate with some final users to make an effective design. Also plan how your bot should react, but do not over do it at the start. Start simple, and grow in complexity step by step later.

For example, you should plan simple interaction at first, no AI. And make sure to measure the users interaction with your bot to have successfully core features. Later you can think of adding adaptive cards for faster and prettier interactions, then move on language understanding and other advanced features.

20180509_220304419_iOS.jpg

You should also set success metrics for your bot to achieve, making sure your bot is useful to your users.

When planning your conversation dialogs, always think to add escape hatches to let your users start over, especially if your bot is doing something critical.

20180509_220444409_iOS.jpg

Finally when your bot is deployed and used, you must have some ways to monitor it and get feedback from your users.

And always take those metrics into account to enhance your bot over time.

20180509_223511040_iOS.jpg

//Build 2018 – Conversational AI: What’s New?

Conversation AI: What’s New?
Chris Mullins, Darren Jefford

As a bot developer, I waited a lot from this session and wasn’t let down! So much new cool features in this BotBuilder SDK v4.

Unfortunately it also renders a lot of the BotBuilder SDK v3 obsolete. A lot of problems that I’ve tackled down with difficulties are now handled by a simple turn key feature.

New features of the BotBuilder SDK v4:

  • Turnkey authentication: Just like WebApp, we now can configure authentication into the Azure portail for our bot, so that it can request the user to connect at the right time.
  • New languages supported: Java and Python
  • New command line tools: « msbot » (manage configuration file of the bot), « luis » (manage Luis), « qnamaker » (manage QnA Maker), « az bot » (command to create/manage bot resources on Azure – useful for automated deployment)
  • LuDown and LuDown CLI – a Markdown variant to give a training sets of data to Luis. « ludown.exe parse toluis » parse the Markdown file and configure Luis with the parsed file.
  • BotBuilder now runs on ASP.NET Core!
  • Middleware (ex. Machine Translation Middleware that translates whatever language the user use to english for the bot to handle, then the bot speaks english but it’s translated to the user’s language through the middleware)
  • Visual Studio has a new Bot Template for the BotBuilder v4

The Bot Framework Emulator has been redone entirely. It nows supports plugins and transcript files. Thanks to the Luis and QnA plugins, we can train our AIs directly from the Bot Framework Emulator based on what we tell our bot.

Luis.ai also packs a bunch of new features:

  • Integrated experiences: Luis new gives others’ API data in a single call (Sentiment analysis, Automatic Machine Translation)
  • Luis has been enhanced for better privacy (GDPR and such)
  • New ways to recognize keywords: Patterns (I want to fly from [city:origin] to [city:destination]) / Regex / Key phrases
  • Luis gives guidance to enhance its training (ex. « There’s not enough utterances for this intent – Bad training », « This utterance is duplicated/too close to another one from another intent »

QnAMaker new features:

  • General Availability since monday
  • Hosted by our Azure Subscription
  • No more throttling of questions
  • GDPR and other privacy laws compliant
  • Better portal. Same Test/Inspect tools as Luis

New Dispatcher tool that connect to LUIS and QnAMaker to check if everything’s alright.

New Project Conversation Learner AI from Microsoft Research, an AI that can predict what to say based on the ongoing conversation with a very simple and user-friendly configuration portal.

New PersonalityChat feature that allows to adapt the way our bot talks to the user based on a setting (Business, Friendly, Humorous).
Ex. User: « Hello », Bot: « Hello, how are you today? »
Ex. User: « Yo », Bot: « ‘Sup »

Links:
http://aka.ms/PersonalityChat
http://aka.ms/BotBuilderOverview

//Build 2018 – Developing for Sets on Windows 10

Developing for Sets on Windows 10
Raymond Chen, Jason Watson

This session was about how to adapt our Win32/UWP apps to provide a great experience on Windows Sets and Windows Timeline.

Windows Sets allows to group apps together in a single window, much like a web browser do using tabs.
With this, we can group all apps related to do a single task.

For example, I have my solution opened in Visual Studio, I debug my website in Firefox, but I need to search a few Stackoverflow topics, while having my Outlook and Slack opened to stay connected to my colleagues.

I could group Visual Studio, my website and the stackoverflow posts inside a single set, while having my Outlook/Slack or anything else inside another Set.

Windows 10 can automatically restore a previous Set and/or tab from the Timeline.

In this session, we were told that in order to provide a great experience with Sets to our users we should:

  • Support multi instancing: This is the default for Win32 apps, but not for UWP. Only a single manifest option is required to enable it
  • User activities: We should store the user’s activities (such as interacting with an item in the app) in the Microsoft Graph. That way Windows 10 is able to display it into the Windows Timeline and provide the user with a really quick way to resume what they’ve done
  • Support Window grouping: When our app need to open another app, we can specify if it should be opened in the same Set or in another one. The user can set their preferences globally in the Windows Settings
  • App visuals: Our apps should customize its tab’s color and icon in the Set. Fortunately it is the exact same API than the Custom TitleBar API of UWP
  • BackButton : Since Windows Sets introduces a new chrome for our app’s window. We should use the normalized BackButton provided by Microsoft

API wise: the ones for UWP are really simple.
Each bullet point is max 5 lines of code.
For Win32 (WinForms/WPF), the API is quite complex. We need to do P/Invoke call.

Links:
http://aka.ms/DesignForSets
http://aka.ms/FluentBackButton

//Build 2018 – What’s New in Xamarin.Forms 3.0

What’s New in Xamarin.Forms 3.0
David Ortinau

In this quick session (45min compared to 75min normally), David talked both about the recent past of Xamarin.Forms, the new Xamarin.Forms 3.0 and what to expect in the future.

Past features and improvements since last year:

  • First Public Roadmap: http://aka.ms/xfroadmap
  • Native Forms: Embedding Xamarin.Forms inside native UI
  • .NET Embedding: .NET code and Xamarin.Forms UI usable by native-native language like Objective-C
  • .NET Standard 2.0 support
  • Fast renderers, Layout compression, Startup optimization XAMLC, etc…

The new features of Xamarin.Forms 3.0 (available today)

  • VisualStateManager: We can now apply styles to our app based on states, just like UWP. This is useful to handle different screen sizes
  • Right to Left support with a single property (FlowDirection= »{x:Static Device.FlowDirection} »)
  • Improved ResourceDictionaries: There’s now MergedDictionaries, hurray!
  • FlexLayout: The equivalent of the CSS Flex layout. This allows for fewer UI layers and so performance boost
  • CSS support: A mostly valid CSS that support currently almost all the control properties

The future of Xamarin.Forms:

  • XAML Standard: Announced last year, we will have access to the first bits with Xamarin.Forms 3.1
  • Compiled Binding (aka x:Bind) coming to Xamarin.Forms. That’s amazing!
  • Project F100: Xamarin will take the top 100 annoyances and try to fix them. Such as the missing Border properties on all controls.

Xamarin.Forms 3.1-pre1 is available today !

//Build 2018 – Behind the scenes: Sets on Windows 10 and Your Phone

Behind the scenes: Sets on Windows 10 and Your Phone
Megan Brown, Shawn Henry

This session showed the features of Project Rome, which exists for quite some time, and how Microsoft leveraged those inside the new Sets and Nearby Share features of Windows 10 and the Your Phone app.

20180508_220917710_iOS.jpg

Project Rome aims at simplifying cross device interaction. It supports a wide range of devices: phone, PC, Xbox, etc.
It allows those devices to communicate between them, being either near by to each other or connected to the cloud.

Project Rome also uses Microsoft Graph to share data between devices.

An SDK is available on Android/iOS/Windows, but the Xamarin SDK is outdated and will be available by next year.

The session was separated into 5 key features:
20180508_221726332_iOS.jpg
Nearby Share allows to share data or files between devices that are near each other. Project Rome will automatically choose the best way for those devices to communicate (Bluetooth, Wifi, Cloud)
Useful for sending a picture from a phone to the PC
Only a handful lines of a code are needed to add this feature to our apps.

20180508_222639812_iOS.jpg
Discovery and Launch allows to discover our devices connected to the cloud and run commands on them at distance.
For instance, we can ask our PC to open a word file directly from our phone.

20180508_223445901_iOS.jpg
This one uses Microsoft Graph to store the activities of the user (such as opening an item in our app) and retrieving it elsewhere. Windows Timeline uses this feature.

20180508_224225138_iOS.jpg
AppService API allows to call a background service on another device (Android or iOS) and retrieve a result. This uses FCM on Android.

20180508_224937350_iOS.jpg
Remote Sessions allows to create a session between multiple devices and to send and receive data while the session is active. This way we can put live collaboration inside our apps.

20180508_225240594_iOS.jpg

//Build 2018 – Visual Studio and Xamarin: The future of app development

Visual Studio and Xamarin: The future of app development
Miguel de Icaza

In this dense but really interesting session, Miguel gave us a complete tour of how to do mobile app development in Xamarin today, as well as what will be the new features of Xamarin.

For the simplicity of this article, I will only list the new tools available and what will be available in the next version of Xamarin.

Don’t hesitate to watch to great session on Channel9 to see all the key points to take into account when designing your mobile apps to optimize user retention.

Announcements :

    • Simplified templates with a new Visual Studio Wizard
    • Automatic Android SDK Management in Visual Studio
    • The XAML editor of Xamarin.Forms is now the same as WPF/UWP
    • Android Designer (Split View Designer of AXML files) now available as a Preview Release
    • Faster build/deploy time when debugging Android apps with Eager Deployment (build and deploy are made in parallel
    • Hyper-V now supports official Android Emulators
    • Visual Studio can now simplifies iOS provisioning by using fastlane
    • Better profiling tools are available in Xamarin
    • Xamarin.iOS can now be compiled with a Full Static Mode to have better performance and memory usage
    • Xamarin.Forms 3.0 is released and comes with CSS / Flex Layout / Visual State Manager / Support for Right to Left languages
    • Xamarin Essentials is a new NuGet package containing most of the most used features for simpler cross platform developments
    • Elmish.XamarinForms is a new framework to make Xamarin a more functional friendly environment (with F#) by writing your app like React (functions return a new view instead of modifying it)
    • Ooui is Xamarin.Forms in the Browser thanks to Web Assembly

20180508_204350043_iOS.jpg20180508_205215943_iOS.jpg20180508_205421734_iOS.jpg20180508_205757867_iOS.jpg20180508_211729360_iOS.jpg20180508_212117449_iOS.jpg20180508_212242391_iOS.jpg20180508_212525352_iOS.jpg20180508_212939499_iOS.jpg

//Build 2018 – Building event-driven Serverless Apps with Azure Functions and Azure Cosmos DB

Building event-driven Serverless Apps with Azure Functions and Azure Cosmos DB
Rafat Sarosh

In this session, Rafat Sarosh explained how to make a completely serverless backend with only Azure Functions and Azure Cosmos DB (with its ChangeFeed feature).

Despite being a very trendy subject, I found this session to be a little boring.
Not enough demos, not really explaining why use Azure Cosmos DB, and the speaker was not dynamic enough.

Azure Functions is part of the new FaaS offering of Azure that help make truly serverless development.
20180508_173113225_iOS.jpg

Azure Functions has 2 runtimes available. Version 1.x which is already GA (General Availability) and 2.x which is currently in preview.

Rafat demoed the tooling in the func CLI, Visual Studio and the Azure Portal to create new functions from templates. Those have been around for quite some time now.

Despite Azure Functions being serverless, you still have the choice to host them on your own App Service in Azure. This has the advantage to control a few scaling/performance options like AlwaysOn. By default, the Hosting Plan (no App Service) will put your functions to sleep if not used for 5 min.

20180508_175144928_iOS.jpg

Azure Functions has a trigger for Azure Cosmos DB that allows it to monitor for changes with the ChangeFeed functionality of Azure Cosmos DB and call a function when new content is available.
That allows to create automated workflows.

20180508_182239971_iOS.jpg

At the end of the session, 2 companies (Johnson Controls and Asos) explained their usage of both Azure Functions and Azure Cosmos DB.

20180508_182643760_iOS.jpg

//Build 2018 – .NET Overview and Roadmap

.NET Overview and Roadmap
Scott Hunter, Scott Hanselman

Announcements:

  • Visual Studio 2017 15.7
  • Visual Studio for Mac 7.5
  • .NET Core 2.1 RC (https://aka.ms/DotNetCore21)
  • .NET Core 3 (Preview later this year)
    • WPF/WinForms can now uses .NET Core instead of .NET FX
    • .NET Core App Bundler (make a single exe containing all dlls)
  • .NET Conf 2018 (September 12-14)
  • Azure SignalR Services (as a Service)
  • New debugging option for WebAPI (http cli)
  • Functional tests with WebApplicationFactory
  • Navigate to source of a NuGet package or decompile sources from a DLL
  • Better support for Editorconfig
  • Blazor (with C# dlls into a browser)

For this session full in announcements and demos, the Scotts started with the new .NET Core 2.1 which is available in Release Candidate today.

20180507_215622057_iOS.jpg

This new version features better performance, both at compile and run time.

20180507_215443781_iOS.jpg

20180507_215526797_iOS.jpg

Next was the Azure SignalR as a Service. This is a new service to allow provision a SignalR server that benefits from the scaling and resilient features of Azure.
This was demoed with a Trivia app that the session attendees could play live while the speakers were managing the app from their localhost. All messages going through the Azure SignalR service.

20180507_215816886_iOS.jpg

We were shown a currently in development new option for debugging WebAPIs.
It’s a command line with commands to list all endpoints, query the API, and debug it.

20180507_220848228_iOS.jpg

Next was the announcement of .NET Core 3.020180507_221753313_iOS.jpg

.NET Core 3.0 can now be chosen as the runtime for Win32 apps (like WinForms) and WPF.

20180507_222021049_iOS.jpg

Then we were shown the new versions of Visual Studio and Visual Studio for Mac

20180507_223246026_iOS.jpg

20180507_223343639_iOS.jpg

One feature that is currently experimental is the ability to navigate to sources of a NuGet Package, or to decompile an existing DLL (like Jetbrains already does)

20180507_224208579_iOS.jpg

To end this session, the Scotts showed us the new features for Web Development, like Blazor (and mono.js/web assembly which can run C# DLL right into the browser)20180507_224208579_iOS.jpg

20180507_230122619_iOS.jpg

//Build 2018 – Microsoft AI overview for developers

Microsoft AI overview for developers
Dr. Harry Shum, Brian Trager

20180507_200538708_iOS.jpg

This session was meant as a tour of all the AI services available in Azure today.

Announcements :

  • QnAMaker General Availability
  • Bot Services 3.0 (100+ new features in Bot Framework – new Bot Framework Emulator)
  • Luis / QnAMaker command line tools
  • az-cli includes Bot Services
  • Machine Learning for .NET (ML.NET)

To start, Harry reminded that Cognitive Services was launched 3 years ago at //Build 2015 and has already achieved 1+ million developers using it.

After that, we were shown videos and live demos that showcased the new services in Cognitive Services.

One of those was live translation with the Microsoft Translator app on smartphone. Brian Trager, which is deaf, talked in english with Harry Shum who responded in chinese.
Microsoft Translator uses a trained AI with Custom Speech, Custom Voice and Custom Translation for a near real-time and totally accurate translation between the 2 speakers (better and quicker than the live transcript used by Microsoft in all sessions at //Build).

20180507_202724009_iOS.jpg

Continuing the tour, we were shown several linked demos using Conversational AI with the Bot Services and lots of other Cognitive Services.

First was a chatbot on an e-commerce website.
The bot used Text Analytics to adapt to the user’s language on the fly.
It used Luis.ai to recognize intents like « I want to buy a new watch » and react accordingly (refreshing the displayed items on the website).
The bot then purposed to upload an image of a watch to analyze it with Custom Vision to find a similar model in the website.
QnAMaker (Generally Available as of today) was also used to answer questions. The new QnAMaker allows to find correct answers for implicit context based on the previous questions (through the use of metadata).
For example, « What is the return policy? » – « You can get a full refund up to 7 days after the purchase » – « What about after that? » – « You can get store credits for the next 30 days ».
This was not possible before.
To end with this demo, the bot was also capable of booking a visit to the nearest retail store by taking into account store hours, the user’s calendar, the road traffic, etc. And setting the visit into the user’s calendar.
The bot finally asked for a selfie of the user.

The second demo was another bot this time in a kiosk inside a mall.
The same user interacts with it and the bot recognizes the person using the previously taken selfie (using Face Recognition API)
The bot was using Text-To-Speech and Speech-To-Text to communicate with the user and was able to know that the user had a meeting inside one of the store in the mall, and displayed a personalized map to let the user know which way is the store.

The third and last demo was a website where the store clerk can view all previously aggregated info about the customer using Microsoft Dynamics.

Moving on the new features of the Bot Framework, Harry showcased the ability to load chat transcript directly into the emulator to avoid retyping everything to test a dialog.

The new Bot Framework Emulator is also capable of managing Luis/QnAMaker (through the use of the new command line tools) for a quicker develop-configure-test cycle.

Then we moved onto Machine Learning and the ONNX format (open source) created by Microsoft but now supported by 15 big companies.

A new toolkit to write Machine Learning in C# used by Microsoft is made available to all : ML.NET

20180507_204004362_iOS.jpg

To end this session, we were shown the integration of all the tooling into Visual Studio.
Like creating a whole project by just right clicking the Custom Vision resource in the Server Explorer tab of Visual Studio.

Useful links:

//Build 2017 – EF Core 2.0 / UWP streamable install / WinForms-WPF

Entity Framework Core 2.0: data, mobile, server, IoT, and more
Rowan Miller

This session presented the new features brought by EF Core 2.0, which is out as Preview 1 right now.

Like .NET Core, EF Core is built against .NET Standard 2.0 which means that it can be used on any platform: Windows, Xamarin, HoloLens, IoT, Linux, etc.

Here’s the summary of the new features demonstrated

  • Global Query Filters
  • Flexible Mapping
  • Context Pooling
  • Compiled Queries
  • Some currently in development features were also shown like complex type mapping (which already exists in EF6) and better migration with « SeedData ».

    Global Query Filters allows to define filters at the DbContext level.
    For example in soft delete scenario, instead of excluding « IsDeleted = True » in every queries, we can define in the configuration of the DbContext to exclude it globally.
    This is done with the following code:

    modelBuilder.Entity<Blog>().HasQueryFilter(p => !p.IsDeleted)

    The interesting part of that line of code is that the QueryFilter part is evaluated at every query.
    This allows to do things like this.

    public class MyDbContext : DbContext
    {
        public string TenantId {get; set; }
    
        public void OnConfigure(...)
        {
            modelBuilder.Entity<Blog>().HasQueryFilter(b => b.TenantId == TenantId);
        }
    }
    

    Flexible Mapping allows more possibilities when mapping a C# object to a table.
    One possibility is mapping a private field to a column.

    modelBuilder.Entity<Blog>.Property<string>("TenantId").HasField("_tenantId")
    

    When we want to retrieve the value of the column, we can write that line

    var value = EF.Property<string>(row, "TenantId")
    

    Context Pooling allows the use a pool of already instantiated DbContext (like a ThreadPool) when working with DI.
    Enabling it is pretty straightforward, just replace « .AddDbContext() » by « .AddDbContextPool() ».
    It has some restrictions because the DbContext needs to be cleaned before each use.

    Compiled Queries are now available in EF Core.

    var query = EF.CompiledQuery((DbContext dbContext, int id) => dbContext.Table.First(a => a.Id == id))
    var result = query(dbContext, id);
    

    During the session, the speaker demonstrated the « EF Everywhere » by running a UWP application with EF Core on a Raspberry Pi named « Unicorn Launcher » that launched a small rocket into the air.

    https://docs.efproject.net
    https://github.com/aspnet/entityframework
    https://github.com/rowanmiller/demo-efcore

    XAML custom controls for UWP: Start to finish
    John Bristowe, Nikola Metulev

    This was a disappointing session.
    I tried my chances with this session based on its description which promised to explain the best pratices and UX design patterns for accessibility and input methods (mouse, keyboard, game controller, HoloLens) when creating custom controls in XAML.

    But in fact, it was a plain simple session on how to use and create controls in XAML (that is the same thing since the introduction of WPF 10 years ago).

    This session was more of an advertisement for UWP Community Toolkit and UI for Telerik.

    Here’s what we saw

    • 1st party controls: Button, TextBox, etc.
    • 3rd party controls: UWP community toolkit, Telerik, etc.
    • Restyled controls (Style, Template)
    • Extended controls (Inheriting controls to add behaviors)
    • Custom controls

    Nextgen UWP app distribution: Building extensible, stream-able, componentized apps
    John Vintzel, Sandeep George

    This session talked about some new features for UWP concerning the distribution of the apps.

    First, we were shown the « App streaming installation » feature that allows to download only a « required » part of the application and run it even before its full download. This can be compared to how Xbox games can be partially downloaded and run.

    This is available right now in the Creators Update.

    All the magic lies in a manifest file named « AppxContentGroupMap.xml » that is included in the UWP app. The Windows Store then uses this file to know how to install it.
    This feature comes with all the needed API to finely manage how to application reacts to missing parts, prioritizes parts to download and gets feedback on the progression of the download.

    APIs are also available to debug the app to simulate a state of partial download.

    For more information, see https://aka.ms/uwpstreaminginstall

    Coming in the Fall Creators Update, optional packages for UWP will be available.
    We can see it as DLC or plugins for existing UWP apps.
    It extends an existing app (or many) by adding contents and/or features and it can be sold separately.
    A UWP app that allows for optional packages can also receive packages from other publishers.

    For more information, see https://aka.ms/uwpoptionalpackages

    Modernize WinForms and WPF apps with maximum code reuse, cross-platform reach, and efficient DevOps
    Joshua Weber, Mike Battista

    This session was also non-surprising.
    It was aimed at those stuck with WinForms/WPF and who didn’t followed the other technologies in the meantime.

    To start, we were explained how to leverage the Desktop Bridge to take advantage of all the new APIs of Windows 10.
    Then, by moving the codebase of an existing app to a .NET Standard 2.0 lib, the code was reused into a Xamarin app.
    To end this session, we were presented Visual Studio Mobile Center to setup CI/CD for the Xamarin app.