Skip to main content

Language, expressiveness & rich content

Since chat and voice-apps by their nature are more personal than most graphical applications through mimicing human speech, working on expressiveness of your apps is very important. Unlike graphical apps where users read text, in spoken text you have to care about pronunciation (which, despite the improvements of text-to-speech the last years still sometimes leaves a lot to wish for), tempo, pause, how to handle errors (see error handling), variation (add it from the start!).

Since Narratory is based on many years of dialog research, you can rest on the fact that the design-patterns used are optimized for dynamic conversations that are both functional and engaging.

This page walks through different ways of tweaking the expressiveness of your bot and how you can give it the illusion of a personality.

Languages#

A crucial step in the design-process, and the easiest way to personalize your app is selecting what your agents language should be (talk in and listen for). Currently, Narratory agents speak one language, but multi-lingual support will come in the future. The language selection is done when you create your agent (see Basic building blocks).

The full list of languages are:

  • ChineseCantonese
  • ChineseSimplified
  • ChineseTraditional
  • Danish
  • Dutch
  • English
  • EnglishAustralia
  • EnglishCanada
  • EnglishGreatBritain,EnglishIndia
  • EnglishUS
  • French
  • FrenchCanada
  • FrenchFrance
  • German
  • Hindi
  • Indonesian
  • Italian
  • Japanese
  • Korean
  • Norwegian
  • Polish
  • PortugueseBrazil
  • PortuguesePortugal
  • Russian
  • Spanish
  • SpanishLatinAmerica
  • SpanishSpain
  • Swedish
  • Thai
  • Turkish
  • Ukranian

When choosing language, it is important to be mindful that not all built-in entities (see built-in entities in NLU docs) are available in all languages. See Dialogflow's list of System entities to learn what entities are available in your selected language.

Gender and voice selection#

Not a Narratory specific option, but your selection of gender an voice is an important one when it comes to designing a persona that fits your brand and/or communication. Each platform has different voices that you configure separately. For Google's ecosystem you can configure it straight in Dialogflow.

Adding expressiveness with RichSays#

Instead of using normal text you can use the RichSay interface to add content or modify pronunciation with SSML.

Working on pronunciation with SSML#

For voice-apps, pronunciation is naturally key in making a good impressions with your users. Usually the text-to-speech does a good job, but sometimes it is necessary to tweak how certain words are pronounced using SSML, which stands for Speech Synthesis Markup Language (SSML). Using this, you can make your agent sound more natural by altering how words are said.

For example, if you want to pronounce "1" as "first" instead of "one" you can use the SSML

  I like the <say-as interpret-as="ordinal">1</say-as> <!-- "I like the first" -->

To use SSML in your Narratory agent you use the ssml parameter in RichSay. Remember to escape the quotes if you use the same type of quotes inside the SSML-string. Two examples follow to visualize:

const whoWonQuestion: UserTurn = {  // Here, we need to escape the double quotes since we use double quotes also for denoting the followup string  intent: ["who won the game?"],  bot: {    text: "I came in as number one",    ssml: 'I came <say-as interpret-as="ordinal">1</say-as>'  }}
const spellBarQuestion: UserTurn = {  // Here, we don't need to escape the double quotes since we use single quotes to denote the followup string  intent: ["how do you spell out bar"],  bot: {    say: "B A R",    ssml: '<speak><say-as interpret-as="characters">bar</say-as></speak>'  }}

Each platform has their own voice synthesis engines, which unfortunately means they support different subsets of the SSML standard. For this purpose, it is currently recommended to review each platforms own SSML reference material, for example Google's SSML reference and Amazon's SSML reference

Adding graphical content#

For clients with a GUI, for example Google Assistant on mobile phones, chat in web widgets or on homepages, adding Graphical content is usually adding to the user-experience - both by enriching the conversation but also by making it easier for users to navigate the conversation. Rich content will be ignored on devices that don't support them.

To add graphical content, you use the content parameter of the RichSay interface and create an instance of a content class (below illustrated with an instance of a Card class with a button included) as follows:

const shareInfo: BotTurn = {  say: {    text: "Here is some information",    content: new Card({      title: "You can find more information here",      description: "There is a lot more information here, \n spanning several lines \n even",      buttons: [        new Button("Press this button for even more info", "https://more-info.com")      ]    })  }}

The supported Rich content types are (here explained with their typescript class definitions):

export class Card extends Content {  title: string  description?: string  subtitle?: string  image?: Image  buttons?: Button[]}
export class Image extends Content {  url: string  alt: string}
export class Button extends Content {  text: string  url: string}
export class List extends Content {  items: Item[]  title?: string  image?: Image}