Logging and improving bots over time

It can't be stressed enough that building great chat and voice interactions requires a lot of testing and iteration. To be able to do this efficiently, logging is central.

When you start testing your app with users, one of the key elements you typically want to work with is to monitor when fallbacks are triggered and potentially add examples to your intents and entities based on this.

Viewing logs of interaction

A built-in log-viewer and integrations with popular bot analytics platforms is currently in the making and will be announced shortly!

Receiving logs through webhooks

Narratory can trigger notifications or save logs in your database using a webhook. You set this on your Agent object using the logWebhook parameter as follows.

const agent: Agent = {  agentName: "Your agent name",  language: Language.English,  // ... And so on  logWebhook: "https://url-to-your-fallback-notification-service",  logLevel: "ALL"}

The logLevel optional parameter can be set to "NONE", "FALLBACKS" (default) or "ALL" depending on if you want to receive calls for all turns or just fallback turns.

The URL given as logWebhook will receive a POST request with JSON data with the following parameters:

Note: if your endpoint is built in Typescript, you can use the Narratory interfaces LogMessage and LogTurn to type the incoming messages by importing them with import { LogMessage, LogTurn } from "narratory"

Parameter	Type	Description
sessionId	string	SessionID set by client
agentName	string	The Agent's name
platform	string	See the system variables section of docs at https://narratory.io/docs/logic#system-variables
turn	LogTurn (see below)	A JSON object with parameters as defined below
lastTurn	LogTurn (see below)	Same format as above. **Only set if the turn is a fallback, i.e the isFallback parameter is true
text	string	A humanly readable string suitable for example for a Slack Notification

The turn (and lastTurn) object has the following parameters:

Parameter	Type	Description
id	string	The UserTurn's ID (automatically created on Agent build if not manually set)
userInput	string	The user's input that triggered this turn
intentName	string	The classified Intent's name
parameters	object	An object with the classified entity parameters, for example { toCity: "London", fromCity: "Paris" }
confidence	number, between 0.0 and 1.0	Intent classification confidence
isFallback	boolean	True if the input triggered a fallback. If true, the parent LogMessage will also include a lastTurn parameter for convenience
botReplies	string[] (an array of strings)	The text replies of the bot upon receiving the above user inputs
endOfConversation	boolean	True if the turn is the last in a conversation, either with no subsequent turn or manually exited by user. Note, Narratory will not know if a user hung up using a telephony integration.
timestamp	number	Unix epoc timestamp

A sample response for a non-fallback log message (from the Narratory Flightbooker example app):

{  "sessionId": "add64e7e-ff0b-4fd8-bcaf-97b2d072e9ee",  "turn": {    "id": "d2189d9a-5178-4d88-ab7e-b0513c603e8e",    "agentName": "Narratory Flightbooker",    "userInput": "I wanna fly to paris",    "intentName": "travel",    "parameters": {      "fromCity_original": "",      "fromCity": "",      "tickets_original": "",      "tickets": "",      "toCity_original": "paris",      "toCity": "Paris",    },    "isEndOfConversation": false,    "isFallback": false,    "confidence": 1,    "botReplies": ["Excellent", "How many people are travelling?"],    "timestamp": 1585598421352  },  "text": "User: I wanna fly to paris [travel - Conf: 1]\nBot: Excellent,How many people are travelling?",  "platform": "google"}