Logging and training bots

It can't be stressed enough that building great chat and voice interactions requires a lot of testing and iteration. To be able to do this efficiently, logging interactions is central.

When you start testing your app with users, one of the key elements you typically want to work with is to monitor when fallbacks are triggered and potentially add examples to your intents and entities based on this.

Viewing logs of interactions

You can view logs of all interactions with your bot in the Logs pane of the Builder.

Training bot by annotating logs

You will soon be able to train your bot based on the sessions it has received by annotating logs with intents and entities.

External logging

In addition to the internal logging capabilities of the Narratory builder, you can also log externally (in real-time) by providing an URL to an API-endpoint of yours. This allows you to, for example, save interactions to a database or push logs to an external analytics tool.

You can add a log webhook in the Settings pane of the Builder. Here, you also set the log level to either Fallbacks, All, or None. This will determine when we call the webhook. You also can decide if you want to send logs to your webhook also for test sessions in the Builder, or if you only want to get logs for your deployed bot.

Webhook format

The URL given as logWebhook will receive a POST request in real-time with JSON data with the following parameters:

Note: if your endpoint is built in Typescript, you can use the Narratory interfaces LogMessage and LogTurn to type the incoming messages by importing them with import { LogMessage, LogTurn } from "narratory-lib" after first running npm install narratory-lib on your backend project.

Parameter	Type	Description
sessionId	string	SessionID set by client
agentName	string	The Agent's name
platform	string	See the system variables section of docs at https://narratory.io/docs/logic#system-variables
turn	LogTurn (see below)	A JSON object with parameters as defined below
lastTurn	LogTurn (see below)	Same format as above. **Only set if the turn is a fallback, i.e the isFallback parameter is true
text	string	A humanly readable string suitable for example for a Slack Notification

The turn (and lastTurn) object has the following parameters:

Parameter	Type	Description
id	string	The UserTurn's ID (automatically created on Agent build if not manually set)
userInput	string	The user's input that triggered this turn
intentName	string	The classified Intent's name
parameters	object	An object with the classified entity parameters, for example { toCity: "London", fromCity: "Paris" }
confidence	number, between 0.0 and 1.0	Intent classification confidence
isFallback	boolean	True if the input triggered a fallback. If true, the parent LogMessage will also include a lastTurn parameter for convenience
botReplies	string[] (an array of strings)	The text replies of the bot upon receiving the above user inputs
endOfConversation	boolean	True if the turn is the last in a conversation, either with no subsequent turn or manually exited by user. Note, Narratory will not know if a user hung up using a telephony integration.
timestamp	number	Unix epoc timestamp

A sample response for a non-fallback log message (from the Narratory Flightbooker example app):

{  "sessionId": "add64e7e-ff0b-4fd8-bcaf-97b2d072e9ee",  "turn": {    "id": "d2189d9a-5178-4d88-ab7e-b0513c603e8e",    "agentName": "Narratory Flightbooker",    "userInput": "I wanna fly to paris",    "intentName": "travel",    "parameters": {      "fromCity_original": "",      "fromCity": "",      "tickets_original": "",      "tickets": "",      "toCity_original": "paris",      "toCity": "Paris",    },    "isEndOfConversation": false,    "isFallback": false,    "confidence": 1,    "botReplies": ["Excellent", "How many people are travelling?"],    "timestamp": 1585598421352  },  "text": "User: I wanna fly to paris [travel - Conf: 1]\nBot: Excellent,How many people are travelling?",  "platform": "google"}