Logging and training bots

It can't be stressed enough that building great chat and voice interactions requires a lot of testing and iteration. To be able to do this efficiently, logging interactions is central.

When you start testing your app with users, one of the key elements you typically want to work with is to monitor when fallbacks are triggered and potentially add examples to your intents and entities based on this.

Viewing logs of interactions

You can view logs of all interactions with your bot in the Logs pane of the Builder.

Builder intro

Training bot by annotating logs

You will soon be able to train your bot based on the sessions it has received by annotating logs with intents and entities.

External logging

In addition to the internal logging capabilities of the Narratory builder, you can also log externally (in real-time) by providing an URL to an API-endpoint of yours. This allows you to, for example, save interactions to a database or push logs to an external analytics tool.

You can add a log webhook in the Settings pane of the Builder. Here, you also set the log level to either Fallbacks, All, or None. This will determine when we call the webhook. You also can decide if you want to send logs to your webhook also for test sessions in the Builder, or if you only want to get logs for your deployed bot.

Builder intro

Webhook format

The URL given as logWebhook will receive a POST request in real-time with JSON data with the following parameters:

Note: if your endpoint is built in Typescript, you can use the Narratory interfaces LogMessage and LogTurn to type the incoming messages by importing them with import { LogMessage, LogTurn } from "narratory-lib" after first running npm install narratory-lib on your backend project.

ParameterTypeDescription
sessionIdstringSessionID set by client
agentNamestringThe Agent's name
platformstringSee the system variables section of docs at https://narratory.io/docs/logic#system-variables
turnLogTurn (see below)A JSON object with parameters as defined below
lastTurnLogTurn (see below)Same format as above. **Only set if the turn is a fallback, i.e the isFallback parameter is true
textstringA humanly readable string suitable for example for a Slack Notification

The turn (and lastTurn) object has the following parameters:

ParameterTypeDescription
idstringThe UserTurn's ID (automatically created on Agent build if not manually set)
userInputstringThe user's input that triggered this turn
intentNamestringThe classified Intent's name
parametersobjectAn object with the classified entity parameters, for example { toCity: "London", fromCity: "Paris" }
confidencenumber, between 0.0 and 1.0Intent classification confidence
isFallbackbooleanTrue if the input triggered a fallback. If true, the parent LogMessage will also include a lastTurn parameter for convenience
botRepliesstring[] (an array of strings)The text replies of the bot upon receiving the above user inputs
endOfConversationbooleanTrue if the turn is the last in a conversation, either with no subsequent turn or manually exited by user. Note, Narratory will not know if a user hung up using a telephony integration.
timestampnumberUnix epoc timestamp

A sample response for a non-fallback log message (from the Narratory Flightbooker example app):

{
"sessionId": "add64e7e-ff0b-4fd8-bcaf-97b2d072e9ee",
"turn": {
"id": "d2189d9a-5178-4d88-ab7e-b0513c603e8e",
"agentName": "Narratory Flightbooker",
"userInput": "I wanna fly to paris",
"intentName": "travel",
"parameters": {
"fromCity_original": "",
"fromCity": "",
"tickets_original": "",
"tickets": "",
"toCity_original": "paris",
"toCity": "Paris",
},
"isEndOfConversation": false,
"isFallback": false,
"confidence": 1,
"botReplies": ["Excellent", "How many people are travelling?"],
"timestamp": 1585598421352
},
"text": "User: I wanna fly to paris [travel - Conf: 1]\nBot: Excellent,How many people are travelling?",
"platform": "google"
}