Logging and improving bots over time

It can't be stressed enough that building great chat and voice interactions requires a lot of testing and iteration. To be able to do this efficiently, logging is central.

When you start testing your app with users, one of the key elements you typically want to work with is to monitor when fallbacks are triggered and potentially add examples to your intents and entities based on this.

Viewing logs of interaction

A built-in log-viewer and integrations with popular bot analytics platforms is currently in the making and will be announced shortly!

Receiving logs through webhooks

Narratory can trigger notifications or save logs in your database using a webhook. You set this on your Agent object using the logWebhook parameter as follows.

const agent: Agent = {
agentName: "Your agent name",
language: Language.English,
// ... And so on
logWebhook: "https://url-to-your-fallback-notification-service",
logLevel: "ALL"
}

The logLevel optional parameter can be set to "NONE", "FALLBACKS" (default) or "ALL" depending on if you want to receive calls for all turns or just fallback turns.

The URL given as logWebhook will receive a POST request with JSON data with the following parameters:

Note: if your endpoint is built in Typescript, you can use the Narratory interfaces LogMessage and LogTurn to type the incoming messages by importing them with import { LogMessage, LogTurn } from "narratory"

ParameterTypeDescription
sessionIdstringSessionID set by client
agentNamestringThe Agent's name
platformstringSee the system variables section of docs at https://narratory.io/docs/logic#system-variables
turnLogTurn (see below)A JSON object with parameters as defined below
lastTurnLogTurn (see below)Same format as above. **Only set if the turn is a fallback, i.e the isFallback parameter is true
textstringA humanly readable string suitable for example for a Slack Notification

The turn (and lastTurn) object has the following parameters:

ParameterTypeDescription
idstringThe UserTurn's ID (automatically created on Agent build if not manually set)
userInputstringThe user's input that triggered this turn
intentNamestringThe classified Intent's name
parametersobjectAn object with the classified entity parameters, for example { toCity: "London", fromCity: "Paris" }
confidencenumber, between 0.0 and 1.0Intent classification confidence
isFallbackbooleanTrue if the input triggered a fallback. If true, the parent LogMessage will also include a lastTurn parameter for convenience
botRepliesstring[] (an array of strings)The text replies of the bot upon receiving the above user inputs
endOfConversationbooleanTrue if the turn is the last in a conversation, either with no subsequent turn or manually exited by user. Note, Narratory will not know if a user hung up using a telephony integration.
timestampnumberUnix epoc timestamp

A sample response for a non-fallback log message (from the Narratory Flightbooker example app):

{
"sessionId": "add64e7e-ff0b-4fd8-bcaf-97b2d072e9ee",
"turn": {
"id": "d2189d9a-5178-4d88-ab7e-b0513c603e8e",
"agentName": "Narratory Flightbooker",
"userInput": "I wanna fly to paris",
"intentName": "travel",
"parameters": {
"fromCity_original": "",
"fromCity": "",
"tickets_original": "",
"tickets": "",
"toCity_original": "paris",
"toCity": "Paris",
},
"isEndOfConversation": false,
"isFallback": false,
"confidence": 1,
"botReplies": ["Excellent", "How many people are travelling?"],
"timestamp": 1585598421352
},
"text": "User: I wanna fly to paris [travel - Conf: 1]\nBot: Excellent,How many people are travelling?",
"platform": "google"
}