Setup

The Setup file is a simple JSON or JSON5 file that describes the configuration of the scene and optionally the different states of the scene. It is required to use the rooomAssistant service.

INFO

JSON (JavaScript Object Notation) is a lightweight data interchange format commonly used for exchanging data between web services and applications. It is easy to read and write for humans and easy to parse and generate for machines. Another option is to use the JSON5 format, is an extension to the popular JSON file format that aims to be easier to write and maintain by hand. The Documentation uses JSON5 as the default format.

Each JSON file starts with a config section that defines various settings for the scene, such as whether to use a text-to-speech engine (e.g. Google, Web API), the URL of the 3D mesh file, camera and light setup, and the position and attachment style of the speech bubbles and GUI elements.

Following the config section, the file contains multiple state sections that describe the different states of the scene. Each state includes information about the speech bubble, which has subsections for text and buttons, as well as any actions that should be triggered when the state is entered.

The text section of the speech bubble can be multilanguage, allowing for localization of the experience, and each button consists of a title and an action that is triggered when clicked.

Basic structure of a JSON file:

{   /* config section */
	_config: {
		tts: { /* text-to-speech settings */ },
		scene: { /* 3d scene setup */
			camera: { /* camera setup */ },
			light: { /* light setup */ },
		},
		avatar: { /* avatar setup */ },
		speechBubble: { /* speech bubble setup */ },
		states: { /* general states setup */ },
	},

    /* state section */
	state1: {
		speechBubble: { /* speechbubble block */
			text: { /* text block */ },
			buttons: [ /* buttons block */ ],
		},
		actions: [ /* actions */ ],
	},

    /* another state section */
	state2: { /* state block */ },
	...
}

Config

In the _config block, you can define various settings for the scene. Note that its written with an underscore at the beginning of the block name.

Text-to-Speech

tts text-to-speech settings
- provider: text-to-speech engine (e.g. google, webapi, silent)
- apiKey: API key (only required for Google) (e.g. AIzaSyD...)
- language: language (e.g. en-US, de-DE or de, en)
- gender: gender (e.g. male, female)
- voice: voice name (e.g. en-US-Wavenet-A, de-DE-Wavenet-A)
- volume: volume (e.g. 1.0)
- rate: rate (e.g. 1.0)
- pitch: pitch (e.g. 1.0)

Scene

scene 3d scene settings
- debug: enable debug mode (babylon inspector) (e.g. true)
- camera: camera settings
  - target: target of the camera as a vector ( e.g. [0, 0, 0])
  - alpha: rotation of the camera around the y axis (e.g. 1.57)
  - beta: rotation of the camera around the x axis (e.g. 3.14)
  - radius: distance of the camera from the target (e.g. 3.0)
  - fov field of view of the camera in radians
- light: light settings
  - diffuse: diffuse color of the light as a color vector or hex string (e.g. [255, 0 , 255], #ff00ff)
  - intensity: intensity of the light (e.g. 1.0)
  - enabled: enable/disable the light (e.g. true)

Avatar

For now only one avatar/mesh is supported. In the future, multiple avatars or meshes will be supported.

avatar: avatar settings
- mesh: url of the avatar mesh file
- transform: transform of the mesh relative to the scene origin
  - position: position of the avatar as a vector (e.g. [0, 0, 0])
  - rotation: rotation of the avatar as a vector (e.g. [0, 0, 0])
  - scaling: scale of the avatar as a vector e.g. (e.g. [1, 1, 1])

Speech Bubble

The Speech Bubble is the main element of the rooomAssistant service. It is used to display text and buttons to the user. The speech bubble can be attached to the avatar or to any other object in the scene. You have to specify the parent of the speech bubble and the offset of the speech bubble relative to the parent.

speechBubble: speech bubble settings
- parent: parent of the speech bubble as node name (e.g. Hips)
- offset: offset of the speech bubble relative to the parent as a vector (e.g. [0, 0, 0])
- colors: colors of the speech bubble as a rgb or hex string (e.g. [255, 0 , 255], #ff00ff)

States

This sections defines the general settings for all states. At the moment, only the start state is supported, where you can define the initial state of the scene.

Sample

_config: {
    tts: {
        provider: 'google', // use google as text-to-speech engine
        apiKey: 'YourGoogleApiKey', // your google API key
        languageCode: 'en-GB', // language code
        gender: 'male', // gender of the voice
        voice: 'en-GB-Wavenet-B', // voice name
    },
    scene: {
        debug: false, // disable debug mode, i.e. babylon inspector
        camera: {
            target: [-0.4, 1, 0], // target of the camera, a little bit to the left of the avatar
            alpha: 1.571, // rotation of the camera around the y axis in radians 90°
            beta: 1.571, // rotation of the camera around the x axis in radians 90°
            radius: 2, // distance of the camera from the target
        },
    },
    avatar: {
        mesh: 'http://www.company.com/sms.glb', // url of the avatar mesh file
        transform: {
            position: [0, 0, 0], // position of the avatar
            rotation: [0, 0, 0], // rotation of the avatar
            scaling: [1.2, 1.2, 1.2], // scale of the avatar
        }
    },
    speechBubble: {
        parent: 'Hips', // parent of the speech bubble, i.e. the hips of the avatar
        offset: [-0.7, 0.8, 0], // offset of the speech bubble relative to the parent
        colors: {
            buttonColor: '#111', // color of the button text
            buttonBackground: '#ffe600', // color of the button background
        },
        lang: 'en', // default language of the speech bubble
    },
    states: {
        startState: 'start', // initial state of the scene
    },
}

States

After the config section, the file contains multiple state sections that describe the different states of the scene.

Each state includes information about the speechBubble, which has subsections for text and buttons.

And actions that should be triggered when the state is entered.

Speech Bubble

speechBubble: speech bubble section
- text: text as a string or multilanguage object (e.g. "Hello World", { "en": "Hello World", "de": "Hallo Welt" })
- buttons: an array of buttons (in JSON an array is defined by square brackets [...])
  - title: title of the button as a string or multilanguage object (e.g. "Click me", { "en": "Click me", "de": "Klick mich" })
  - action: an array of actions that is triggered when the button is clicked, each action has the following properties:
    - name: name of the action (e.g. animations.play)
    - args: arguments of the action (e.g. idle)

Actions

Another block that can be defined in a state is the actions block. It contains an array of actions that are triggered when the state is entered. It has the same structure as the buttons block in the speechBubble section.

actions: an array of actions that is triggered when the state is entered, each action has the following properties:
- name: name of the action (e.g. animations.play)
- args: arguments of the action (e.g. idle)

INFO

The actions gives you the possibility to modify the scene when the state is entered or a button is clicked. For example, you can play an animation, change the color/texture of an object. You can hide or show objects, change camera position and modify the text and buttons of the speech bubble.

You can find a list of all available actions in the API section.

Sample

stateOne: { // name of the state
	speechBubble: {
		text: {
			en: 'Hello World', // text in english
			de: 'Hallo Welt', // text in german
			// ... more languages if needed
		},
		buttons: [ // array of buttons
			{
				text: { en: 'Open Help', de: 'Hilfe öffnen' }, // text of the button in english and german
				actions: [ // actions that are triggered when the button is clicked
					{ name: 'utils.postMessage', arg: 'help' } // send a message to the parent window
				],
			},
			{
				text: { en: 'Close', de: 'Schließen' }, // text of the button in english and german
				actions: [{ name: 'states.set', arg: 'stateTwo' }], // jump to stateTwo when the button is clicked
			},
		],
	},
	actions: [
		{ name: 'animations.play', arg: 'Welcome' }, // play the welcome animation when the state is entered
		{ name: 'animations.play', args: ['Idle', 3000] } // after 3 seconds play the idle animation
	],
},
stateTwo: { // name of the state: stateTwo
	speechBubble: { // speech bubble section
		text: {
			en: 'You made it. Well done!', // text in english
			de: 'Du hast es geschafft. Gut gemacht!', // text in german
		},
		buttons: [ // array of buttons
 			{
				text: { en: 'Lets dance', de: 'Lass uns tanzen' } // text of the button in english and german
				actions: [{ name: 'animations.play', arg: 'Dance' }], // play the dance animation when the button is clicked
			},
			{
				text: { en: 'Thats enough', de: 'Das reicht' }, // text of the button in english and german
				actions: [{ name: 'animations.play', arg: 'Idle' }], // play the idle animation when the button is clicked
			},
			{
				text: { en: 'Back to start', de: 'Zurück zum Anfang' }, // text of the button in english and german
				background: '#111', // background color of the button
				color: '#fff', // text color of the button
				actions: [{ name: 'states.set', arg: '_back' }], // jump back to the start state
			},

		],
	},
	actions: [
		{ name: 'animations.play', arg: 'Cheer' }, // play the cheer animation when the state is entered
		{ name: 'animations.play', args: ['Idle', 5000] } // after 5 seconds play the idle animation
	],
}

Setup ​

Config ​

Text-to-Speech ​

Scene ​

Avatar ​

Speech Bubble ​

States ​

Sample ​

States ​

Speech Bubble ​

Actions ​

Sample ​

Setup

Config

Text-to-Speech

Scene

Avatar

Speech Bubble

States

Sample

States

Speech Bubble

Actions

Sample