Skip to main content

ECMAScript 6 support in Tovie Platform

Beta

Before fall 2022, Tovie Platform supported only one JavaScript (JS) dialect, namely its implementation according to the ECMAScript 5 (ES5) specification on the Nashorn engine. It was proposed in 2009 and has become significantly outdated, so the features for writing JS code in Tovie Platform do not fully reflect modern professional standards.

  • In Tovie Platform, developers cannot use or can only partially use popular features of modern JS versions. In turn, new users with development experience cannot fully apply their skills.

  • The JS runtime in Tovie Platform is isolated from the ecosystem of packages formed by the JS developer community and it does not allow external dependencies to be imported into projects, other than those built into the platform: Underscore.js and Moment.js.

Now, a new JS runtime environment is available in Tovie Platform that solves both of these problems and enables the development of projects in a modern JS dialect (ES6) with external dependencies support.

tip
Formally, ES6 is a shortened name for the ECMAScript 2015 standard, however Tovie Platform also supports features of later standards. For simplicity, in this article they are combined under the name ES6.

scriptEs6 tag​

Tovie DSL has the scriptEs6 reaction tag. The tag works similarly to script with the difference that when the bot enters the state, the code inside the tag is run in the new runtime environment. This allows you to use the ES6 features in it, which are not available inside a regular script tag:

state: HeadsOrTails
    q!: * {heads * tails} *
    scriptEs6:
        const bit = $reactions.random(2); // Declate a variable via const
        $reactions.answer(`Result: ${bit ? "heads" : "tails"}.`); // Form a string using a template

The Node.js platform version 20 is used to run the code. This means that inside it, features up to ECMAScript 2023 are available, including code splitting into modules, new data structures (Set, Map), classes, asynchronous functions, and much more.

tip
You can learn more about the features of ES6 compared to ES5, for example, in the book by Dr. Axel Rauschmayer Exploring ES6.

There are other Tovie DSL tags inside which JS code is written: if, elseif, init. For now, these tags have no equivalents where ES6 can be used.

Import dependencies​

As before, it is recommended not to describe complex business logic directly inside the scriptEs6 tag, but to put it in separate JS files, which are then imported into the script using the require tag. However, there are important differences in how named objects (variables, functions, classes) from files written in ES5 and ES6 are used.

ES5 dependencies​

If the dependency code is written in ES5 and runs in the old environment, the JS file must be imported into the script using the require tag without parameters.

In addition, if variables or functions are declared at the top level of this file, they are automatically placed in the global scope. They can be accessed from any script or scriptEs6 tag, as well as from any other imported JS files.

require: dialog.js
require: defaults.js

theme: /

    state: Hello
        q!: * hello *
        script:
            sayHello();

ES6 dependencies​

In the new runtime environment, each JS file is a separate module. By default, all named objects are only accessible from within the module. To make them externally visible, export them using the export keyword.

In ES6, there are two ways to export something from a module: named exports and default exports.

  • To access ES6 dependency objects from scriptEs6 tags, declare a default export in the dependency file via the export default directive. Next, import the file in the script via require with additional parameters.

  • To access such objects from other ES6 dependencies, you can export them in either of the two ways, and import them using the import keyword. If these exports are not needed in the script itself, then there is no need to include the file using the require tag.

tip
Learn more about modules, exports, and imports in MDN Web Docs
require: dialog.js
    type = scriptEs6
    name = dialog

# There is no need to import defaults.js as it is not used in scriptEs6

theme: /

    state: Hello
        q!: * hello *
        scriptEs6:
            dialog.sayHello();

Require tag parameters​

To import an ES6 JS file into your script, use the require tag with the following parameters:

  • type must always have the scriptEs6 value.
  • name is any string that can be used as the name of a JS variable.

The name parameter specifies the name that will be given to the value exported from the module. It can be accessed from a script under this name.

tip
Explicitly qualifying each module with a corresponding name helps avoid cluttering the global namespace.

Global variables​

For backward compatibility with the old runtime environment, it is still possible to access values declared inside the module as global variables, without having to specify the namespace each time.

If you want to use global variables, declare them as properties of the global system object in the necessary module, then import the module into the script. In this case, declaring the default export inside the module and specifying the name parameter in the require tag are still required.

caution
The use of global variables is not recommended: this approach clutters the global namespace and leads to implicit dependencies between modules.

In the example below, a $ object which is an alias of the $context object is declared using Proxy, which allows its properties to be accessed in a more concise way:

require: globals.js
    type = scriptEs6
    name = globals

theme: /

    state: Back || noContext = true
        q: back
        scriptEs6:
            $reactions.transition($.contextPath); // Same as $context.contextPath

Runtime configuration in chatbot.yaml​

The new runtime does not require any configuration: if a script has a scriptEs6 tag or has a dependency of that type imported, then that code will automatically run as ES6.

However, for testing purposes, you can partially customize the behavior of the new runtime. To do this, use the scriptRuntime section in the chatbot.yaml configuration file.

scriptRuntime:
  requirementsFile: package.json
  npmRcFile: .npmrc
  forceEs6: true

  • requirementsFile is the path to a JSON manifest declaring external dependencies, which will be used in the script, relative to the src directory. The default value is package.json.

  • npmRcFile is the path to a npm configuration file, relative to the src directory. The default value is .npmrc.

  • forceEs6: if true is specified, then the code of all reaction tags (not only script, but also, for example, the a tag) will run in the new environment. The default value is false.

    caution
    Running all tags in the new runtime is not currently optimized. It is not recommended to enable the forceEs6 parameter if it is important to preserve the bot’s response speed.

Import npm packages​

The key advantage of the new runtime over the old one is that you can use not only the built-in JS API to extend the bot’s capabilities, but also external dependencies: npm packages written by third-party developers.

How to import and use packages​

  1. Create a package.json file in the src directory. If you have overridden requirementsFile, create the file in the path that you specified yourself.

  2. Specify the dependencies property in this file. Its value must be an object where keys are the names of the required packages and values are their versions.

    {
      "dependencies": {
        "luxon": "^3.0.3"
      }
    }
    tip
    If you are developing a project locally, it is recommended to install dependencies via npm or any other package manager, for example yarn. Package managers can determine the optimal dependency versions and automatically fill the package.json file. Other properties can be set in the file, but Tovie Platform will only use dependencies.

  3. Use the package in the same way as internal dependencies: import the objects via import and access them in your code.

    require: time.js
        type = scriptEs6
        name = time

    theme: /

        state: WhatTimeIsIt
            q!: * what time is it *
            scriptEs6:
                $reactions.answer(time.now());

Package restrictions​

If external dependencies are used in a script, Tovie Platform installs them by running the npm install command before deploying the bot to a channel. While the bot is running, Tovie Platform cannot execute any commands (for example, those declared in the scripts section of package.json). Because of this:

  • In Tovie Platform, packages that require running commands like npm start will not work correctly. In particular, this includes Express and any other packages that start local servers.
  • You can’t write code for the new runtime environment in TypeScript, because it requires a separate step of source code compilation via tsc.

We also recommend being cautious with packages that generate artifacts needed for their operation. An example of such a package is Prisma: it generates a client for DBMS connections based on the specified data schema. ECMAScript 6 bots can work with the file system, but it is not intended for permanent file storage and can be cleared at any time.

caution
The more packages you use, the slower the bot deployment will be. Include only those packages that are really necessary for the project to work.

npm configuration​

Advanced feature

You can use the .npmrc configuration file for additional control over the installation of npm packages. Place it in the src directory or in the path you specified as the npmRcFile parameter in chatbot.yaml.

For example, you can use not only packages from the public https://registry.npmjs.org/ registry, but also from a private one. To do this, specify the registry address and authorization token in the .npmrc file:

registry=https://nexus.tovie.ai/repository/npm-group/
_authToken=NpmToken.<myToken>

Use built-in JS API​

Tovie Platform provides a built-in JS API: a set of global variables, functions, and services that can be accessed from any point in the script. It can be used to implement commonly used features relevant to a wide variety of projects:

  • The $session and $client variables provide a native storage of the session and client data so you do not need to connect your own database to the bot.
  • The $http service allows you to integrate the bot with almost any external system that can be accessed via HTTP API.
  • The $analytics, $pushgate, and a number of other services allows you to call various Tovie Platform subsystems from a script.
info

There is also the built-in $storage service that is available only in ES6.

Currently, you can use a limited subset of the JS API inside scriptEs6 tags and ES6 dependencies: not all features were ported to the new runtime. The list of unavailable features is provided below, and we will reduce it in the future.

tip
If the feature you need is not directly related to Tovie Platform, use external dependencies to compensate for the missing functionality. For example, you can use the Axios HTTP client instead of $http.

Unavailable features​

Functions:

Services:

  • $http
  • $nlp: the createClassifier, fixKeyboardLayout, match, matchExamples, matchPatterns, and setClass methods.

Asynchronous methods​

In the old runtime environment, all built-in JS API service methods are synchronous. They block the request processing thread and return the result directly. In the new runtime, this behavior was changed for some services without backward compatibility:

Now these methods are asynchronous and return Promise. To work with the returned values, you can use the standard promise interface: the then and catch methods, or you can wait for them to complete using the await keyword.

require: patterns.sc
    module = sys.zb-common

theme: /

    state: PartOfSpeech
        q!: * part of speech * word $AnyWord *
        scriptEs6:
            const markup = await $caila.markup($parseTree._AnyWord);
            $temp.pos = markup?.words?.[0]?.annotations?.pos;
        if: $temp.pos
            a: The POS tag of {{$parseTree._AnyWord}} is {{$temp.pos}}.
        else:
            a: I don’t know...
tip
When executed, the code inside the scriptEs6 tag is wrapped in asynchronous function, so the await keyword is allowed.

Built-in variables in asynchronous functions​

In the ECMASCript 6 runtime, you can write asynchronous functions in the scriptEs6 tags and JS files.

Currently, only the following built-in variables are supported in such functions:

Syntax​

  • Write a value into a variable:

    $session.example = "Example";

  • Get a value from a variable:

    var example = await $session.example;

Example​

Operations with the same variable are guaranteed to be executed sequentially.

async function sendCode() {
    $session.code = 1234;
    $conversationApi.sendTextToClient(await $session.code);
}

In this example, the $conversationApi.sendTextToClient() method does not send the response instantly. It will be sent only after the operation of writing the value to $session.code is finished.

Run XML tests​

Tovie Platform provides XML tests for automatic bot testing before publishing. As with JS API, not all the features were ported to the new runtime.

If you want to test a code that runs inside the scriptEs6 tag, the following tags will not work in the test case:

tip
If you are developing a project locally, you can use any convenient tool for JS code testing (but not for script testing). For example, the Jest framework.

  • MDN Web Docs is a web development portal by Mozilla. Its section on JavaScript is closest in status to the official language documentation.

  • Exploring JS: books on JS from the above-mentioned Dr. Axel Rauschmayer, which also cover the latest ECMAScript specifications.

  • Introduction to Node.js is the official introduction to Node.js. You will learn more about package management, built-in modules, and other features of development for this platform from it.