languageDetector

The languageDetector module detects the language of the user’s request and writes the two-letter ISO language code to $session.language.

Use this module to develop multilingual bots. With languageDetector you can detect the language of requests and give a response based on the value of $session.language. The module also features flexible settings: you can specify in which cases language detection is needed, customize the logic, and manage language detection methods.

To use languageDetector:

1. Add a key for Detect Language API to the project.
2. Import the module into the script.

How the module works

When you import the languageDetector module, the preProcess handler is added to the script.

tip

The module’s work depends on the settings you provided when importing. Learn more about the settings in the Configuration article.

You can also view the module’s source code in the script editor inside the Dependencies section.

For each request, the bot checks whether it is necessary to detect the language. If you need to detect the language for the request:

1. The bot receives the ISO code of the request language:

   • If there are 5 or more words in the request, the bot calls the $caila.detectLanguage method. You can view the list of supported languages on the FastText website.

   • If the request is less than 5 words, the bot calls the Detect Language API. This API better detects the language in short texts. You can view the list of supported languages on the Detect Language API website.

     caution

     To use the API, add a key to the project.

2. The bot checks whether the language is included in the targetLanguages list and whether it needs to be replaced with a related language using the sameGroupLanguageReplacement parameter.

Next, the bot acts depending on the results:

• If the language is successfully identified, the bot writes the ISO code of the language to $session.language.
• If the language cannot be identified, the bot transitions to the state from the stateWithLanguageQuestion parameter. If such a state is not specified, the bot specifies the previous value in $session.language using the previousLanguagePreference parameter or the default language from the defaultLanguage.

Detect Language API Key

If the user’s request is less than 5 words, the bot calls the Detect Language API. Also, if the $caila.detectLanguage method has detected a language not from the list of targetLanguages, the bot tries again to detect the language using the API.

To allow the bot to send requests to the API, add a key to the project:

1. Sign up for Detect Language API.

2. Get an API key on the website.

   info

   By default, you have access to a free subscription plan in the Detect Language API, which has limitations on the number of requests and data volume.

3. In the Tovie Platform project, go to the Secrets and variables section.

4. Add a secret:

   • Name: getLanguageToken.
   • Value: API key from the Detect Language API website.

caution

If you don’t want to use the Detect Language API, set the onlyCailaMethod parameter to true in the module settings. In this case, the bot will always detect the language only using $caila.detectLanguage.

Module import

Import the module into the script using the require tag. You can pass module settings using the injector parameter.

require: languageDetector/languageDetector.sc
    module = sys.zb-common
    injector = {
        detectorMode: "every",
        defaultLanguage: "en"
        }

How to use in a script

require: languageDetector/languageDetector.sc
    module = sys.zb-common
    injector = {
        detectorMode: "default", # The bot detects the language in the first and second request.
        defaultLanguage: "en", # The default language is English.
        targetLanguages: ["en", "fr"], # The target languages are English and French.
        detectorStates: ["/Complaint"], # The bot always detects the language for the /Complaint state.
        stateWithLanguageQuestion: "/SelectLanguage", # State with a question about the language.
        stateForChangingLanguage: "/SpeakInOtherLanguage" # State with intent for changing language.
        }

theme: /

    # ...

    # A state with multilingual intent.
    # This state is specified in detectorStates. The bot always detects the language if a request triggers this intent.
    state: /Complaint
        intent!: /complaint
        if: $session.language === "fr"
            a: Nous sommes vraiment désolés que …
        if: $session.language === "en"
            a: We are very sorry that …

    # The bot goes to this state if it cannot detect the language.
    # After this state, the bot will stop trying to detect the language. Exception: /Complaint.
    state: SelectLanguage
        # The bot offers to select a language.
        a: Select language

        state: Language
            intent: /languages
            # The bot extracts the language from the response and writes it to $session.language.
            script:
                $session.language = $parseTree._languages.name;

    # The bot goes to the state if the user asks to speak another language.
    # Example of a user request: "speak English".
    # After this state, the bot will stop trying to detect the language. The exception is /Complaint.
    state: SpeakInOtherLanguage
        intent!: /speak in another language
        # The bot extracts the language from the request and writes it to $session.language
        script:
            $session.language = $parseTree._languages.name;