Build a TypingMind Plugin

Develop your own plugin OpenAI Function Spec Code Implementation Plugin User Settings (JSON)Accessing User Settings in Plugin Code Custom Output When to use custom output Custom output structure Output Type Test your plugin Example plugins

TypingMind Plugins allow you to extend the capabilities of the AI assistant by providing external tools. To get started, click the “Plugins” button on the start screen.

🧪

Note: TypingMind Plugins is currently in beta. This document will be updated to have more sections soon.

👋

Join our Discord channel to discuss about plugins! 👉 typingmind.com/discord-plugins

Develop your own plugin

Plugins are simple JavaScript code that will be executed on your local browser using the OpenAI Function Calling API.

Requirements for developing a plugin:

You can write code in JavaScript

You understand the OpenAI Function Calling API

To create a plugin for TypingMind, you will need to provide both the OpenAI spec for the function, and the implementation of the function itself in JavaScript.

Click “Create New Plugin” to start creating your own plugins.

⚠️

At the moment you can only create new plugins manually. Later, we will add a plugins marketplace and a way to import your plugins from GitHub soon.

Enter the relevant information about the plugin.

Plugin Name: to be shown to the user.

Overview: introduction about the plugin and how to use it. Markdown is supported.

The two most important parts of developing a plugin is to provide an OpenAI Function Spec and the Code Implementation.

OpenAI Function Spec

✅ Make sure it’s in the correct JSON format.

Follow the instructions in OpenAI document to understand how to write one. You can see the spec of existing plugins like Simple Calculator, JavaScript Interpreter, or the Generate Random Number example plugin below for reference.

✅ Unique Function Name

Note that the name field of the spec must be unique across all plugins you currently have. This is because OpenAI API will use this name to call the function, having multiple functions with the same name will confuse the AI.

✅ Use meaningful names & descriptions

It’s important to provide a meaningful function name, function description, parameter name, parameter description, etc.

The AI will use these names and descriptions to understand and decide when to use your function. Providing a meaningful and accurate name will help the AI use your function/plugin better.

✅ Example of a JSON OpenAI Function Spec:

json
{
  "name": "generate_random_number_in_range",
  "description": "Generate a random integer number between number 'a' and number 'b'.",
  "parameters": {
    "type": "object",
    "properties": {
      "a": {
        "type": "number",
        "description": "The number a (the smaller number)"
      },
      "b": {
        "type": "number",
        "description": "The number b (the larger number)"
      }
    },
    "required": [
      "a",
      "b"
    ]
  }
}

Code Implementation

✅ Make sure it’s a valid JavaScript

Write the function implementation in JavaScript. You can use async functions if necessary.

✅ Make sure the function name match

It’s important that you define a function with the exact same name as defined in the JSON spec at the top level. For example, if you have "name": "get_calculation_result" in your JSON spec, then the code must have a function named get_calculation_result defined at the top level.

✅ Only two parameters

By OpenAI’s convention, your function always accept only two parameters, both of type object.

First parameter (params): The parameter created by the AI assistant everytime your function is created. The object contains multiple key/value pairs that the AI will decide when they execute your function.

Second parameter (userSettings): If your plugin has a User Settings (discussed in a later section), the key/value pair of the user settings will be passed into this object.

Example:

javascript
async function get_search_results(params, userSettings) {
  const { keyword, pages } = params;
  const { searchAPIKey } = userSettings;
  // your code here...
}

✅ Execution Environment

The code will be executed directly in your local browser, there is no server component.

A sandbox environment will be created for the code by using <iframe sandbox="allow-scripts">. This sandbox environment provides some level of isolation, privacy, and security for the plugin code. Learn what you can and cannot do in an iframe sandbox here.

If you are using someone else’s code, make sure you read and understand what it does. Never use untrusted code in your plugin.

✅ Be aware of CORS issues

Because the code is executed locally on your browser, if you decide to send a request to an external server, make sure that external server is configurated correctly to accept the requests without any CORS related issues.

✅ Example code of a plugin:

javascript
function generate_random_number_in_range({a, b}) {
    var min = Math.ceil(a);
    var max = Math.floor(b);
    return Math.floor(Math.random() * (max - min + 1) + min);
}

Plugin User Settings (JSON)

This is a feature that allows you to define necessary input fields for your plugin which would be filled by users. The parameters in this section accept JSON formatted strings.

An example of the JSON format:

json
[
  {
    "name": "searchEngineID",
    "label": "Search Engine ID"
  },
  {
    "name": "searchEngineAPIKey",
    "label": "Search Engine API Key",
    "type": "password"
  }
]

This essentially tells the UI to render input fields where users can enter their own specific data. Each object in the userSettings array represents a single user input field.

Here's what each key represents:

name: This is the identifier the plugin will use to retrieve the user's input.

label: This is the label displayed to the user to help identify what information should be entered. Try to make this intuitive and relevant to the data you're asking for.

type (optional): This is used to specify the type of data expected from the users. For instance, input type could be 'password', 'email', 'number' etc. By default, if not specified, the type is assumed to be 'text'.

placeholder (optional): The placeholder text to be shown in the input field.

To use these settings within your plugin, you can access them from the user's input and either store them for later use or use them immediately as needed.

Accessing User Settings in Plugin Code

Once you've defined user settings for your plugin, the next step is to retrieve those values within your plugin's function. User settings are passed into your function as its second parameter.

Here's how you can access user settings from your function:

javascript
function search_images_via_google(params, userSettings) {
  // Your code here
}

In the above example, search_images_via_google is the function that gets called in your plugin. You see that params is the first parameter and userSettings is the second parameter:

params is an object that contains the runtime parameters passed by the AI or users.

userSettings is the object that contains all the custom settings configured by the user.

The userSettings parameter is essentially an object where the keys are the names of the user settings you previously defined and the value is whatever the user provided for those settings.

For instance, if you have the following user settings:

json
[
  {
    "name": "searchEngineID",
    "label": "Search Engine ID"
  },
  {
    "name": "searchEngineAPIKey",
    "label": "Search Engine API Key",
    "type": "password"
  }
]

You can retrieve those settings in your function as follows:

javascript
function search_images_via_google(params, userSettings) {
  const searchEngineID = userSettings.searchEngineID;
  const searchEngineAPIKey = userSettings.searchEngineAPIKey;

  // Now you can use these values in your code
}

In this example, searchEngineID and searchEngineAPIKey are keys within the userSettings object, and the corresponding values are what the user provided in the settings fields.

By accessing user settings in this way, you add a layer of customization to your plugin, enabling it to cater to the specific needs and preferences of different users.

Custom Output

When to use custom output

Normally, the return value of your plugin will be passed directly to the AI, then the AI will decide how to answer the user based on the returned value.

However, in some cases, you prefer to render the output to the users directly by yourself. Custom Output allows you to do that.

For example, the DALL-E 3 plugin will render the generated images directly to the users in Markdown format. This saves time for the users because the AI will not need to “type” the long image URLs manually after it received the return value from the plugin.

In order to do this, the DALL-E 3 plugin returns a special JavaScript object with the following structure:

javascript
return {
  _TM_CUSTOM_OUTPUT: true,
  type: 'markdown',
  data: markdownText,
  response: "The images have been rendered and shown to the user."
};

Custom output structure

Here are the required fields of a custom output object:

Field	Required?	Description	Accepted values
`_TM_CUSTOM_OUTPUT`	Yes	This is a special key to let Typing Mind knows you want to render custom output.	Must be `true` (boolean)
`type`	Yes	The type of output that your plugin will return. Currently, only `markdown` and `html` is supported. Check the “Output Type” section for more information.	Must be `"markdown"` or `"html"` (string)
`data`	Yes	Depends on the value the `type` field, this data field contains necessary information to render the output.	Check the “Output Type” section for more information.
`response`	Yes	This is the response you want to pass to the AI. Normally, you want to say something like “The content has been rendered and shown to the user.” so that the AI understand what happened.	Can be any string.

Output Type

Markdown Type

type must be "markdown"

data must be a string containing the markdown text.

Fully supported GitHub Flavored Markdown.

Example: See the built-in “DALL-E 3” plugin in Typing Mind.

HTML Type

type must be "html"

data must be an object following this format:

javascript
{
	code: "<html>...", // your HTML/JS/CSS code
	options: {
		height: 400 // the height of the container in pixel, max 1000px
	}
}

Your HTML will be rendered in a safe sandbox iframe container using <iframe sandbox="allow-scripts">. The container’s width will always be 100% of the user’s device’s width. The container’s height can be chosen by suppling the {options: {height: 300}} in the custom output object.

Example: See the built-in “Render HTML” plugin in Typing Mind.

Test your plugin

After filling in all the information, click “Save” and enable your newly created plugin.

Then, you can process to chat with the AI.

Note that the AI will decide to use your plugin only when necessary. Make sure your JSON spec has meaningful and accurate names and description.

💡

Hallucination problem: Sometimes, the AI assistant hallucinates and try to call your function with incorrect parameter, or even attempt to call function names that does not exists. This is a known issue and there is not much we can do about this. Just click “Regenerate” and let the AI assistant try again.

Example plugins

Click here to see some example plugins created by the community:

🔘Plugins Examples

Last Update: Jun 29, 2023