Skip to main content
Version: 2.1.0

Build a new plugin for marketplace

Introduction

ToolJet marketplace is a place where you can find custom plugins and install them in your ToolJet instance. This document will help you to build a new plugin for ToolJet marketplace.

Prerequisites

Getting started

1. Enabling the marketplace for your instance

To enable the marketplace for your instance, you need to set the ENABLE_MARKETPLACE and ENABLE_MARKETPLACE_DEV_MODE environment variable to true in your .env file. Marketplacwe is disabled by default. Once you set the environment variable, restart your ToolJet instance. You can find the instructions to run ToolJet locally here. Marketplace can be accessed from '/integrations' route.

2. Installing tooljet-cli

ToolJet marketplace uses tooljet-cli to build and publish plugins. You can install it using npm.

npm install -g tooljet-cli

# verify the installation
tooljet --version

3. Creating a new plugin - Github plugin

Let's create a new Github plugin for ToolJet marketplace, which will authenticate a user using Github Personal Access Token and will include basic operations like fetching user details, fetching repositories, fetching issues and fetching pull requests.

# create a new plugin
tooljet plugin create github

Provide the plugin name and select the plugin type, which is a api in this case. Select yes when asked to create a new plugin for marketplace.

Provide the repository URL if hosted on GitHub, otherwise leave it blank.

When you create a plugin using the ToolJet CLI, an object is automatically added to the plugins.json file, which is located in the ToolJet/server/src/assets/marketplace/ directory. This object contains metadata about the plugin, such as its name, description, version, author, and other details. This plugins.json file serves as a registry of all the plugins that are available for use in ToolJet. When ToolJet server starts up, it reads this file and loads all the plugins that are listed in it.

note

It's important to note that the plugins.json file should not be manually edited as it is automatically generated by the ToolJet CLI. Any changes made to this file may cause issues with the proper functioning of the plugins in the system.

All marketplace plugins are stored in the /marketplace directory of the ToolJet repository. You can find the Github plugin here.

The directory structure of a typical ToolJet plugin looks like this:

github/
package.json
lib/
icon.svg
index.ts
operations.json
manifest.json
  • manifest.json should include information such as the name of plugin, description, etc.
  • operations.json should include the metadata of all the operations supported by the plugin.
  • index.ts is the main file. It defines a QueryService for the plugin. The QueryService handles running of queries, testing connections, caching connections, etc.
  • icon.svg is the icon for the plugin.
  • package.json is auto generated by the cli.
info

Why do we need a manifest.json file or an operations.json file?

The manifest.json files are consumed by a React component to create dynamic UI for connection forms by defining the schema of an API or data source. The schema includes information about the source, such as its name, type, and any exposed variables. It also includes options for authentication and other properties that can be customized by the user. The properties section defines the specific fields and their types that are required for connecting to the API or data source. The React component reads the manifest.json file and generates the necessary UI components based on the schema, allowing users to enter the required information for connecting to the source. This can include text inputs, dropdowns, checkboxes, and other UI elements, depending on the schema defined in the manifest.json file.

The operations.json file contains a schema definition for a particular data source, for example, Github. It describes the available operations and their parameters that can be used to query the data source.

A React component uses this schema to create queries in ToolJet applications to generate a UI that allows users to select the desired operation and provide the required parameters.

The component would use the properties defined in the operations.json file to create various UI elements, such as dropdowns, and input fields, and handle user interactions to create the final query. Once the user has filled in the required parameters, the component would use them to generate a query that can be executed against the data source, and return the results to the user.

In conclusion, manifest.json and operations.json files play an important role in creating dynamic UI components in ToolJet applications. These files define the schema for data sources and available operations, which is then consumed by React components to generate the necessary UI elements for users to interact with. By using these files, ToolJet enables users to easily connect to various APIs and data sources, perform queries and retrieve data in a user-friendly way.

4. Running the watcher script

The watcher script will watch for changes in the plugin directory and will automatically build the plugin. Reload the app if the plugin is already installed. This will help us to see the changes in the plugin immediately.

# run the watcher script from /marketplace
npm run start:dev
info

In order to obtain the most recent updates to the plugin, it is necessary to reload the plugin from the marketplace. When in development mode, a reload button will appear, allowing the latest files to be read from the file stream and served.

5. Defining the manifest.json file

We need to include the necessary options to construct the connection form.

  "properties": {
"credentials": {
"label": "Authentication",
"key": "auth_type",
"type": "dropdown-component-flip",
"description": "Single select dropdown for choosing credentials",
"list": [
{
"value": "personal_access_token",
"name": "Use Personal Access Token"
}
]

},
"personal_access_token": {
"token": {
"label": "Token",
"key": "personal_token",
"type": "password",
"description": "Enter personal access token",
"hint": "You can generate a personal access token from your Github account settings."
}
}
}

It includes information about authentication options, specifically a dropdown to choose a type of credentials and a field to enter a personal access token. The label, key, type, description, and hint properties are used to define the specific fields and their types required for connecting to the API or data source.

6. Defining the operations.json file

  "properties": {
"operation": {
"label": "Operation",
"key": "operation",
"type": "dropdown-component-flip",
"description": "Single select dropdown for operation",
"list": [
{
"value": "get_user_info",
"name": "Get user info"
},
{
"value": "get_repo",
"name": "Get repository"
},
{
"value": "get_repo_issues",
"name": "Get repository issues"
},
{
"value": "get_repo_pull_requests",
"name": "Get repository pull requests"
}
]
},
"get_user_info": {
"username": {
"label": "Username",
"key": "username",
"type": "codehinter",
"lineNumbers": false,
"description": "Enter username",
"width": "320px",
"height": "36px",
"className": "codehinter-plugins",
"placeholder": "Enter username"
}
},
"get_repo": {
"owner": {
"label": "Owner",
"key": "owner",
"type": "codehinter",
"lineNumbers": false,
"description": "Enter owner name",
"width": "320px",
"height": "36px",
"className": "codehinter-plugins",
"placeholder": "developer"
},
"repo": {
"label": "Repository",
"key": "repo",
"type": "codehinter",
"lineNumbers": false,
"description": "Enter repository name",
"width": "320px",
"height": "36px",
"className": "codehinter-plugins",
"placeholder": "tooljet"
}
},
"get_repo_issues": {
"owner": {
"label": "Owner",
"key": "owner",
"type": "codehinter",
"lineNumbers": false,
"description": "Enter owner name",
"width": "320px",
"height": "36px",
"className": "codehinter-plugins",
"placeholder": "developer"
},
"repo": {
"label": "Repository",
"key": "repo",
"type": "codehinter",
"lineNumbers": false,
"description": "Enter repository name",
"width": "320px",
"height": "36px",
"className": "codehinter-plugins",
"placeholder": "tooljet"
},
"state": {
"label": "State",
"key": "state",
"className": "codehinter-plugins col-4",
"type": "dropdown",
"description": "Single select dropdown for choosing state",
"list": [
{
"value": "open",
"name": "Open"
},
{
"value": "closed",
"name": "Closed"
},
{
"value": "all",
"name": "All"
}
]
}
},
"get_repo_pull_requests": {
"owner": {
"label": "Owner",
"key": "owner",
"type": "codehinter",
"lineNumbers": false,
"description": "Enter owner name",
"width": "320px",
"height": "36px",
"className": "codehinter-plugins",
"placeholder": "developer"
},
"repo": {
"label": "Repository",
"key": "repo",
"type": "codehinter",
"lineNumbers": false,
"description": "Enter repository name",
"width": "320px",
"height": "36px",
"className": "codehinter-plugins",
"placeholder": "tooljet"
},
"state": {
"label": "State",
"key": "state",
"type": "dropdown",
"className": "codehinter-plugins col-4",
"description": "Single select dropdown for choosing state",
"list": [
{
"value": "open",
"name": "Open"
},
{
"value": "closed",
"name": "Closed"
},
{
"value": "all",
"name": "All"
}
]
}
}
}

The operations.json file defines the operations that can be performed on the data source. It includes information about the operation type, the fields required to perform the operation, and the type of each field. The label, key, type, description, and hint properties are used to define the specific fields and their types required for connecting to the API or data source.

7. Add the npm package of Gitub to the plugin dependencies

# change directory to the plugin directory and install the npm package
cd plugins/github
npm i octokit --workspace=@tooljet-marketplace/github
info

Steps to install npm package to a plugin

npm i <npm-package-name> --workspace=<plugin-name-in-package-json>

The command npm i <npm-package-name> --workspace=<plugin-name-in-package-json> is used to install a specific npm package into a particular workspace of a multi-package repository.

The --workspace flag is used to specify the workspace where the package should be installed. In this case, we are installing the package in the @tooljet-marketplace/github workspace.

8. Implement the query execution logic in index.ts

The QueryService for the Github plugin handles the logic for running queries in index.ts. The QueryService receives the metadata of the data source, including the credentials and configurations for connecting and parameters for the query that was run.

For the Github datasource, the sourceOptions will include the credentials required for authentication, such as the personal access token. The queryOptions will have the configurations and parameters for the specific query, including the operation to be performed, such as getting the list of repositories for a specific user.

The QueryService will use this information to create and execute the necessary API requests against the Github API. The resulting data will be returned to the caller, which can then be further processed as required.

Create a new file query_operations.ts in the plugins/github/src directory and add the following code to it.

import { Octokit } from 'octokit'
import { QueryOptions } from './types'


export async function getUserInfo(octokit: Octokit, options: QueryOptions): Promise<object> {
const { data } = await octokit.request(
'GET /users/{username}',
{
username: options.username
}
);
return data;
}

export async function getRepo(octokit: Octokit, options: QueryOptions): Promise<object> {
const { data } = await octokit.request(
'GET /repos/{owner}/{repo}',
{
owner: options.owner,
repo: options.repo
}
);
return data;
}

export async function getRepoIssues(octokit: Octokit, options: QueryOptions): Promise<object> {
const { data } = await octokit.request(
'GET /repos/{owner}/{repo}/issues',
{
owner: options.owner,
repo: options.repo,
state: options.state || 'all'

}
);
return data;
}

export async function getRepoPullRequests(octokit: Octokit, options: QueryOptions): Promise<object> {
const { data } = await octokit.request(
'GET /repos/{owner}/{repo}/pulls',
{
owner: options.owner,
repo: options.repo,
state: options.state || 'all'
}
);
return data;
}

The query_operations.ts file contains the functions that will be used to execute the queries. The functions will be called by the QueryService in index.ts.

The Github class has three methods:

  • run: This method is called when a query needs to be executed. It takes in sourceOptions and queryOptions as input, which represent the source metadata and the query configuration, respectively. The run method uses the octokit library to make API requests to the GitHub API and returns the result of the query in a QueryResult object.

  • testConnection: When a new data source is being added to a ToolJet application, the connection can be tested. This method is called when a connection needs to be tested. It takes in sourceOptions as input, which represents the source metadata. The testConnection method tests the connection by attempting to get the authenticated user and returns a ConnectionTestResult object that indicates whether the connection was successful or not.

note

Every data source might not have a way to test connection. If not applicable for your data source, you can disable the test connection feature by adding "customTesting": true, to the manifest.json of your plugin. ::

  • getConnection: This method is a helper method that returns an authenticated octokit client that is used to make requests to the GitHub API. It takes in sourceOptions as input, which represents the source metadata, and returns an authenticated octokit client.