Home / How-to Guides
Edit
Report

Provide AI prompt templates for end users with the AI Prompt Template plugin and Mistral

Uses: Kong Gateway AI Gateway deck
Deployment Platform
Related Documentation
Gateway Documentation decK Documentation
Tags
#ai #openai
Related Resources
AI Gateway
AI Proxy
AI Prompt Template
Minimum Version
Kong Gateway - 3.6
TL;DR

Configure the AI Proxy plugin to route traffic, then use the AI Prompt Template plugin to define and enforce reusable prompt formats.

Prerequisites

Kong Konnect

This is a Konnect tutorial and requires a Konnect personal access token.

  1. Create a new personal access token by opening the Konnect PAT page and selecting Generate Token.

  2. Export your token to an environment variable:

     export KONNECT_TOKEN='YOUR_KONNECT_PAT'

  3. Run the quickstart script to automatically provision a Control Plane and Data Plane, and configure your environment:

     curl -Ls https://get.konghq.com/quickstart | bash -s -- -k $KONNECT_TOKEN --deck-output

    This sets up a Konnect Control Plane named quickstart, provisions a local Data Plane, and prints out the following environment variable exports:

     export DECK_KONNECT_TOKEN=$KONNECT_TOKEN
 export DECK_KONNECT_CONTROL_PLANE_NAME=quickstart
 export KONNECT_CONTROL_PLANE_URL=https://us.api.konghq.com
 export KONNECT_PROXY_URL='http://localhost:8000'

    Copy and paste these into your terminal to configure your session.

Kong Gateway running

This tutorial requires Kong Gateway Enterprise. If you don’t have Kong Gateway set up yet, you can use the quickstart script with an enterprise license to get an instance of Kong Gateway running almost instantly.

  1. Export your license to an environment variable:

     export KONG_LICENSE_DATA='LICENSE-CONTENTS-GO-HERE'

  2. Run the quickstart script:

    curl -Ls https://get.konghq.com/quickstart | bash -s -- -e KONG_LICENSE_DATA

    Once Kong Gateway is ready, you will see the following message:

     Kong Gateway Ready

decK   v1.43+

decK is a CLI tool for managing Kong Gateway declaratively with state files. To complete this tutorial, install decK version 1.43 or later.

This guide uses deck gateway apply, which directly applies entity configuration to your Gateway instance. We recommend upgrading your decK installation to take advantage of this tool.

You can check your current decK version with deck version.

Required entities

For this tutorial, you’ll need Kong Gateway entities, like Gateway Services and Routes, pre-configured. These entities are essential for Kong Gateway to function but installing them isn’t the focus of this guide. Follow these steps to pre-configure them:

  1. Run the following command:

    echo '
_format_version: "3.0"
services:
  - name: example-service
    url: http://httpbin.konghq.com/anything
routes:
  - name: example-route
    paths:
    - "/anything"
    service:
      name: example-service
' | deck gateway apply -

To learn more about entities, you can read our entities documentation.

Mistral

This tutorial uses Mistral:

  1. Create a Mistral account.
  2. Get your API key.
  3. Export a decK environment variable with the Mistral API key:
export DECK_MISTRAL_API_KEY='YOUR MISTRAL API KEY'

Configure the AI Proxy plugin

Start by configuring the AI Proxy plugin to route prompts to Mistral AI.

echo '
_format_version: "3.0"
plugins:
  - name: ai-proxy
    config:
      route_type: llm/v1/chat
      auth:
        header_name: Authorization
        header_value: Bearer ${{ env "DECK_MISTRAL_API_KEY" }}
      model:
        provider: mistral
        name: mistral-tiny
        options:
          mistral_format: openai
          upstream_url: https://api.mistral.ai/v1/chat/completions
' | deck gateway apply -

Configure the AI Prompt Template plugin

Now, we can configure the AI Prompt Template plugin with predefined, reusable prompt templates for common tasks. This allows users to fill in the blanks with variable placeholders (``).

The plugin will automatically block all untemplated requests via allow_untemplated_requests: false setting.

This configuration defines five prompt templates:

Template name

Description
summarizer Summarizes long text into concise bullet points.
code-explainer Explains source code in beginner-friendly terms.
email-drafter Drafts professional emails based on topic and recipient.
product-describer Generates marketing descriptions from product details and features.
qna Answers user questions clearly and factually.

Configure the AI Prompt Template plugin:

echo '
_format_version: "3.0"
plugins:
  - name: ai-prompt-template
    config:
      allow_untemplated_requests: false
      templates:
      - name: summarizer
        template: |-
          {
              "messages": [
                {
                  "role": "system",
                  "content": "You summarize long texts into concise bullet points."
                },
                {
                  "role": "user",
                  "content": "Summarize the following text: {{text}}"
                }
              ]
          }
      - name: code-explainer
        template: |-
          {
              "messages": [
                {
                  "role": "system",
                  "content": "You are a helpful assistant who explains code to beginners."
                },
                {
                  "role": "user",
                  "content": "Explain what the following code does: {{code}}"
                }
              ]
          }
      - name: email-drafter
        template: |-
          {
              "messages": [
                {
                  "role": "system",
                  "content": "You write professional emails based on user input."
                },
                {
                  "role": "user",
                  "content": "Draft an email about {{topic}} to {{recipient}}."
                }
              ]
          }
      - name: product-describer
        template: |-
          {
              "messages": [
                {
                  "role": "system",
                  "content": "You write engaging product descriptions."
                },
                {
                  "role": "user",
                  "content": "Describe the product: {{product_name}, which has the following features: {{features}}."
                }
              ]
          }
      - name: qna
        template: |-
          {
              "messages": [
                {
                  "role": "system",
                  "content": "You answer questions clearly and accurately."
                },
                {
                  "role": "user",
                  "content": "Answer the following question: {{question}}"
                }
              ]
          }
' | deck gateway apply -

Validate configuration

Now, you can validate that the AI Prompt Template plugin configuration is correct by sending allowed and denied prompts.

Allowed prompts

This request uses the summarizer template:

curl -X POST "$KONNECT_PROXY_URL/anything" \
     --no-progress-meter --fail-with-body  \
     -H "Content-Type: application/json" \
     --json '{
       "messages": "{template://summarizer}",
       "properties": {
         "text": "Of all human sciences the most useful and most imperfect appears to me to be that of mankind: and I will venture to say, the single inscription on the Temple of Delphi contained a precept more difficult and more important than is to be found in all the huge volumes that moralists have ever written. I consider the subject of the following discourse as one of the most interesting questions philosophy can propose, and unhappily for us, one of the most thorny that philosophers can have to solve. For how shall we know the source of inequality between men, if we do not begin by knowing mankind? And how shall man hope to see himself as nature made him, across all the changes which the succession of place and time must have produced in his original constitution? How can he distinguish what is fundamental in his nature from the changes and additions which his circumstances and the advances he has made have introduced to modify his primitive condition? Like the statue of Glaucus, which was so disfigured by time, seas and tempests, that it looked more like a wild beast than a god, the human soul, altered in society by a thousand causes perpetually recurring, by the acquisition of a multitude of truths and errors, by the changes happening to the constitution of the body, and by the continual jarring of the passions, has, so to speak, changed in appearance, so as to be hardly recognisable. Instead of a being, acting constantly from fixed and invariable principles, instead of that celestial and majestic simplicity, impressed on it by its divine Author, we find in it only the frightful contrast of passion mistaking itself for reason, and of understanding grown delirious."
       }
     }'

curl -X POST "http://localhost:8000/anything" \
     --no-progress-meter --fail-with-body  \
     -H "Content-Type: application/json" \
     --json '{
       "messages": "{template://summarizer}",
       "properties": {
         "text": "Of all human sciences the most useful and most imperfect appears to me to be that of mankind: and I will venture to say, the single inscription on the Temple of Delphi contained a precept more difficult and more important than is to be found in all the huge volumes that moralists have ever written. I consider the subject of the following discourse as one of the most interesting questions philosophy can propose, and unhappily for us, one of the most thorny that philosophers can have to solve. For how shall we know the source of inequality between men, if we do not begin by knowing mankind? And how shall man hope to see himself as nature made him, across all the changes which the succession of place and time must have produced in his original constitution? How can he distinguish what is fundamental in his nature from the changes and additions which his circumstances and the advances he has made have introduced to modify his primitive condition? Like the statue of Glaucus, which was so disfigured by time, seas and tempests, that it looked more like a wild beast than a god, the human soul, altered in society by a thousand causes perpetually recurring, by the acquisition of a multitude of truths and errors, by the changes happening to the constitution of the body, and by the continual jarring of the passions, has, so to speak, changed in appearance, so as to be hardly recognisable. Instead of a being, acting constantly from fixed and invariable principles, instead of that celestial and majestic simplicity, impressed on it by its divine Author, we find in it only the frightful contrast of passion mistaking itself for reason, and of understanding grown delirious."
       }
     }'

Denied prompts

All requests that don’t use any of the configured templates will be automatically blocked by the plugin. For example:

curl -X POST "$KONNECT_PROXY_URL/anything" \
     --no-progress-meter --fail-with-body  \
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "What is Pythagorean theorem?"
         }
       ]
     }'

You should see the following response:

this LLM route only supports templated requests

curl -X POST "http://localhost:8000/anything" \
     --no-progress-meter --fail-with-body  \
     -H "Content-Type: application/json" \
     --json '{
       "messages": [
         {
           "role": "user",
           "content": "What is Pythagorean theorem?"
         }
       ]
     }'

You should see the following response:

this LLM route only supports templated requests

Cleanup

Clean up Konnect environment

If you created a new control plane and want to conserve your free trial credits or avoid unnecessary charges, delete the new control plane used in this tutorial.

Destroy the Kong Gateway container

 
curl -Ls https://get.konghq.com/quickstart | bash -s -- -d
Something wrong?
Report an Issue | Edit this Page

Help us make these docs great!

Kong Developer docs are open source. If you find these useful and want to make them better, contribute today!
Contribute

Still need help

Ask in our Forum
Contact Support