Incoming webhook integrations

An incoming webhook allows a third-party service to push data to Zulip when something happens. There's several ways to do an incoming webhook in Zulip:

• Use our REST API endpoint for sending messages. This works great for internal tools or cases where the third-party tool wants to control the formatting of the messages in Zulip.
• Use one of our supported integration frameworks, such as the Slack-compatible incoming webhook, Zapier integration, or IFTTT integration.
• Adding an incoming webhook integration (detailed on this page), where all the logic for formatting the Zulip messages lives in the Zulip server. This is how most of Zulip's official integrations work, because they enable Zulip to support third-party services that just have an "outgoing webhook" feature (without the third party needing to do any work specific to Zulip).

In an incoming webhook integration, the third-party service's "outgoing webhook" feature sends an HTTP POSTs to a special URL when it has something for you, and then the Zulip "incoming webhook" integration handles that incoming data to format and send a message in Zulip.

New official Zulip webhook integrations can take just a few hours to write, including tests and documentation, if you use the right process.

Quick guide

• Set up the Zulip development environment.

• Use Zulip's JSON integration, https://webhook.site/, or a similar site to capture an example webhook payload from the third-party service. Create a zerver/webhooks/<mywebhook>/fixtures/ directory, and add the captured payload as a test fixture.

• Create an Integration object, and add it to WEBHOOK_INTEGRATIONS in zerver/lib/integrations.py. Search for webhook in that file to find an existing one to copy.

• Write a draft webhook handler under zerver/webhooks/. There are a lot of examples in that directory that you can copy. We recommend templating off a short one, like zendesk.

• Add a test for your fixture at zerver/webhooks/<mywebhook>/tests.py. Run the tests for your integration like this:

  tools/test-backend zerver/webhooks/<mywebhook>/

  Iterate on debugging the test and webhooks handler until it all works.

• Capture payloads for the other common types of POSTs the third-party service will make, and add tests for them; usually this part of the process is pretty fast.

• Document the integration (required for getting it merged into Zulip). You can template off an existing guide, like this one. This should not take more than 15 minutes, even if you don't speak English as a first language (we'll clean up the text before merging).

Hello world walkthrough

Check out the detailed walkthrough for step-by-step instructions.

Checklist

Files that need to be created

Select a name for your incoming webhook and use it consistently. The examples below are for a webhook named MyWebHook.

• zerver/webhooks/mywebhook/__init__.py: Empty file that is an obligatory part of every python package. Remember to git add it.
• zerver/webhooks/mywebhook/view.py: The main webhook integration function as well as any needed helper functions.
• zerver/webhooks/mywebhook/fixtures/messagetype.json: Sample json payload data used by tests. Add one fixture file per type of message supported by your integration.
• zerver/webhooks/mywebhook/tests.py: Tests for your webhook.
• zerver/webhooks/mywebhook/doc.md: End-user documentation explaining how to add the integration.
• static/images/integrations/logos/mywebhook.svg: A square logo for the platform/server/product you are integrating. Used on the documentation pages as well as the sender's avatar for messages sent by the integration.
• static/images/integrations/mywebhook/001.svg: A screenshot of a message sent by the integration, used on the documentation page. This can be generated by running tools/generate-integration-docs-screenshot --integration mywebhook.
• static/images/integrations/bot_avatars/mywebhook.png: A square logo for the platform/server/product you are integrating which is used to create the avatar for generating screenshots with. This can be generated automatically from static/images/integrations/logos/mywebhook.svg by running tools/setup/generate_integration_bots_avatars.py.

Files that need to be updated

• zerver/lib/integrations.py: Add your integration to WEBHOOK_INTEGRATIONS. This will automatically register a URL for the incoming webhook of the form api/v1/external/mywebhook and associate it with the function called api_mywebhook_webhook in zerver/webhooks/mywebhook/view.py. Also add your integration to DOC_SCREENSHOT_CONFIG. This will allow you to automatically generate a screenshot for the documentation by running tools/generate-integration-docs-screenshot --integration mywebhook.

Common Helpers

• If your integration will receive a test webhook payload, you can use get_setup_webhook_message to create our standard message for test payloads. You can import this from zerver/lib/webhooks/common.py, and it will generate a message like this: "GitHub webhook is successfully configured! 🎉"

General advice

• Consider using our Zulip markup to make the output from your integration especially attractive or useful (e.g. emoji, Markdown emphasis or @-mentions).

• Use topics effectively to ensure sequential messages about the same thing are threaded together; this makes for much better consumption by users. E.g. for a bug tracker integration, put the bug number in the topic for all messages; for an integration like Nagios, put the service in the topic.

• Integrations that don't match a team's workflow can often be uselessly spammy. Give careful thought to providing options for triggering Zulip messages only for certain message types, certain projects, or sending different messages to different streams/topics, to make it easy for teams to configure the integration to support their workflow.

• Consistently capitalize the name of the integration in the documentation and the Client name the way the vendor does. It's OK to use all-lower-case in the implementation.

• Sometimes it can be helpful to contact the vendor if it appears they don't have an API or webhook we can use; sometimes the right API is just not properly documented.

• A helpful tool for testing your integration is UltraHook, which allows you to receive webhook calls via your local Zulip development environment. This enables you to do end-to-end testing with live data from the service you're integrating and can help you spot why something isn't working or if the service is using custom HTTP headers.

URLs

The base URL for an incoming webhook integration bot is https://dsp2022.zulip.aalto.fi/api/v1/external/INTEGRATION_NAME?api_key=API_KEY where INTEGRATION_NAME is the name of the specific webhook integration and API_KEY is the API key of the bot created by the user for the integration.

The list of existing webhook integrations can be found in zerver/lib/integrations.py (at WEBHOOK_INTEGRATIONS) or by browsing the Integrations documentation.

Parameters accepted in the URL include:

• api_key: Required. The API key of the bot created by the user for the integration. To get a bot's API key, see the API keys documentation.

• stream: The stream for the integration to send notifications to. Can be either the stream ID or the URL-encoded stream name. By default the integration will send direct messages to the bot's owner.

  A stream ID can be found when browsing streams in the web app via the URL.

• topic: The topic in the specified stream for the integration to send notifications to. The topic should also be URL-encoded. By default the integration will have a topic configured for stream messages.

• only_events, exclude_events: Some incoming webhook integrations support these parameters to filter which events will trigger a notification. For details, see the integration's integration documentation page.