WhatsApp Integration Architecture & Gotchas

This document details the architecture and implementation of WhatsApp integrations in the Mera Brand application ecosystem. We support two primary integration flows, both powered by the Evolution API.

1. Django Direct Integration

Our backend (Django) communicates directly with Evolution API to facilitate programmatic messaging. This includes functionalities like system notifications, marketing broadcasts, and triggered updates.

Flow: - The Django application initiates an API request to the Evolution API to create an instance and generate a WhatsApp Web pairing code. - The user inputs their phone number, views the pairing code on our UI, and connects their WhatsApp app. - Once connected, Django sends POST requests (with apikey headers) to Evolution API endpoints to dispatch messages.

2. Chatwoot Omnichannel Integration

We utilize Chatwoot as the central customer support portal for our users. Evolution API seamlessly bridges Chatwoot and WhatsApp, acting as the middleware.

Flow: - An Evolution API instance is mapped to a specific Chatwoot account/inbox using the /chatwoot integration endpoint. - Inbound: When a user replies on WhatsApp, Evolution API triggers a webhook to Chatwoot's API channel inbox. The message appears instantly on the Chatwoot dashboard. - Outbound: When an agent replies in Chatwoot, Chatwoot fires a webhook event back to Evolution API, which dispatches the message to WhatsApp.

Important Gotchas & Hard-Learned Lessons

During the setup of our self-hosted Chatwoot and Evolution API ecosystem on a single EC2 instance, we encountered several obscure technical hurdles. If you ever have to rebuild, migrate, or debug this system in the future, check these first:

1. The Nginx "Underscores in Headers" Silently Dropped Tokens

  • Symptom: Inbound messages (from WhatsApp to Chatwoot) were silently failing to appear in Chatwoot. Evolution API logs showed 401 Unauthorized and Invalid Access Token errors when it tried to hit Chatwoot's webhook URL.
  • Root Cause: Chatwoot expects the API token in a header called api_access_token (with underscores). By default, Nginx considers headers with underscores invalid and silently strips them out before passing the request to the upstream server (Chatwoot).
  • The Fix: We had to explicitly enable underscores in headers in the Nginx configuration for the Chatwoot reverse proxy block. nginx server { ... underscores_in_headers on; ... }

2. SSRF Protection Blocking Chatwoot to Evolution API Communication

  • Symptom: Chatwoot could not send webhooks to Evolution API. Requests hung or failed with connection errors.
  • Root Cause: Chatwoot has strict Server-Side Request Forgery (SSRF) protections. It refuses to make outbound HTTP requests to private IP addresses or local domains (e.g., if chat.merabrand.space resolves to the local EC2 instance IP).
  • The Fix:
  • We had to update Chatwoot's environment variables to whitelist the URLs and allow private network fetching: env SAFE_FETCH_ALLOW_PRIVATE_NETWORK=true SSRF_ALLOWED_URLS=chat.merabrand.space,https://chat.merabrand.space
  • We added a Docker extra_hosts mapping in Chatwoot's docker-compose.yml so that Chatwoot maps chat.merabrand.space directly to the Docker Gateway IP (172.17.0.1), allowing secure, direct internal routing.

3. The Dreaded 5-Second Webhook Timeout (False "Failed to send" Errors)

  • Symptom: Text messages and media uploads (photos/documents) sent from Chatwoot were successfully delivered to the WhatsApp recipient, but Chatwoot's UI displayed a red Failed to send warning with the tooltip Read timeout with #TCP socket closed.
  • Root Cause: When Chatwoot fires a webhook to Evolution API to send a message, it expects a response almost instantly. However, Evolution API (via Baileys) communicates with WhatsApp's servers to send the message and wait for acknowledgment before returning a 200 OK to Chatwoot. This round-trip often takes 5-6 seconds (especially for media). Chatwoot has a hardcoded default webhook timeout of exactly 5 seconds, causing it to prematurely sever the connection and assume the send failed.
  • The Fix (TRICKY): Simply updating WEBHOOK_TIMEOUT=30 in the .env file did not work. Chatwoot ignores the .env variable if the config has ever been initialized because it prioritizes its internal PostgreSQL installation_configs table and caches the value heavily in Redis (serialized as YAML/JSON). To properly update the timeout: bash # Inside the Chatwoot docker container with native env vars: bundle exec rails runner "InstallationConfig.where(name: 'WEBHOOK_TIMEOUT').destroy_all; InstallationConfig.create(name: 'WEBHOOK_TIMEOUT', value: 30); GlobalConfig.clear_cache;" This forcefully writes the 30-second timeout to the Postgres DB and clears the Redis cache, allowing the system to wait patiently for media uploads.

4. "Ghost" Instances in Evolution API

  • Symptom: The Evolution API logs were flooded with 401 Unauthorized background errors attempting to hit endpoints like /api/v1/accounts/10/contacts/filter.
  • Root Cause: When cleaning up test data, we deleted Chatwoot Accounts (e.g., Account ID 10) directly via Rails console. However, the Evolution API instances mapped to those accounts were left behind as orphans. They continued polling or sending status updates to Chatwoot with invalidated API tokens.
  • The Fix: Always delete the Evolution API instance (using their DELETE /evolution/instances/logout API) before you wipe the Chatwoot account. If ghost instances occur, manually delete them from the Evolution API container.