Cloudflare integration (Worker)

Set up and deploy a Cloudflare Worker that captures HTTP request data and forwards it to Profound Agent Analytics.

Overview

The integration uses a Cloudflare Worker that runs as middleware to collect request metadata and send it to our Agent Analytics API. The Worker captures important request information like IP addresses, user agents, and referrers without affecting the actual request handling.

Prerequisites

• A Cloudflare account with access to Workers

• Node.js installed on your development machine

• Access to your domain’s Cloudflare configuration

• A Profound Log Ingestion Token for Agent Analytics

Using Cloudflare Enterprise plan? Check out our integration guide using Cloudflare Logpush. Logpush is always preferable to a Worker: it is out-of-band and cannot affect live traffic.

A Worker on a * route sits in the request path for every request to that hostname, so it can affect production traffic. Use the code below as-is, and in particular:

• Never clone the response or read the response body inside the Worker. Buffering bodies can exceed the Worker’s 128 MB isolate memory limit, which Cloudflare surfaces to the visitor (or crawler) as Error 1102 Worker exceeded resource limits.

• Keep the logging call inside ctx.waitUntil() and swallow its errors so a slow or failing log request can never turn into a 5xx.

• Keep sensitive paths out of the Worker entirely. The path list in the code below only skips logging; requests still pass through the Worker. To take checkout, cart, and admin paths out of the Worker’s path completely, narrow the route pattern itself (or add a higher-priority route that is not bound to the Worker).

Symptoms of a Worker that breaks these rules are intermittent 5xx responses on large or streamed responses, and downstream fallout such as Google Merchant Center / Google Ads disapprovals (“destination not working”, “image not accessible”) when AdsBot-Google hits one of those errors during a crawl.

Running a Shopify storefront? Do not put a Worker in front of it. Use one of the log drain options in our Shopify guide instead, which collects logs out-of-band.

Implementation guide

1. Set up your development environment

   Create a new Worker project and install dependencies:

   Create a new Worker project

   First, use the Cloudflare Worker CLI to create a new Worker project:

   In this example, we’re using version 2.37.4 of the Cloudflare Worker CLI. You may use the latest version, but there might be some minor process differences.

   npm create cloudflare@2.37.4 -- log-collector
   

   After npm launches, you’ll be prompted to select a starting point. Select “Hello World” as your starting category.

   

   Select the “Hello World” worker template.

   

   Select the “TypeScript” language.

   

   Now the CLI will create a new project with the name log-collector and install the necessary dependencies.

   Select Git for version control.

   

   Deploy the application now. The application will be deployed with a Cloudflare development domain and will not affect your production environment.

   

   Cloudflare CLI will prompt you to login and select the account you want to deploy the application to. Please choose the account your domain is associated with.

   Cloudflare should automatically open a browser window with “Hello World” displayed. You can navigate to the project directory once the application is deployed.

   cd log-collector
   
2. Configure your Worker

   Edit your wrangler.json file to configure the PROFOUND_API_URL environment variable and the route binding:

   Replace pattern example.com/* with your actual domain. Use your target site URL (usually marketing site). For example, if your marketing site is https://www.example.com, you should use www.example.com/* as the pattern. The zone_name should be your canonical domain without the www.

   If you are not sure about the correct configuration, please contact Profound support.

   wrangler.json

   {
     "$schema": "node_modules/wrangler/config-schema.json",
     "name": "log-collector",
     "main": "src/index.ts",
     "compatibility_date": "2025-01-29",
     "observability": {
       "enabled": true
     },
     "route": {
       "pattern": "example.com/*",
       "zone_name": "example.com"
     },
     "vars": { "PROFOUND_API_URL": "https://artemis.api.tryprofound.com/v1/logs/cloudflare_worker" }
   }
   

   Then copy the TypeScript code into src/index.ts:

   src/index.ts

   /**
    * Cloudflare Worker for Log Collection
    *
    * Forwards request metadata to Profound's log collection API. The origin
    * response body is never read or modified, and every error in the logging
    * path is swallowed, so logging cannot alter the response. Failures are
    * logged with enough detail to tell a rejected request apart from a
    * timeout or a connection error.
    */
   
   export interface Env {
       PROFOUND_API_URL: string;
       PROFOUND_LOG_INGESTION_TOKEN: string;
   }
   
   // Paths that are not logged. Matched on full path segments, so '/cart' does
   // not match '/cartography'.
   const EXCLUDED_PATHS = ['/checkout', '/cart', '/admin', '/api'];
   
   // Characters kept from a rejected response body when logging the failure.
   const DETAIL_LIMIT = 512;
   
   // Percent-decoded and lowercased so encoded variants such as '/%63heckout'
   // and '/checkout%2Fpayment' match the same way the origin routes them.
   function normalizePath(pathname: string): string {
       try {
           return decodeURIComponent(pathname).toLowerCase();
       } catch {
           return pathname.toLowerCase();
       }
   }
   
   function isExcluded(pathname: string): boolean {
       const path = normalizePath(pathname);
       return EXCLUDED_PATHS.some(
           (excluded) => path === excluded || path.startsWith(`${excluded}/`),
       );
   }
   
   export default {
       async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
           const response = await fetch(request);
   
           const skip = response.status === 101 || isExcluded(new URL(request.url).pathname);
   
           if (!skip) {
               ctx.waitUntil(
                   sendLog(request, response, env).catch((error: unknown) =>
                       console.error(
                           'Failed to send logs:',
                           error instanceof Error ? `${error.name}: ${error.message}` : error,
                       ),
                   ),
               );
           }
   
           return response;
       },
   } satisfies ExportedHandler<Env>;
   
   async function sendLog(request: Request, response: Response, env: Env) {
       const requestUrl = new URL(request.url);
   
       // +2 for ': ' and +2 for '\r\n' per header
       const headerSize = Array.from(response.headers.entries()).reduce(
           (total, [key, value]) => total + key.length + value.length + 4,
           0,
       );
   
       // Derived from content-length so the body is never buffered. Streamed or
       // chunked responses have no content-length and are reported as headers only.
       // HEAD and bodyless statuses advertise a content-length that is never sent,
       // so their body size is counted as zero.
       const contentLength = Number(response.headers.get('content-length'));
       const transmitsBody =
           request.method !== 'HEAD' && response.status !== 204 && response.status !== 304;
       const bodySize = transmitsBody && Number.isFinite(contentLength) ? contentLength : 0;
       const bytes = headerSize + bodySize;
   
       const logData = {
           timestamp: Date.now(),
           host: requestUrl.hostname,
           method: request.method,
           pathname: requestUrl.pathname,
           query_params: Object.fromEntries(requestUrl.searchParams),
           ip: request.headers.get('cf-connecting-ip'),
           userAgent: request.headers.get('user-agent'),
           referer: request.headers.get('referer'),
           bytes,
           status: response.status,
       };
   
       const logResponse = await fetch(env.PROFOUND_API_URL, {
           method: 'POST',
           headers: {
               'Content-Type': 'application/json',
               'X-API-Key': env.PROFOUND_LOG_INGESTION_TOKEN,
           },
           body: JSON.stringify([logData]),
           signal: AbortSignal.timeout(5000),
       });
   
       // Only the first chunk of a rejected response is read, and only up to
       // DETAIL_LIMIT characters of it are kept, so a misconfigured URL that
       // returns a large body can never fill the isolate. The origin response
       // is still never read.
       if (!logResponse.ok) {
           const reader = logResponse.body?.getReader();
           const chunk = await reader?.read();
           await reader?.cancel();
   
           const detail = chunk?.value
               ? new TextDecoder().decode(chunk.value).slice(0, DETAIL_LIMIT)
               : '';
   
           console.error(`Log ingestion rejected the request: ${logResponse.status} ${detail}`);
           return;
       }
   
       await logResponse.body?.cancel();
   }
   

   The Worker logs two distinct failures. Log ingestion rejected the request: <status> <body> means Profound received the request and refused it. This is usually an authentication or payload problem, and the status and body identify which. Failed to send logs: <name>: <message> means the request never completed. TimeoutError is the five-second timeout. Other names point to a connection or configuration problem. Both messages go to the Worker’s logs. You can read them under Workers & Pages > your Worker > Logs.

   bytes is derived from the content-length response header instead of the response body. Reading the body (for example via response.clone() and blob()) forces Cloudflare to buffer the whole payload in the Worker isolate, which fails with a 5xx on large or streamed responses. This Worker never reads or modifies the response body. It streams the origin response through unchanged, but it is still in the request path, so origin fetch failures surface as they would without it. bytes therefore counts headers plus content-length, and only when the response actually transmits a body (not HEAD, 204, or 304).

3. Deploy your Worker

   Log in to Cloudflare

   Use the Wrangler CLI to deploy the Worker:

   # Login to Cloudflare
   npx wrangler login
   

   Configure the Profound Log Ingestion Token

   Secrets is a feature of Cloudflare Workers that allows you to store sensitive information like Log Ingestion Tokens in a secure environment.

   # Configure the Log Ingestion Token secret
   npx wrangler secret put PROFOUND_LOG_INGESTION_TOKEN
   

   You will be prompted to enter the Log Ingestion Token. Copy and paste the token and press enter.

   

   Deploy the Worker

   # Deploy the Worker
   npx wrangler deploy
   
4. Test your implementation

   Verify your Worker is functioning correctly:

   Navigate to Agent Analytics in Profound and check that logs appear in the Logs panel. The AI log filter is on by default. Disable it using the filter in the top of the Logs panel.

   If logs appear, you are all set. You should see data populate in the Analytics dashboard.

Troubleshooting

• If logs aren’t appearing, verify your PROFOUND_API_URL environment variable and PROFOUND_LOG_INGESTION_TOKEN secret are configured correctly

• Log ingestion rejected the request: 401 or 403 in the Worker logs: the PROFOUND_LOG_INGESTION_TOKEN secret is missing, misspelled, or not the token issued for this site. Set it again with npx wrangler secret put PROFOUND_LOG_INGESTION_TOKEN

• Failed to send logs: TimeoutError: the log request passed the five-second timeout. A small number of these is expected and costs you only the affected log lines, since the timeout keeps the logging call away from your visitors. A sustained rate is worth reporting to Profound support with a few requestId values from the Worker logs

• Failed to send logs: with any other error name: check PROFOUND_API_URL against the value in the configuration step above. A URL that resolves to a hostname on your own Cloudflare zone makes the Worker call itself, which fails on every request

• Check Cloudflare Workers > Analytics for any execution errors

• Ensure your route pattern matches your domain configuration

• Verify the Worker is receiving requests by checking the Cloudflare dashboard metrics

• Intermittent 5xx after deploying the Worker: check Metrics > Errors > Invocation Statuses for Exceeded Memory (Error 1102) or Script threw exception (Error 1101). Both mean the Worker itself is failing, not your origin. Confirm you are running the code above (no clone(), no body reads) and that the logging call is wrapped in ctx.waitUntil() with a .catch()

• Google Ads / Merchant Center disapprovals such as “destination not working” or “image not accessible” after deploying the Worker: these come from AdsBot-Google or the Merchant Center fetcher receiving one of the Worker errors above. Fix the Worker errors, then request a re-review

• To rule the Worker out entirely, remove the route binding (npx wrangler triggers delete or delete the route in the dashboard) and confirm the errors stop

Additional resources

• Cloudflare Workers Documentation

• Wrangler CLI Documentation

• Contact [e-mail] for API-related questions

Security considerations

• Store Log Ingestion Tokens as secrets in production environments

• Regularly rotate Log Ingestion Tokens

• Monitor Worker usage and logs for unusual patterns