Page MenuHomePhabricator

[SPIKE] Establish a technical strategy for a standard, generalized Click Through Rate (CTR) instrument
Open, HighPublic5 Estimated Story Points

Description

As an instrumentation developer, I want to easily add a Click-Through Rate instrument to my code without having to custom create one from scratch so I can speed up my time to delivery.

We have thought and talked about ways we can create elegant easy to use standard instruments to collect user interaction based on predefined "essential metrics". It's been discussed a little bit in T363979: [EPIC] Create a standardised page lifecycle instrument mixin, various slack threads and meetings.

Now that we have the MPIC Alpha implementation plan, are collaborating with the Growth team on FY24@5 SDS 2.1.3, and the Click Through Rate has been defined as part of FY 24/25 SDS 2.2, it's time for us to define a technical approach.

Event Timeline

Change #1030064 had a related patch set uploaded (by Phuedx; author: Phuedx):

[mediawiki/extensions/WikimediaEvents@master] WIP: Add standard instruments

https://gerrit.wikimedia.org/r/1030064

@VirginiaPoundstone: As discussed, I've added a patch containing the sample CTR instrument ready for review/refinement during the sprint with Growth.

I'm holding off on this until next week's sprint with Growth.

Proposal for an API is drafted in gerrit. Best audience for this is a team who does not yet have a CTR instrument.

Milimetric set the point value for this task to 5.Oct 3 2024, 11:10 AM

Background

We (Data Products) have proposed implementations of several so-called "instrument mixins". The goal of creating instrument mixins is to reduce Time To Data for feature teams who are trying to understand how users are using their feature. Instrument mixins helps us achieve this goal by:

  1. Reducing both the implementation and maintenance costs
  2. Reducing the cost of visualizing the data collected by collecting standardized events
  3. Reducing the amount of so-called "custom data" collected

In order to allow feature teams to use these mixins and for them to create their own, we need to provide an interface for mixins to implement and one or more APIs.

Proposal

Interface

type Config = Record<string, any>;

interface Mixin {

	/**
	 * Configures and starts the mixin.
	 * 
	 * @param streamName
	 * @param streamID
	 * @param [config]
	 */
	start: (
		streamName: string,
		streamID: string,
		config?: Config
	) => void;

	/**
	 * Stops the mixin.
	 * 
	 * @param streamName
	 * @param streamID
	 */
	stop: ( streamName: string, schemaID: string ) => void;
}

Standard API

The Standard API allows instrument developers to use one or more mixins with as little friction as possible.

interface Instrument {

	/**
	 * Includes the mixin to this instrument.
	 * 
	 * The mixin is configured and started immediately.
	 *
	 * @param mixin
	 * @param [config]
	 */
	include: ( mixin: Mixin, config?: Config ) => self;
}
const {
	SessionLength,
	PageviewLength,
	NavigationTiming /*, etc. */
} = require( 'ext.metricsPlatform.mixins' );

const i =
	  mw.eventLog.newInstrument( STREAM_NAME, STREAM_ID )
		  .include(
			  SessionLength,
			  PageviewLength,
			  NavigationTiming
		  );

Backwards-Compatible API

The Backwards-Compatible API allows instrument maintainers to use one more mixins with their existing instrument, which may or may not use the Metrics Platform Client API.

const { SessionLength } = require( 'ext.metricsPlatform.mixins' );

SessionLength.start( STREAM_NAME, STREAM_ID );

Background

Product Analytics have published a definition Clickthrough Rate (CTR), which has been signed off by stakeholders.

Other teams have implemented their own CTR instruments. For example, the Web team maintains a *Click-tracking for Vector and Minerva* instrument, a non-compliant implementation of CTR that collects **a lot** of extraneous data.

We (Data Products) should implement and maintain a compliant CTR implementation and dashboard that is so trivial to use that very nearly all instrumentation engineers choose to use it rather than implement their own.

Implementation

const STREAM_NAME = 'TODO';
const SCHEMA_ID = '/analytics/product_metrics/web/base/1.2.0';

// State
// =====

/**
 * @typedef {Object} StateEntry
 * @property {string} selector
 * @property {string} friendlyName
 * @property {Element} element
 * @property {number} elementClickCount
 * @property {string} funnelEntryToken
 */

/** @type {Map<HTMLElement,StateEntry>} */
const state = new WeakMap();

// Event Listeners
// ===============

const intersectionObserver = new IntersectionObserver(
	( entries, observer ) => {
		entries.forEach( ( { target } ) => {
			if ( !state.has( target ) ) {
				// TODO

				return;
			}

			const {
				funnelEntryToken,
				elementFriendlyName
			} = state.get( target );

			mw.eventLog.submitInteraction(
				STREAM_NAME,
				SCHEMA_ID,
				'impression',
				{
					action_source: 'ClickThroughRateInstrument',
					funnel_entry_token: funnelEntryToken,
					funnel_event_sequence_position: 1,
					element_friendly_name: elementFriendlyName
				}
			);

			observer.unobserve( target );
		} );
	},
	{
		threshold: 1
	}
);

document.addEventListener( 'click', ( { target } ) => {
	if ( state.has( target ) ) {
		const entry = state.get( target );

		++entry.elementClickCount;

		mw.eventLog.submitInteraction(
			STREAM_NAME,
			SCHEMA_ID,
			'click',
			{
				action_source: 'ClickThroughRateInstrument',
				funnel_entry_token: entry.funnelEntryToken,
				funnel_event_sequence_position: 2 + entry.elementClickCount,
				element_friendly_name: entry.friendlyName
			}
		);
	}
} );

// API
// ===

/**
 * An instrument that tracks impressions and clicks of a DOM element.
 *
 * ## Usage
 *
 * const { ClickThroughRateInstrument } = require( 'ext.wikimediaEvents.metricsPlatform' );
 *
 * const result = ClickThroughRateInstrument.start(
 *     '[data-pinnable-element-id="vector-main-menu"] .vector-pinnable-header-unpin-button',
 *     'pinnable-header.vector-main-menu.unpin'
 * );
 *
 * // A few moments later…
 *
 * ClickThroughRateInstrument.stop( result );
 * 
 *
 * ## Events
 *
 * ### Impression
 *
 * The `action=impression` event is submitted soon after the element is fully visible in the
 * viewport. The event is submitted once. The event has the following fields:
 *
 * | Field                          | Type      | Value(s)                       |
 * | ------------------------------ | --------- | ------------------------------ |
 * | action                         | string    | `"impression"`                 |
 * | action_source                  | string    | `"ClickThroughRateInstrument"` |
 * | funnel_entry_token             | Token     |                                |
 * | funnel_event_sequence_position | usmallint | `1`                            |
 * | element_friendly_name          | string    |                                |
 *
 * ### Click
 *
 * The `action=click` event is submitted when the user clicks the element. The event can be
 * submitted more than once. The event has the following fields:
 *
 * | Field                          | Type      | Value(s)                       |
 * | ------------------------------ | --------- | ------------------------------ |
 * | action                         | string    | `"click"`                      |
 * | action_source                  | string    | `"ClickThroughRateInstrument"` |
 * | funnel_entry_token             | Token     |                                |
 * | funnel_event_sequence_position | usmallint | `2`, `3`, `4`, etc.            |
 * | element_friendly_name          | string    |                                |
 *
 * ## Notes
 *
 * 1. The `action=click` event is submitted when the user clicks the element. The instrument
 *    detects this by listening to the [Element: click event][0], which occurs when:
 *
 *    > * a pointing-device button (such as a mouse's primary button) is both pressed and released
 *    >   while the pointer is located inside the element.
 *    > * a touch gesture is performed on the element
 *    > * the `Space` key or `Enter` key is pressed while the element is focused
 *
 * [0]: https://developer.mozilla.org/en-US/docs/Web/API/Element/click_event
 *
 * @class ClickThroughRateInstrument
 * @singleton
 */
const ClickThroughRate = {

	/**
	 * @param {string} selector
	 * @param {string} friendlyName
	 * @return {StateEntry|null}
	 */
	start( selector, friendlyName ) {
		const e = document.querySelector( selector );

		if ( !e ) {
			// TODO

			return null;
		}

		if ( state.has( e ) ) {
			// TODO

			return null;
		}

		const result = {
			selector,
			friendlyName,
			element: e,
			elementClickCount: 0,
			funnelEntryToken: mw.user.generateRandomSessionId()
		};

		state.set( e, result );
		intersectionObserver.observe( e );

		// Return a copy of the internal state so that it can't be modified by third-parties but
		// can still be used to stop the instrument tracking the element (see
		// ClickThroughRateInstrument#stop() below).
		return Object.assign( {}, result );
	},

	stop( { element } ) {
		intersectionObserver.unobserve( element );
		state.delete( element );
	}
};

module.exports = ClickThroughRate;

☝️ I've only just caught that I never posted those two proposals 😬 😓