Possible race condition when dataTagGetData from 2 rapidly fired data layer events

[hugentobler]

At the moment, when marking "add_data_layer" to true on the Data Tag, the following function is called in browser window to retrieve the latest dataLayer data. However, if one has 2 of the same event firing in rapid succession in the client, the Data Tag is always only able to grab the latest dataLayer data of the second event. Resulting in a race condition. Any suggestions on how to solve this? For now I must unpack the dataLayer data on the client-side and send it as an event or user property.

function dataTagGetData(containerId) {
    window.dataTagData = {
        document: {
            characterSet: window.document.characterSet,
        },
        innerWidth: window.innerWidth,
        innerHeight: window.innerHeight,
        screen: {
            width: window.screen.width,
            height: window.screen.height,
        },
        dataModel: window.google_tag_manager[containerId].dataLayer.get({
            split: function() { return []; }
        }),
    };

    return window.dataTagData;
}

[Bukashk0zzz]

Hi, Thanks for the issue.

I think a workaround for this should be in gtm.uniqueEventId.
Theoretically, you can use let gtmId = copyFromDataLayer('gtm.uniqueEventId') || 0;
and pass it to this function and receive a specified dataLayer push.

But this needs to be tested.

I added this bug to our backlog.
But we will be happy if you send us a PR with the fix.

[hugentobler]

Hi thanks, this is what I attempted and tested, but then I realised that it seems the only way to get the uniqueEventId is by using copyFromDataLayer, which still suffers from the race condition. Any ideas?

template.js

function addCommonDataForPostRequest(data, eventData) {
  const eventId = copyFromDataLayer('gtm.uniqueEventId');
  if (data.add_common || data.add_data_layer) {
    const dataTagData = callInWindow(
      'dataTagGetData',
      getContainerVersion()['containerId'],
      eventId
    );

    if (data.add_data_layer && dataTagData.dataModel) {
      if (dataTagData.eventMatched) {
        for (let dataKey in dataTagData.dataModel) {
          eventData[dataKey] = dataTagData.dataModel[dataKey];
        }
      }
    }

    ...

data-tag.js

function dataTagGetData(containerId, eventId) {
  var dataLayer = window.google_tag_manager[containerId].dataLayer;

  var dataModel = eventId
    ? dataLayer.get(eventId)
    : dataLayer.get({
      split: function() { return []; }
  });

  window.dataTagData = {
      document: {
          characterSet: window.document.characterSet,
      },
      innerWidth: window.innerWidth,
      innerHeight: window.innerHeight,
      screen: {
          width: window.screen.width,
          height: window.screen.height,
      },
      dataModel: dataModel,
      eventMatched: !!dataModel  // false if the event ID was not found
  };

  return window.dataTagData;
}

[Bukashk0zzz]

Sorry for now, no better ideas.
But we will discuss this issue internally with the team when we come to it.

Thanks for your input.

[kucherenko-ae]

Hi.
Just now I encountered the same issue.

the simplest idea that came to mind was to pass a slice of dataLayer to the data-tag at the moment of a trigger.

So, created custom JS variable:

function () {
  return google_tag_manager[{{Container ID}}].dataLayer.get({
    split: function() {
      return []
    }
  })
}

And updated Data Tag template:

1. added new field for this "slice"

2. changed the code a bit:

function addCommonDataForPostRequest(data, eventData) {
  if (data.add_common || data.add_data_layer) {
    const dataTagData = callInWindow(
      'dataTagGetData',
      getContainerVersion()['containerId']
    );

    if (data.add_data_layer) {
      const dlSlice = data.dl_slice || dataTagData.dataModel;
      
      if (dlSlice) {
        for (let dataKey in dlSlice) {
          eventData[dataKey] = dlSlice[dataKey];
        }
      }
    }

    if (data.add_common) {
      eventData = addCommonData(data, eventData);
      eventData.screen_resolution =
        dataTagData.screen.width + 'x' + dataTagData.screen.height;
      eventData.viewport_size =
        dataTagData.innerWidth + 'x' + dataTagData.innerHeight;
    }
  }
  if (data.add_consent_state) {
    eventData = addConsentStateData(eventData);
  }

  if (data.add_common_cookie) {
    eventData = addCommonCookie(eventData);
  }

  return eventData;
}

It may not be very elegant, but it worked

[giovaniortolani]

Hi @kucherenko-ae, thanks for your suggestion!

We have developed a fix but not shipped it yet to production. It's available on this PR #33.

We cannot implement your solution because it depends on this Custom JS variable, which would be impractical for everyone using this tag to implement. Therefore, we addressed it using minimal modifications that wouldn't require a bigger change on the user side.

[kucherenko-ae]

Hello @giovaniortolani
Sure — my solution will only be suitable for those specifically looking for one. Of course, it won’t fit most Data Tag users))

Btw, I saw PR33, I think there might be an issue too - for example, in cases when multiple components push something to dataLayer (just messages) and in the end - event:

1. eventBus emits some trigger, for example: user::click
2. component foo does some work (in middleware maybe? why not) and pushes data: dl.push({ some_foo_data: "foo_dynamic_data" }
3. component bar does its work too and pushes: dl.push({ some_bar_data: "bar_dynamic_data" }
4. after all middlewares and prior processors/handlers, an analytics module sees this trigger and fires event dl.push({ event: 'user:clicked' })
5. repeat

this is a rare case, of course - but the current solution in the PR, which extracts an event object, might still face the same race condition.

I think instead of returning currentEventObj you need to exclude all messages and events after a desired eventId (using slice() for example), then convert the dl array to an object (like the get() function of dl does) and return that.
there, I mean, instead this:

[giovaniortolani]

Hello @kucherenko-ae!

I'm not sure if I understood what is the issue described here

[...] for example, in cases when multiple components push something to dataLayer (just messages) and in the end - event: [...]

Could you elaborate more on that, please?

Are you referring to the currentEventObj not returning the data present on the messages?
If so, that's expected in the current implementation. It provides the actual event object that triggered the tag, useful when the dataModel is not correct / not synced, which happens when the tag fires and the data-tag.js script is not loaded yet.

I think instead of returning currentEventObj you need to exclude all messages and events after a desired eventId (using slice() for example), then convert the dl array to an object (like the get() function of dl does) and return that.
there, I mean, instead this:

That would be an option. However, the code would have to mimic exactly what the GTM internal code does when it creates the dataModel, which is the following (recursive merging). Otherwise, the returned object would have some small differences when it comes to merging objects and arrays.

Anyway, you gave a good idea. We'll consider implementing it in the currentEventObj, and taking care of all the edge case scenarios.

[kucherenko-ae]

I mean that even when retrieving only the event object by ID, you will still face race conditions with other messages in the dataLayer.
For example, consider the following dataLayer:

[
  { "data": "foo" },
  { "event": "user:click", "even_data": "one", "gtm.uniqueEventId": 1 },
  { "data": "bar" },
  { "event": "user:click", "event_data": "two", "gtm.uniqueEventId": 2 }
]

by current code in the PR 33:

• event_data will be correct, because it’s retrieved from the event object by ID.
• But for both events, data value will be the same (bar), since the dataModel still takes a slice of the dataLayer as before.

dataTagGetData("name", 1) -> { "event": "user:click", "event_data": "one", "data": "bar" }
dataTagGetData("name", 1) -> { "event": "user:click", "event_data": "two", "data": "bar" }

yes, I think you understand me correctly:

Are you referring to the currentEventObj not returning the data present on the messages?

based on this, I suggested to make a slice of dataLayer yourself, then currentEventObj is not needed at all and just return your slice as dataModel.

[kucherenko-ae]

by the way, found custom variable in gtm, that I've created for a long time ago, it should repeat (+-) dL logic:

I can't say why and for I've created it.
now I see difference in some technical data, like missed developer_id etc. (I don't think they are important for use on the server side, or you can define them and take them from the main slice (via get()))
I'll just leave this here, maybe it will be useful to you. ))

UPD: also added merge for arrays (like dl does) and left only updated version:

function getDlSlice (containerId) {
  var propsForExclude = ['event', 'gtm.start', 'gtm.triggers', 'gtm.uniqueEventId'];
  var dlName = google_tag_manager[containerId].dataLayer.name;
  var dl = window[dlName];

  function getType (target) {
    return Object.prototype.toString.call(target).slice(8, -1).toLowerCase();
  }
  
  function deepMergeDataLayerEntry (target, source) {
    var targetType = getType(target);
    var sourceType = getType(source);

    if (targetType === 'object' && sourceType === 'object') {
      return Object.entries(source).reduce(function (result, entry) {
        result[entry[0]] = deepMergeDataLayerEntry(result[entry[0]], entry[1]);
        return result;
      }, Object.assign({}, target));
    }

    if (targetType === 'array' && sourceType === 'array') {
      var maxLength = Math.max(target.length, source.length);
      var newArr = [];

      for (var i = 0; i < maxLength; i++) {
        if (source.hasOwnProperty(i)) {
          newArr[i] = (deepMergeDataLayerEntry(target[i], source[i]));
        } else {
          newArr[i] = (target[i]);
        }
      }

      return newArr;
    }

    return source;
  }
  
  var dlSlice = dl.reduce(function (acc, entry) {
    if (getType(entry) === 'object') {
      return deepMergeDataLayerEntry(acc, entry);
    }
    return acc;
  }, {})

  propsForExclude.forEach(function (key) {
    delete dlSlice[key];
  })

  return dlSlice;
}

[giovaniortolani]

Hi @kucherenko-ae,

We implemented a simplified version of the dataModel builder (which your function getDlSlice returns).

It's currently being tested and it may become official. If you want to take a look, the changes are in the PR #41.

[kucherenko-ae]

Hello @giovaniortolani
It so happened that I'd already partially modified your GTM template and rewritten the data tag helper.
We were losing events when switching between pages, so I added an event queue with storage, and also added support for batch sending and other things.

That's a whole other story, of course.
The interesting thing is that I never migrated the dataLayer slice logic from GTM to the helper ))

Thanks for the update, I'll probably partially use your dataLayer slice solution.