At Curalate, we need to be able to use data to demonstrate that our products hold value for our clients. One of our products, Fanreel, uses user-generated content to enhance online shopping experiences and product discovery. We record and store usage metrics from Fanreel but we also need to take those usage metrics and connect them to product purchases. If Fanreel analytics were a puzzle, purchase information would be the last piece and historically, Google Analytics or Adobe Omniture served as this last piece. However, every ecommerce site is different so sometimes the intricacies of Google Analytics and Adobe Omniture got in the way.
We wanted to have a simple “one size fits all” solution so we have turned to a simple tracking pixel. Specifically, our first tracking pixel is a checkout pixel which lives on our clients’ checkout confirmation pages to collect transaction data and serve as the last piece of our analytics puzzle.
What Is a Tracking Pixel?
A tracking pixel is a 1x1 transparent image that sends data from the webpage the pixel lives on. When the page loads, a GET request is made for the image along with query parameters that contain user data.
This tracking pixel is sending information from a pixel with id 12345 about a user named Shippy who is works at Curalate and whose position is engineer.
- a servlet to receive the tracked data and return the tracking pixel image
- a way to stream/store the data
We solve step 3 with AWS Kinesis Firehose. We like Kinesis Firehose because it’s easily configurable and because it fits nicely into our existing data pipeline which uses AWS Redshift extensively as a data store.
Let’s go through how you can set up a tracking pixel and AWS Kinesis Firehose to collect some information on our friend Shippy, the engineer who works at Curalate:
You can minify this file and put it in the head tag of whatever pages you want your tracking pixel to live on.
As for Your Pixel JS Library File…
This file defines all of the functions needed to gather the data you’re interested in and to generate a request for the pixel that will send the data to your server. It will also pull events off of the global queue and execute them. This library should provide functions so that whoever is placing your pixel on their website can use these functions to include exactly the data you want the pixel to collect.
Step 2: Set Up a Servlet
The servlet ties everything together.
- receive the data sent by the tracking pixel
- return a 1x1 transparent image to the page that requested the pixel
- send the data to your Firehose delivery stream
Step 3: Set Up AWS Kinesis Firehose
Now that you have all of this data collected by your tracking pixel, how do you store it? At Curalate, we’ve turned to AWS Kinesis Firehose. Firehose is specifically designed to provide an easy and seamless way to capture and load data into AWS, not to mention that setup literally consists of clicks on the AWS console. Firehose is also great since since data streamed to a Firehose delivery stream can ultimately land in ElasticSearch, S3, or Redshift.
Since we love Redshift and already use it very heavily, that’s our chosen destination for our checkout pixel. The AWS setup documentation is quite thorough but here are a few tips that we picked up from our setup for our checkout pixel:
- Adjust the buffer size and buffer interval of your Firehose delivery stream to control your throughput. If you have a lot of data, consider reducing the buffer size and interval to get quicker updates into your final AWS destination.
And That’s It!