Configuring Splunk for Video Analytics with Datazoom
Visualize real-time, video data to extract deep insights
Splunk Cloud captures, indexes, and correlates real-time data in a searchable repository from which it can generate graphs, reports, alerts, dashboards, and visualizations. When integrated with Datazoom’s real-time, video data infrastructure, the combination becomes a powerful pair for visualizing video streaming analytics.
In this article, we will review the process for setting up Splunk Cloud + Datazoom at a high level. Whether you’re an existing Splunk Cloud user interested in expanding your current use of the platform to include video analytics or a prospective Splunk user, it’s easy to get started – click here and sign up for a Datazoom account (complete with free access + 5 GB of data for your first 15 days). If you’re totally new to Splunk Cloud (and Datazoom) but also interested in seeing this combination in action, you can sign up for Splunk’s own 15 day-trial here. Both trials offer the full functionality of Datazoom and Splunk Cloud, respectively.
Before we get started…
Please review the following prerequisite steps necessary before you can visualize video data in Splunk Cloud. As always, our Help Center is a valuable resource for additional information along the way.
Prerequisite 1: It’s important to have created a Collector for your video player and have its configuration deployed before beginning.
Prerequisite 2: It is equally important to have created a Connector for Splunk Cloud and have it as part of your active Data Pipe with “timeSinceLastFluxData” and “Play Request” selected and enabled prior to proceeding.
Building Metrics
Splunk has its own query language to write the expressions which are used to calculate metrics. We’ll review four now.
Creating a metric for the Number of Minutes Watched:
1) Login to your Splunk Cloud instance and navigate to Search.
2) Copy and paste the following into Search
index="<your configured index>"
| stats sum(event.metrics.timeSinceLastFluxData) AS timeWatched
| eval timeWatched=timeWatched /1000/603) Select the time period you’d like to search against in the right side of the “Search” tab found in the upper-left of the Splunk UI, then click the hourglass icon.
4) Select the Visualization tab from the results window and then format as necessary. We changed the result’s precision to two decimal places and added the unit “Minutes” after the result.
5) Select “Save As” under the “Search and Reporting” text in the upper right to save the search as a Dashboard Panel.
6) Select “New” then enter a name for your new dashboard next to “Dashboard Title” then click “Save.”
Creating a metrics for Number of Play Requests:
1) Copy and paste the following into Search:
index="<your index here>"
| spath "event.type"
| search "event.type"=Play_Request
| stats count2) Select the time period you’d like to search against in the right side of Search then click the hourglass icon.
3) Select the Visualization tab from the results window. Format as necessary. We added the unit “Play Requests” after the result.
4) Select “Save As” under the “Search and Reporting” text in the upper right to save the search as a Dashboard Panel.
5) Select “Existing” then select the name of the dashboard you created previously.
Creating a metric for the Number of Play Starts:
1) Copy and paste the following into Search:
index="<your configured index>"
| spath "event.type"
| search "event.type"=First_Frame
| stats count2) Select the time period you’d like to search against in the right side of Search then click the hourglass icon.
3) Select the Visualization tab from the results window. Format as necessary. We added the unit “Play Starts” after the result.
4) Select “Save As” under the “Search and Reporting” text in the upper right to save the search as a Dashboard Panel.
5) Select “Existing” then select the name of the dashboard you created previously.
Creating a metric to visualize User Location(s):
1) Copy and paste the following into Search:
index="<your configured index>"
| spath "user_details.session_id"
| search "user_details.session_id">"0"
| dedup user_details.session_id
| geostats count latfield=geo_location.latitude longfield=geo_location.longitude2) Select the time period you’d like to search against in the right side of Search then click the hourglass icon.
3) Select the Visualization tab from the results window. Format as necessary.
4) Select “Save As” under the “Search and Reporting” text in the upper right to save the search as a Dashboard Panel.
5) Select “Existing” then select the name of the dashboard you created previously.
Editing the Dashboard
Once you’ve created and saved your searches, you can click on the Dashboard tab and navigate to the dashboard you created earlier. From here you can edit the dashboard further and adjust as necessary.
The Advantages of Custom Metric Design
Following these steps will yield a basic dashboard for tracking some simple video statistics. At Datazoom, we’re currently amassing a database of sample queries you can use as a starting point to building more sophisticated dashboards and reports.
Using customized queries such as Splunk’s, Datadog’s, and New Relic’s NRQL provides users with the ability to obtain more objective metrics tailored to their team’s specifications. Today, some of the nicest, most convivial video analytics vendors use unobjective formulas to compute metrics which do not align when compared against one another. While out of the box analytics packages are convenient, it is high time the industry pivot away from one-size-fits all approached to online video.
Optimizing GA Dashboards to Visualize Video Insights
When paired with a real-time source of standardized, video data like Datazoom, Google Analytics transforms from a robust, customer data analytics tool into a cost-free alternative to other solutions for video QoE analytics.
Some media networks are making the change. You can too by following the high-level plan for getting set up with Google Analytics and Datazoom which you can read here.
This time, we’re going to introduce Google Data Studio as a tool for expanding upon the metrics computed by Google Analytics. Google Data Studio provides a platform for customizing the way matrics are calculated in more detail than Google Analytics’ more “out of the box” interface. Pairing a Datazoom + Google Analytics configuration with Google Data Studio yields a query function similar to systems like Splunk, New Relic Insights, and Datadog, which richer, more user-centric metrics can be designed.
Here, we’ll review some steps you can follow to get set up with a couple of basic metrics. You can use the same general ideas to write other useful metrics for your team.
Before we begin, please review and complete our Google Analytics Connector setup steps. This will ensure that you can take advantage of the full functionality of Datazoom’s free trial while utilizing Google Data Studio to understand your data. As well, click here to review some of the important differences and limitations between Google Analytics and Google Data Studio’s free and paid versions.
1. Link Google Analytics with Google Data Studio
Prerequisite: You will require a Google Analytics account and have deployed the requisite Global Site Tag (gtag.js) as per Google’s instructions before you begin.
Follow the steps outlined in this Connect to Google Analytics document. Once you’ve successfully completed the steps, you will see the following screen:
Click the “Refresh Fields” icon on the bottom left of your Data Source page in case there are any new custom dimensions or metrics that need to be included.
2. Creating the Metrics
Prerequisite: You will require a Collector for your video player and have its configuration deployed before you begin.
Prerequisite: You will require a Data Pipe with at least “timeSinceLastFluxData” and “Play Request” selected and enabled before you begin.
Create a metric for the number of minutes watched
Click the + next to “ADD A FIELD”
Enter Minutes Viewed in the upper left under “Field Name”
Enter timeSinceLastFluxData/60000
Click “Save” in the lower right
Create a metric for the number of play requests
Click the + next to “ADD A FIELD”
Enter Play Requests in the upper left under “Field Name”
Enter COUNT_Distinct(CASE WHEN Event Action = “Play_Request” THEN sessionViewId ELSE “” END)
Click “Save” in the lower right
3. Building the Report
Now that we’ve created two metrics, put them to use in a Google Data Studio Report. Click the “CREATE REPORT” button. You will see the following:
Click “ADD TO REPORT”. This will bring up a new report screen. From here you can begin to build visualizations using the metrics just created. Be sure to give your report a name so you can find it easily later. Click “Untitled Report” in the upper left to rename the report:
Click on the “Add a Chart” item in the toolbar and select the left icon under “Table:”
Enter “title” in the search field in the far right under “Available Fields”. Click and drag the resulting “title” onto the green dimension “Medium” to replace it:
Enter “Minutes Viewed” in the search field in the far right under “Available Fields”. Click and drag the resulting “Minutes Viewed” onto the blue metric “Pageviews” to replace it:
Click on the “Add a Chart” item in the toolbar and select the left icon under “Scorecard:”
Enter “Play Requests” in the search field in the far right under “Available Fields”. Click and drag the resulting “Play Requests” onto the blue metric “Pageviews” to replace it:
Continue using the tools in Data Studio to add a Date Filter and begin adjusting the size and layout of the report to your liking.
Click the “VIEW” button in the upper right to be taken to the report.
You can now view the number of minutes viewed by Title and the number of Play Requests in that time period:
Going Beyond
Below are additional examples of the types of reports you can build using the free trial Datazoom account and the free version of Google Analytics with Google Data Studio.
For a bit more detailed overview of these steps, and other resources for using Datazoom in tandem with other analytics systems like Splunk, New Relic, and Datadog, check out our documentation at the Datazoom Help Center.
With these resources, a world of insight is now at your fingertips:
Happy analyzing!
Configuring Google Analytics for video with Datazoom
For those at the beginning of the journey to becoming data-driven, Google Analytics (GA) provides a solid foundation for familiarizing a team with an analytics platform. Traditionally used by e-commerce outlets looking to understand behavior on a website, GA provides real-time feeds, behavioral data, campaign performance tracking and conversion pipelines. Up to certain usage limits*, Google Analytics can be a powerful, cost-free and relatively easy to use tool for video analytics as well. When paired with Datazoom’s video data infrastructure platform, real-time, standardized video data can be routed from the player and into your dashboards. Here, we’ll lay out the five steps for getting started.
These metrics and pre-configured reports provide quicker access to new insights, resulting in, hopefully, fully optimized marketing campaigns driving more revenue.
Google Analytics offers two models (You can check them side by side here)
Standard – *no charge up to 500,000 sessions and 20 custom dimensions. More information here.
Premium “360” – additional charge
Before we proceed, it’s important that you have a Google Analytics account with tracking script configured on one or more webpages. For help, check out the Google Analytics Academy or this documentation here.
1) Set-Up a Google Analytics connector in Datazoom
First, go to app.datazoom.io and either sign-in or create a free Datazoom trial account. Then build your Connector according to this Tutorial or by following the basic steps we lay out here (at our Help Center).
For Google Analytics specifically, you’ll require the Tracking ID associated with your account as well as the Custom Dimension-Metric Index Mapping configuration as detailed here.
2) Build Data Pipe
With Connector in hand, we can now link the Collectors associated with your video distribution components with Google Analytics with the help of a Data Pipe. For help setting one up, check out our tutorial videos.
3) Play Video
Soon after you start playing video, you’ll see real-time, raw video data being captured by Datazoom making its way into Google Analytics under the Realtime/Events section.
4) Basic Metrics.
Once you see data in the Real-time screen, you can begin to build visualizations and reports with the included tools under the “Customization” section of the main Google Analytics screen.
Some examples of the widgets illustrated above that you can build are illustrated below. These “bread-and-butter” widgets are useful starting points:
A. Sessions by Asset Title
B. Sessions by ASN Organization
C. Percent of New Sessions by streamingType
5) Improve metrics in Google Data Studio (next article).
While Google Analytics provides a springboard for getting started with analytics systems in general, it is, after all, designed for e-commerce and not video. In our next article, we’ll dive into how Google Data Studio can make the insights made possible by Google Analytics configured with Datazoom even more successful for managing your video content.
At this point, you’re ready to get started configuring useful widgets to power your video analytics strategy with Google Analytics. Pat yourself on the back once you come to your senses (maximizing the usability of a free tool for a task it wasn’t even intended for can be a little thrilling). Check out a real-world use case here!
But we’re not done yet.
In Part 2, we’ll enhance the usability of the metrics computed by Google Analytics by using Google Data Studio (another free tool) to build crisp, easy to use dashboards to make these metrics even more accessible.
There’s no time like the present to bring clarity to your video data strategies! Maximize the potential of Google Analytics for video when you get started by signing up for your free trial of Datazoom here.