In this post, we will explore the implementation of Retail Calendars in Looker and delve into the advantages they offer. We will focus on how to leverage these calendars to build scalable Period Over Period (PoP) comparison functionalities. This includes importing retail calendar data into a Looker view, extending the view, and creating new dimensions to enable seamless period-based analysis. We will also cover how to join these calendars to explores, ensuring flexibility in reporting. Additionally, we will walk through implementation details with code snippets and provide comparison examples to illustrate the impact of these techniques.

By the end of this blog, you will have a robust, user-friendly solution to perform powerful period comparisons, such as analyzing a Retail Month against the same month in the previous year.

A Retail Calendar is a specialized calendar used in the retail industry to track sales and inventory over specific periods, often aligning with seasonal trends and business cycles. Unlike a standard calendar, retail calendars such as the 4-5-4 calendar are structured to maintain consistent weeks for accurate year-over-year (YoY) comparisons. For a detailed overview, the NRF website provides an excellent explanation.

Looker uses a standard calendar by default, based on the Gregorian system. You can customize the starting day of the week (e.g., Sunday or Monday) by including week_start_day: sunday in the model configuration. While this works well for general reporting, it falls short when comparing periods with varying numbers of days or inconsistent week structures.

To implement a retail calendar:

• Create a View: Start by creating a view containing retail calendar data. This data can be sourced from:
  • A Google Sheet with updated dates.
  • SQL queries to generate the necessary calendar structure.
• Note that different businesses may adopt other variations, such as a 4-4-5 calendar. One such example can be found here : Google Sheet Link
• Join the View: Join the retail calendar view on the date field with the date field of any other view where needed. This ensures the retail calendar’s alignment with other datasets in your model.

The primary advantage of using a retail calendar lies in its ability to compare identical periods containing the same number of days, weekends, and holidays. For instance, comparing January of this year with January of last year using the Gregorian calendar can lead to discrepancies due to varying week structures. A retail calendar resolves this issue, ensuring accurate YoY comparisons.
If you haven’t already, we recommend checking out our foundational approach to building Period Over Period capabilities in Looker, as this blog builds upon that methodology. Our approach overcomes some of the limitations of Looker’s default setup, especially in terms of scalability.

For a recent project, we implemented a 4-4-5 retail calendar to meet client-specific needs. Here’s how:

​Join the PoP and retail calendar views to any required explore. For example, if the explore is named daily_sales, the join relationship will depend on the nature of the daily_sales table.

Use these joins to add retail period comparisons to PoP analysis. Examples include:

• This Retail Month vs. Same Retail Month Last Year.
• This Retail Quarter vs. Same Retail Quarter Last Year.

The below screenshots show how all the dimensions work with and without to date.

By integrating a retail calendar into Looker, you can significantly enhance your ability to perform accurate period comparisons. This approach not only ensures consistency but also scales seamlessly across multiple datasets and explores. Whether you’re working with a 4-5-4 or 4-4-5 calendar, this solution empowers you to make data-driven decisions with precision and ease.
Ready to implement this in your Looker instance? Dive in and see how simple it can be to create powerful, scalable period-over-period analyses!​

​Understanding Period Over Period Comparisons
Before diving into the implementation, let’s quickly explore why Period Over Period comparisons are so useful. When measuring performance, it’s critical to compare current performance against previous periods. Some common comparisons include:

• This year vs. last year
• This month vs. last month
• This quarter vs. same quarter last year

PoP comparisons help you identify trends and patterns, enabling better business decision-making. For instance, you might want to track:

• How your business performed this year compared to last year
• Whether you hit your sales targets for this quarter
• Business performance for specific periods like Cyber Weekend

Limitations of Looker’s Built-in Methods
​While Looker’s native methods work well, there are some limitations:

• Hard to scale. The logic can only be used within a single view
• Predefined periods may not suit all use cases.
• Filtering options are limited, and may not allow for custom periods.
• Arbitrary Period comparisons are often restricted to days, limiting flexibility.

Because of these limitations, we at Nupanch developed a customized approach tailored to the needs of our clients.

Nupanch’s Approach to PoP Analysis in Looker
Our approach simplifies Period Over Period analysis by creating predefined "analysis periods" and "breakdowns." This method uses one parameter, one filter, and several dimensions that help filter dates according to the selected parameter. Here’s how it works:

1. This Week vs Same Week Last Year (WoWLY): Compares the current week with the same week last year, with a filter to adjust for incomplete weeks.
2. This Month vs Same Month Last Year (MoMLY): Compares the current month with the same month last year, adjusting for incomplete months.
3. This Year vs Last Year (YoYLY): Compares the current year to the previous year, with adjustments for incomplete years.
4. Cyber 5 This Year vs Cyber 5 Last Year: Compares Cyber 5 periods from two consecutive years. For example, to compare 2023 vs. 2022, we would look at the respective date ranges (Nov 23-28, 2023 vs Nov 24-29, 2022). The filter subtracts 364 days to create accurate comparisons.

Furthermore, by using a date dimension table we are able to scale the period-over-period capability to several explores.

As seen above with the To-Date filter you can limit the previous period to the days in the current period and compare apples to apples.

Appendix: Key Code Snippets
Here are some handy snippets for reference when setting up your analysis:
Current_date : ​

Current_day : ​

That said, there are ways to get some idea of how a new product will sell. Instead of directly looking for comparable products, information about product life cycles can be used.

Newly launched products are likely to have a life cycle similar to at least some products launched in the past couple of years. With this in mind, we can input product life cycle information to a learning algorithm to create "clusters" of products with similar sales patterns. Once we have the clusters defined, we identify commonalities within each cluster. Then, when a new product is launched, we only need to identify the cluster in which it falls to get a sales forecast.

Here are the steps you will want to follow -

1. Identify the newly launched products in the past couple of years.
2. For each of these products, obtain its number of units sold in months 1 through 12.
3. Transform the data points to those that represent the life cycle -
   • 3 data points for the number of units sold in months 1-3.
   • 3 data points for average units sold per month from Q2 (second quarter), Q3 and Q4.
   • 3 data points for the ratio of units sold in Q2/Q1, Q3/Q2 and Q4/Q3.
4. For each cluster output from the clustering algorithm, identify characteristics that define the cluster. Some examples of characteristics are -
   • Brand/manufacturer of the product
   • Top-level and lower-level categories of products
   • Price range of product
5. When a new product is being launched, identify the cluster in which it will fall using the cluster characteristics defined in step 4.
6. That cluster's sales pattern will likely apply to the newly launched product.

​
Note: There is always a margin of error in forecasts, so this should always be supplemented with other qualitative information about the new product.

Need help setting up the above steps? Reach out here.

    1) Use The Right Visualization

The importance of accurate data presentation cannot be stressed enough. The wrong visualization can obscure insights. Here's a basic guide to using the right chart types -

• Identifying correlations between variables - scatter plots
• Showing a trend over time - line charts
• Showing distribution of a single variable - histograms
• Comparing variables - column charts and bar graphs

Additionally, a table of data can also be the right presentation format. When the result is less than 4X4 cells, it is easy for the eye to interpret it.

    2) 5 Whys Approach

The trick to getting the data to talk is to ask a ton of questions. You won’t be able to generate a ton of questions right from the beginning, though, and that’s normal. The idea is to let the answers to your questions beget more questions.

Find the smallest indicator you can observe from your initial set of reports, and ask why that indicator might exist. The moment you ask why, your brain generates guesses and opinions. Do this 5 times, and you’ll be drowning in ideas for reports.

Let’s continue with the example of analyzing customer churn -

• Let’s say you observe that the group of customers that has interacted with your product/service in the past has a higher renewal rate. That’s great, but not enough to be actionable. You obviously don’t want to discourage new customers from subscribing to this product.
• What about their past experience might have caused them to renew?
• You build some more investigative reports to dig into their previous activity. It turns out a significant percentage had earlier signed up for a newsletter. Hmm.
• You now have completely new direction for the analysis. What was the newsletter about? How is it related to the product you’re analyzing churn for? A litany of questions crosses your mind and you can’t wait to get these answers!

The above example is a bit too optimistic, but you can see how digging deeper into a seemingly small observation can lead to actionable insights.

    3) Invite Opinions

We did this exercise with one of our clients recently. We built out one iteration of reports, presented its findings to the broader team, and invited their questions and thoughts. It was incredible - being experts in their industry, they were able to provide context to raw numbers. We were easily able to distinguish findings that mattered from the ones that were the result of a one-off experiment. This exercise gave us clear next steps for the second round of reports.

Do you have other ideas and tips for getting answers from data? We’d love to hear them in the comments below.

Here are some ways in which calculating residual lifetime value can be useful for your business -

• Define your customers not only by what they have spent so far, but by how much they are likely to spend going forward
• Identify & understand the traits associated with your best and worst customers
• Improve acquisition decision-making
• Sales / demand forecasting
• Customer service queuing
• Proxy for a company’s value

The most popular of these statistical models is the Pareto/NBD model. Here is the paper that sets the foundation for it. It’s full of math, so here’s the short version -
​
The model takes as input a historical purchase log, and calculates basic RFM (recency, frequency, monetary) stats. It then spits out the following data points for each customer -

• PAlive - the probability (between 0-1) that the customer is “alive” i.e. still a customer of your business.
• Expected Transactions - the predicted number of transactions the customer is likely to make in the next X months. X is defined by you and depends on how long you’d like to predict for. As a rule of thumb this should not be greater than the duration for which the model has been trained.
• Average Order Value - the predicted average transaction value of the customer going forward.

What you will need to predict CLV

1. Historical Transaction Log - A list of orders with the customer_id, date and amount of the order. This list should include orders made by customers acquired between 2 years ago and 6 months ago. For example, if today was Jan 1, 2018, the customers to be included would be the ones acquired between Jan 1, 2016 and June 30, 2017. It is okay to include customers older than 2 years (in fact, the more data the better), but not newer than 6 months ago.
2. R Script - The code to calculate the predictions for each customer. The "BTYD" R package is perfect for this purpose. Here is a walkthrough of the code.
3. Export to BI Tool - Ideally, you will want a way to automate the above steps 1-2 times a month. If you are using a BI tool, use API access to set up an automated export/import process.

Need help setting this up? Feel free to reach out here.

Note: This method is suitable for non-subscription businesses only. For subscription businesses, there are a host of other methods that can be used. Another post to follow shortly!

Having worked as an analyst with RJMetrics and then Magento for a combined 3 years in the city of brotherly love, I’ve worked with over 200 small to medium sized e-commerce customers, advising them on best practices in using the product, building reports, as well as maintaining an optimized data warehouse. I now work independently, consulting with merchants to make the most of their reporting and analytics efforts. It only took me 3 years to realize that there should be content about the most common questions that customers asked about (and sometimes dreaded) when using Magento Business Intelligence. Read on!

What’s a replication method, and which one should we use?

For a Magento database, most tables have an auto-incrementing numeric primary key column called “entity_id”. Hence, using the “Auto-Incrementing Primary Key” replication method is the recommended option for most tables. For some other tables that have a combination of two or more columns as the primary key, like “catalog_category_product”, the “Primary Key Batch Monitoring” option is best. That said, if the “updated_at” column on the database tables is set up correctly - i.e. it is a timestamp that is automatically updated when a row is updated - the “Modified At” replication method is the best option.

Is there a difference in the direction of setting up paths - one-to-many or many-to-one?

Yes. When setting up paths between tables, think about the two tables you are connecting. For example, let’s say you are connecting the “orders” and “line items” tables. Ask yourself, “Does a single order have multiple line items, and does a line item belong to multiple orders?” In most cases, the answer to the first question is “yes”, and the second is “no”. Hence, in the case of these tables, the “orders” table is the “one” side of the relationship while the “line items” table is the “many” side.

​
​Consequently, when bringing a column from the “orders” table to the “line items” table, it can be brought over as a “Joined Column”. All information about an order is also valid for all its associated line items. On the other hand, a column cannot be brought from “line items” to “orders” using a “Joined Column” because an order could have more than one distinct associated line item. So, when bringing data from “line items” to “orders”, an aggregation function - such as sum, average, max, min etc. - needs to be performed.

The concept is illustrated visually below.






















​​In the above example, let’s consider two scenarios -

1. Bringing over “order_date” from “orders” to “line_items” - To do this, we can link the two tables on the order_id column, and create a new “Joined Column” in “line_items” called “order_date”. This column will contain the same value for each line item of an order. For example, the rows with “item_id” 1000 and 1001 will have the order_date Jan 1, 2017.
2. Bringing over “quantity” from “line_items” to “orders” - Since each order has more than one associated line item, “quantity” will need to be summed up from the “line_items” table and then brought over to “orders”. To do this, choose the “Many-to-one” type of column, and then choose the “Sum” operation. For example, the newly created “quantity” column in “orders” table will contain a value of 6 for the order with order_id 3.

Which table should metrics be built on?

The short answer is that metrics should be built on the lowest “grain”. “grain” refers to the granularity of data in the table. The “orders” table contains information about each order - its total amount, date, customer who purchased the order. However, it does not contain information about the line items in the order. This information lives in the “line items” table. Hence, “line items” is at a lower granularity than the “orders” table i.e. it contains all information that the “orders” table contains and then some additional information.

Hence, everything that can be built on the “orders” table can also be built on the “line items” table. Further, the “line items” table allows you to segment your data to a greater degree than the “orders” table due to the presence of more granular data in it.

How should many-to-many relationships be analyzed?

Magento BI does not have an inbuilt way to handle many-to-many relationships. To analyze data from tables sharing a many-to-many relationship, the recommended method is to use SQL - either in the SQL Report Builder or in a Data Warehouse View. The two tables will need to be joined, with each joined combination becoming a new row. For example, let’s consider the case of each product being in multiple categories, and each category having multiple products. If there were a total of 4 products, each belonging to 3 different categories, the total number of combinations would be 4X3 = 12 rows. Once this join has been performed, either in the SQL Report Builder or as a Persistent View, information about these combinations can be added to these reports/views.

These are some of the most common questions customers of Magento BI have about how to best use the product. With MBI evolving quickly, I’m sure there will be many more questions that haven’t been covered above. Have other questions about best practices or looking for advice? Leave a comment or reach out here.