Real-Time Analytics in Power BI

Analytics systems working in near- or real-time can deliver insights based on what is going on right now, giving companies an edge over those making decisions based on past events and outdated information. Power BI was not initially designed to handle real-time data (like most traditional BI tools). Companies utilizing these tools rely on constant data refreshes via a data gateway, which is adequate for some daily reports and dashboards. But as streaming data becomes more popular and 24×7 customer expectations increase, refreshing is not enough. To meet evolving business demands, Power BI offers two options for real-time reports and dashboards.

Real-time Decision Making is Vital to Your Business

You may have made the investment in a data and analytics tool that turns data into insights. You are able to visually explore and analyze your data in a single platform. While this is crucial to fostering a data-driven culture, if you aren’t making decisions based on real-time data, you will fall short of your competition. For example:

• Consider customer service – when customers contact support, they expect agents to have real-time data at their fingertips so that issues can be resolved during a single session or call. Additionally, support center managers should be able to refer to a real-time call queue to see where bottlenecks in wait times are occurring.
• Operational efficiency requires real-time data. Companies in distribution need to know real-time updates regarding shipments and inventory, so that corrective action can be taken. An e-commerce company during holiday season needs real-time data to ensure orders are processed and delivered on time.
• Employees can be motivated by real-time data. When employees can see how they are performing against their peers, they are motivated to take quicker corrective action to look better on the dashboard.

No industry is exempt from the benefits of real-time decision making. Even if your data isn’t time sensitive, making decisions on real-time data allows you to make quicker, more proactive decisions than your competitors. Let’s discuss how to achieve this in Power BI.

Power BI Features for Real-time Analysis

The longstanding method of refreshing data in Power BI was to use an on-premises data gateway, which acts as a passthrough for data to go from the source into the Power BI service. Power BI Pro users can schedule up to 8 refreshes per day, and up to 48 refreshes with Premium. That means even with Power BI Premium, scheduled refreshes can occur at most every 10 minutes in an 8 hour workday. However, there are two ways of achieving real-time analysis in Power BI: Automatic Page Refresh and Streaming.

Automatic Page Refresh

A client needed to display near-real time data coming from their manufacturing machines on monitors throughout their shop floor. Streaming datasets in Power BI was not a suitable option, because the visuals are too limited and building the APIs would take too much time. Instead, we recommended Automatic Page Refresh, because it easily connects to their existing database and setup can be done in hours. We can also leverage all the visuals and features of Power BI that can’t be accessed with streaming datasets.

How Automatic Page Refresh Works

Introduced in October 2019, Automatic Page Refresh is a preview feature that allows for the full customization of Power BI Desktop while also having data refreshed as frequently as one second. Reports in Power BI Desktop only need to enable this feature via the Preview Feature option and import data via DirectQuery (Import and LiveConnect are not supported yet). Each individual page in a report can have Automatic Refresh enabled or disabled, and the refresh intervals can vary between pages.

Automatic page refresh requires additional settings for use in the Power BI service. Reports can be published with Automatic Page Refresh enabled, but they will not refresh unless these settings are properly configured:

• The dataset must be connected to an on-premises data gateway
• The report must be published to a workspace with a Premium or Embedded capacity (denoted by a diamond icon)
  • Within the capacity settings under Workloads, Automatic Page Refresh must be enabled. The refresh interval should match the value specified in Power BI Desktop:

• Capacities can be created and managed in Azure. These capacities incur charges while they are running, regardless if they are being actively used by a workspace.
• Gateway refreshes do not need to be enabled. The data will update using the Automatic Page Refresh.

When choosing a refresh interval, it is important to note how many visuals and concurrent users will be using the report. Each user and each visual on the page is an additional query down to the source system. Refreshing every second may be sub-optimal depending on the complexity of the query and the hardware serving the data source. To better optimize Power BI’s performance, check out the Performance Analyzer feature to minimize resource-exhaustive queries.

To explore this feature in Power BI Desktop, complete the following:

• Download Power BI Desktop and enable Automatic Page Refresh in the Preview Features
• Connect to a data source that is frequently updated

Each visual will show a wheel spinning in the top left when it is refreshing. To explore how this works in the Power BI service:

• Sign up for an Azure account and create a Power BI Embedded Capacity
• In the Power BI service, create a workspace using the Embedded Capacity
  • Set the capacity’s minimum refresh interval to match the refresh in Power BI Desktop
• Publish a report to the workspace
  • The report must have Automatic Page Refresh enabled

Streaming Data

Introduced in 2016, streaming data sets can come from an Azure data stream, PubNub, or REST API, and they offer true real-time data. To take advantage of this feature, first create a streaming dataset in the Power BI service. Then specify the name of the dataset and the values received from the data stream. As of now, the only datatypes supported are Text, Number, and DateTime. Once the dataset is configured, the Power BI service will create a Push URL. The Push URL is where your data will be sent in the Power BI service. This means you bypass Power BI Desktop entirely.

A drawback to this approach is that visuals available for streaming data are limited to card, line chart, clustered bar chart, clustered column chart, and gauge, and they cannot be customized like they can in Power BI Desktop. Also, the dataset cannot have nested JSON objects; Power BI can only parse through 1 level of a JSON object. However, you can have multiple streaming datasets in a single dashboard to work around this.

Need help with software evaluation and selection?

With the analytics industry changing at such a rapid pace, we know it’s a challenge to keep up with new features and technologies. As data and analytics consultants, we have experience with a wide range of BI products and can help you cut through the noise to determine which tools can solve the problems you’re facing.