Progress.DataSourceProgress

In this article, we will explore the M code behind the Progress.DataSourceProgress function, which is used to monitor the progress of data loading operations in Power Query.

What is Progress.DataSourceProgress?

Progress.DataSourceProgress is a Power Query M function that returns a record with information about the progress of a data loading operation. This function is typically used within custom functions or scripts to monitor the progress of a data load and to provide feedback to the user.

The record returned by Progress.DataSourceProgress contains the following fields:

– CurrentByteCount: The number of bytes that have been loaded so far.

– TotalByteCount: The total number of bytes that need to be loaded.

– IsDataAvailable: A Boolean value that indicates whether data is available for processing.

– IsDone: A Boolean value that indicates whether the data loading operation is complete.

– Message: A text message that describes the current state of the data loading operation.

How to use Progress.DataSourceProgress

To use Progress.DataSourceProgress, you need to include it in your custom functions or scripts. Here is an example of how to use Progress.DataSourceProgress to monitor the progress of a data loading operation:

let

Source = Excel.Workbook(File.Contents(“C:SalesData.xlsx”), null, true),

SalesData_Table = Source{[Item=”SalesData”,Kind=”Table”]}[Data],

ProgressMonitor = (Current, Total) =>

let

Progress = Progress.DataSourceProgress(),

Message = “Loading SalesData (” & Text.From(Progress[CurrentByteCount]) & ” of ” & Text.From(Progress[TotalByteCount]) & ” bytes)”

in

if Progress[IsDone] then

Message & ” – Done.”

else

Message,

#”Invoked Custom Function With Progress Monitor” = Table.TransformColumns(SalesData_Table, {“SalesAmount”, each ProgressMonitor(_, Table.RowCount(SalesData_Table))})

in

#”Invoked Custom Function With Progress Monitor”

() =>

let

Progress = [CurrentByteCount = 0, TotalByteCount = 0, IsDataAvailable = false, IsDone = false, Message = “”],

InternalStepSize = 32 1024,

InternalCurrentByteCount = Extension.CurrentByteCount,

InternalTotalByteCount = Extension.TotalByteCount,

InternalIsDataAvailable = Extension.IsDataAvailable,

InternalIsDone = Extension.IsDone,

InternalMessage = Extension.Message

in

[

CurrentByteCount = if InternalCurrentByteCount <> null then InternalCurrentByteCount else Progress[CurrentByteCount],

TotalByteCount = if InternalTotalByteCount <> null then InternalTotalByteCount else Progress[TotalByteCount],

IsDataAvailable = if InternalIsDataAvailable <> null then InternalIsDataAvailable else Progress[IsDataAvailable],

IsDone = if InternalIsDone <> null then InternalIsDone else Progress[IsDone],

Message = if InternalMessage <> null then InternalMessage else Progress[Message]

]