Progress.DataSourceProgress
A
- AccessControlEntry.ConditionToIdentities
- Access.Database
- ActiveDirectory.Domains
- AdobeAnalytics.Cubes
- AdoDotNet.DataSource
- AdoDotNet.Query
- AnalysisServices.Database
- AnalysisServices.Databases
- AzureStorage.BlobContents
- AzureStorage.Blobs
- AzureStorage.DataLake
- AzureStorage.DataLakeContents
- AzureStorage.Tables
B
- BinaryFormat.Binary
- BinaryFormat.Byte
- BinaryFormat.ByteOrder
- BinaryFormat.Choice
- BinaryFormat.Decimal
- BinaryFormat.Double
- BinaryFormat.Group
- BinaryFormat.Length
- BinaryFormat.List
- BinaryFormat.Null
- BinaryFormat.Record
- BinaryFormat.SignedInteger16
- BinaryFormat.SignedInteger32
- BinaryFormat.SignedInteger64
- BinaryFormat.Single
- BinaryFormat.Text
- BinaryFormat.Transform
- BinaryFormat.UnsignedInteger16
- BinaryFormat.UnsignedInteger32
- BinaryFormat.UnsignedInteger64
- BinaryFormat.7BitEncodedSignedInteger
- BinaryFormat.7BitEncodedUnsignedInteger
- Binary data
- Binary.ApproximateLength
- Binary.Buffer
- Binary.Combine
- Binary.Compress
- Binary.Decompress
- Binary.From
- Binary.FromList
- Binary.FromText
- Binary.InferContentType
- Binary.Length
- Binary.Range
- Binary.Split
- Binary.ToList
- Binary.ToText
- Binary.View
- Binary.ViewError
- Binary.ViewFunction
- Byte.From
C
- Cdm.Contents
- Character.FromNumber
- Character.ToNumber
- Combiner.CombineTextByDelimiter
- Combiner.CombineTextByEachDelimiter
- Combiner.CombineTextByLengths
- Combiner.CombineTextByPositions
- Combiner.CombineTextByRanges
- Comparer.FromCulture
- Comparer.Ordinal
- Comparer.OrdinalIgnoreCase
- Controlling byte order
- Csv.Document
- Cube.AddAndExpandDimensionColumn
- Cube.AddMeasureColumn
- Cube.ApplyParameter
- Cube.AttributeMemberId
- Cube.AttributeMemberProperty
- Cube.CollapseAndRemoveColumns
- Cube.Dimensions
- Cube.DisplayFolders
- Cube.MeasureProperties
- Cube.MeasureProperty
- Cube.Measures
- Cube.Parameters
- Cube.Properties
- Cube.PropertyKey
- Cube.ReplaceDimensions
- Cube.Transform
- Currency.From
D
- DateTime.AddZone
- DateTime.Date
- DateTime.FixedLocalNow
- DateTime.From
- DateTime.FromFileTime
- DateTime.FromText
- DateTime.IsInCurrentHour
- DateTime.IsInCurrentMinute
- DateTime.IsInCurrentSecond
- DateTime.IsInNextHour
- DateTime.IsInNextMinute
- DateTime.IsInNextNHours
- DateTime.IsInNextNMinutes
- DateTime.IsInNextNSeconds
- DateTime.IsInNextSecond
- DateTime.IsInPreviousHour
- DateTime.IsInPreviousMinute
- DateTime.IsInPreviousNHours
- DateTime.IsInPreviousNMinutes
- DateTime.IsInPreviousNSeconds
- DateTime.IsInPreviousSecond
- DateTime.LocalNow
- DateTime.Time
- DateTime.ToRecord
- DateTime.ToText
- Date.AddDays
- Date.AddMonths
- Date.AddQuarters
- Date.AddWeeks
- Date.AddYears
- Date.Day
- Date.DayOfWeek
- Date.DayOfWeekName
- Date.DayOfYear
- Date.DaysInMonth
- Date.EndOfDay
- Date.EndOfMonth
- Date.EndOfQuarter
- Date.EndOfWeek
- Date.EndOfYear
- Date.From
- Date.FromText
- Date.IsInCurrentDay
- Date.IsInCurrentMonth
- Date.IsInCurrentQuarter
- Date.IsInCurrentWeek
- Date.IsInCurrentYear
- Date.IsInNextDay
- Date.IsInNextMonth
- Date.IsInNextNDays
- Date.IsInNextNMonths
- Date.IsInNextNQuarters
- Date.IsInNextNWeeks
- Date.IsInNextNYears
- Date.IsInNextQuarter
- Date.IsInNextWeek
- Date.IsInNextYear
- Date.IsInPreviousDay
- Date.IsInPreviousMonth
- Date.IsInPreviousNDays
- Date.IsInPreviousNMonths
- Date.IsInPreviousNQuarters
- Date.IsInPreviousNWeeks
- Date.IsInPreviousNYears
- Date.IsInPreviousQuarter
- Date.IsInPreviousWeek
- Date.IsInPreviousYear
- Date.IsInYearToDate
- Date.IsLeapYear
- Date.Month
- Date.MonthName
- Date.QuarterOfYear
- Date.StartOfDay
- Date.StartOfMonth
- Date.StartOfQuarter
- Date.StartOfWeek
- Date.StartOfYear
- Date.ToRecord
- Date.ToText
- Date.WeekOfMonth
- Date.WeekOfYear
- Date.Year
- DB2.Database
- Decimal.From
- Diagnostics.ActivityId
- Diagnostics.Trace
- DirectQueryCapabilities.From
- Double.From
- Duration.Days
- Duration.From
- Duration.FromText
- Duration.Hours
- Duration.Minutes
- Duration.Seconds
- Duration.ToRecord
- Duration.TotalDays
- Duration.TotalHours
- Duration.TotalMinutes
- Duration.TotalSeconds
- Duration.ToText
E
F
G
H
I
L
- Lines.FromBinary
- Lines.FromText
- Lines.ToBinary
- Lines.ToText
- List.Accumulate
- List.AllTrue
- List.Alternate
- List.AnyTrue
- List.Average
- List.Buffer
- List.Combine
- List.ConformToPageReader
- List.Contains
- List.ContainsAll
- List.ContainsAny
- List.Count
- List.Covariance
- List.Dates
- List.DateTimes
- List.DateTimeZones
- List.Difference
- List.Distinct
- List.Durations
- List.FindText
- List.First
- List.FirstN
- List.Generate
- List.InsertRange
- List.Intersect
- List.IsDistinct
- List.IsEmpty
- List.Last
- List.LastN
- List.MatchesAll
- List.MatchesAny
- List.Max
- List.MaxN
- List.Median
- List.Min
- List.MinN
- List.Mode
- List.Modes
- List.NonNullCount
- List.Numbers
- List.Percentile
- List.PositionOf
- List.PositionOfAny
- List.Positions
- List.Product
- List.Random
- List.Range
- List.RemoveFirstN
- List.RemoveItems
- List.RemoveLastN
- List.RemoveMatchingItems
- List.RemoveNulls
- List.RemoveRange
- List.Repeat
- List.ReplaceMatchingItems
- List.ReplaceRange
- List.ReplaceValue
- List.Reverse
- List.Select
- List.Single
- List.SingleOrDefault
- List.Skip
- List.Sort
- List.Split
- List.StandardDeviation
- List.Sum
- List.Times
- List.Transform
- List.TransformMany
- List.Union
- List.Zip
- Logical.From
- Logical.FromText
- Logical.ToText
N
- Name
- Number.Abs
- Number.Acos
- Number.Asin
- Number.Atan
- Number.Atan2
- Number.BitwiseAnd
- Number.BitwiseNot
- Number.BitwiseOr
- Number.BitwiseShiftLeft
- Number.BitwiseShiftRight
- Number.BitwiseXor
- Number.Combinations
- Number.Cos
- Number.Cosh
- Number.Exp
- Number.Factorial
- Number.From
- Number.FromText
- Number.IntegerDivide
- Number.IsEven
- Number.IsNaN
- Number.IsOdd
- Number.Ln
- Number.Log
- Number.Log10
- Number.Mod
- Number.Permutations
- Number.Power
- Number.Random
- Number.RandomBetween
- Number.Round
- Number.RoundAwayFromZero
- Number.RoundDown
- Number.RoundTowardZero
- Number.RoundUp
- Number.Sign
- Number.Sin
- Number.Sinh
- Number.Sqrt
- Number.Tan
- Number.Tanh
- Number.ToText
O
P
R
- RData.FromBinary
- Record.AddField
- Record.Combine
- Record.Field
- Record.FieldCount
- Record.FieldNames
- Record.FieldOrDefault
- Record.FieldValues
- Record.FromList
- Record.FromTable
- Record.HasFields
- Record.RemoveFields
- Record.RenameFields
- Record.ReorderFields
- Record.SelectFields
- Record.ToList
- Record.ToTable
- Record.TransformFields
- Replacer.ReplaceText
- Replacer.ReplaceValue
- RowExpression.Column
- RowExpression.From
- RowExpression.Row
S
- Salesforce.Data
- Salesforce.Reports
- SapBusinessWarehouse.Cubes
- SapHana.Database
- SharePoint.Contents
- SharePoint.Files
- SharePoint.Tables
- Single.From
- Soda.Feed
- Splitter.SplitByNothing
- Splitter.SplitTextByAnyDelimiter
- Splitter.SplitTextByCharacterTransition
- Splitter.SplitTextByDelimiter
- Splitter.SplitTextByEachDelimiter
- Splitter.SplitTextByLengths
- Splitter.SplitTextByPositions
- Splitter.SplitTextByRanges
- Splitter.SplitTextByRepeatedLengths
- Splitter.SplitTextByWhitespace
- SqlExpression.SchemaFrom
- SqlExpression.ToExpression
- Sql.Database
- Sql.Databases
- Sybase.Database
T
- Tables.GetRelationships
- Table.AddColumn
- Table.AddFuzzyClusterColumn
- Table.AddIndexColumn
- Table.AddJoinColumn
- Table.AddKey
- Table.AddRankColumn
- Table.AggregateTableColumn
- Table.AlternateRows
- Table.ApproximateRowCount
- Table.Buffer
- Table.Column
- Table.ColumnCount
- Table.ColumnNames
- Table.ColumnsOfType
- Table.Combine
- Table.CombineColumns
- Table.CombineColumnsToRecord
- Table.ConformToPageReader
- Table.Contains
- Table.ContainsAll
- Table.ContainsAny
- Table.DemoteHeaders
- Table.Distinct
- Table.DuplicateColumn
- Table.ExpandListColumn
- Table.ExpandRecordColumn
- Table.ExpandTableColumn
- Table.FillDown
- Table.FillUp
- Table.FilterWithDataTable
- Table.FindText
- Table.First
- Table.FirstN
- Table.FirstValue
- Table.FromColumns
- Table.FromList
- Table.FromPartitions
- Table.FromRecords
- Table.FromRows
- Table.FromValue
- Table.FuzzyGroup
- Table.FuzzyJoin
- Table.FuzzyNestedJoin
- Table.Group
- Table.HasColumns
- Table.InsertRows
- Table.IsDistinct
- Table.IsEmpty
- Table.Join
- Table.Keys
- Table.Last
- Table.LastN
- Table.MatchesAllRows
- Table.MatchesAnyRows
- Table.Max
- Table.MaxN
- Table.Min
- Table.MinN
- Table.NestedJoin
- Table.Partition
- Table.PartitionValues
- Table.Pivot
- Table.PositionOf
- Table.PositionOfAny
- Table.PrefixColumns
- Table.Profile
- Table.PromoteHeaders
- Table.Range
- Table.RemoveColumns
- Table.RemoveFirstN
- Table.RemoveLastN
- Table.RemoveMatchingRows
- Table.RemoveRows
- Table.RemoveRowsWithErrors
- Table.RenameColumns
- Table.ReorderColumns
- Table.Repeat
- Table.ReplaceErrorValues
- Table.ReplaceKeys
- Table.ReplaceMatchingRows
- Table.ReplaceRelationshipIdentity
- Table.ReplaceRows
- Table.ReplaceValue
- Table.ReverseRows
- Table.RowCount
- Table.Schema
- Table.SelectColumns
- Table.SelectRows
- Table.SelectRowsWithErrors
- Table.SingleRow
- Table.Skip
- Table.Sort
- Table.Split
- Table.SplitAt
- Table.SplitColumn
- Table.ToColumns
- Table.ToList
- Table.ToRecords
- Table.ToRows
- Table.TransformColumnNames
- Table.TransformColumns
- Table.TransformColumnTypes
- Table.TransformRows
- Table.Transpose
- Table.Unpivot
- Table.UnpivotOtherColumns
- Table.View
- Table.ViewError
- Table.ViewFunction
- Teradata.Database
- Text.AfterDelimiter
- Text.At
- Text.BeforeDelimiter
- Text.BetweenDelimiters
- Text.Combine
- Text.Contains
- Text.End
- Text.EndsWith
- Text.From
- Text.FromBinary
- Text.InferNumberType
- Text.Insert
- Text.Length
- Text.Lower
- Text.Middle
- Text.NewGuid
- Text.PadEnd
- Text.PadStart
- Text.PositionOf
- Text.PositionOfAny
- Text.Proper
- Text.Range
- Text.Remove
- Text.RemoveRange
- Text.Repeat
- Text.Replace
- Text.ReplaceRange
- Text.Reverse
- Text.Select
- Text.Split
- Text.SplitAny
- Text.Start
- Text.StartsWith
- Text.ToBinary
- Text.ToList
- Text.Trim
- Text.TrimEnd
- Text.TrimStart
- Text.Upper
- Time.EndOfHour
- Time.From
- Time.FromText
- Time.Hour
- Time.Minute
- Time.Second
- Time.StartOfHour
- Time.ToRecord
- Time.ToText
- Transform operations
- Trigonometry
- Type.AddTableKey
- Type.ClosedRecord
- Type.Facets
- Type.ForFunction
- Type.ForRecord
- Type.FunctionParameters
- Type.FunctionRequiredParameters
- Type.FunctionReturn
- Type.Is
- Type.IsNullable
- Type.IsOpenRecord
- Type.ListItem
- Type.NonNullable
- Type.OpenRecord
- Type.RecordFields
- Type.ReplaceFacets
- Type.ReplaceTableKeys
- Type.TableColumn
- Type.TableKeys
- Type.TableRow
- Type.TableSchema
- Type.Union
V
- Value.Add
- Value.Alternates
- Value.As
- Value.Compare
- Value.Divide
- Value.Equals
- Value.Expression
- Value.Firewall
- Value.FromText
- Value.Is
- Value.Lineage
- Value.Metadata
- Value.Multiply
- Value.NativeQuery
- Value.NullableEquals
- Value.Optimize
- Value.RemoveMetadata
- Value.ReplaceMetadata
- Value.ReplaceType
- Value.Subtract
- Value.Traits
- Value.Type
- Value.VersionIdentity
- Value.Versions
- Value.ViewError
- Value.ViewFunction
- Variable.Value
In this article, we will explore the M code behind the Progress.DataSourceProgress function, which is used to monitor the progress of data loading operations in Power Query.
What is Progress.DataSourceProgress?
Progress.DataSourceProgress is a Power Query M function that returns a record with information about the progress of a data loading operation. This function is typically used within custom functions or scripts to monitor the progress of a data load and to provide feedback to the user.
The record returned by Progress.DataSourceProgress contains the following fields:
– CurrentByteCount: The number of bytes that have been loaded so far.
– TotalByteCount: The total number of bytes that need to be loaded.
– IsDataAvailable: A Boolean value that indicates whether data is available for processing.
– IsDone: A Boolean value that indicates whether the data loading operation is complete.
– Message: A text message that describes the current state of the data loading operation.
How to use Progress.DataSourceProgress
To use Progress.DataSourceProgress, you need to include it in your custom functions or scripts. Here is an example of how to use Progress.DataSourceProgress to monitor the progress of a data loading operation:
let
Source = Excel.Workbook(File.Contents(“C:SalesData.xlsx”), null, true),
SalesData_Table = Source{[Item=”SalesData”,Kind=”Table”]}[Data],
ProgressMonitor = (Current, Total) =>
let
Progress = Progress.DataSourceProgress(),
Message = “Loading SalesData (” & Text.From(Progress[CurrentByteCount]) & ” of ” & Text.From(Progress[TotalByteCount]) & ” bytes)”
in
if Progress[IsDone] then
Message & ” – Done.”
else
Message,
#”Invoked Custom Function With Progress Monitor” = Table.TransformColumns(SalesData_Table, {“SalesAmount”, each ProgressMonitor(_, Table.RowCount(SalesData_Table))})
in
#”Invoked Custom Function With Progress Monitor”
In this example, we load the SalesData table from an Excel workbook and then apply a custom function called ProgressMonitor to the SalesAmount column. The ProgressMonitor function uses Progress.DataSourceProgress to monitor the progress of the data loading operation and provides feedback to the user.
Understanding the M code behind Progress.DataSourceProgress
To understand how Progress.DataSourceProgress works, let's take a closer look at the M code behind this function:
() =>
let
Progress = [CurrentByteCount = 0, TotalByteCount = 0, IsDataAvailable = false, IsDone = false, Message = “”],
InternalStepSize = 32 1024,
InternalCurrentByteCount = Extension.CurrentByteCount,
InternalTotalByteCount = Extension.TotalByteCount,
InternalIsDataAvailable = Extension.IsDataAvailable,
InternalIsDone = Extension.IsDone,
InternalMessage = Extension.Message
in
[
CurrentByteCount = if InternalCurrentByteCount <> null then InternalCurrentByteCount else Progress[CurrentByteCount],
TotalByteCount = if InternalTotalByteCount <> null then InternalTotalByteCount else Progress[TotalByteCount],
IsDataAvailable = if InternalIsDataAvailable <> null then InternalIsDataAvailable else Progress[IsDataAvailable],
IsDone = if InternalIsDone <> null then InternalIsDone else Progress[IsDone],
Message = if InternalMessage <> null then InternalMessage else Progress[Message]
]
The Progress.DataSourceProgress function is a function that takes no arguments and returns a record. The record is initialized with default values for each field using a let statement.
The function then retrieves the current progress values from the Extension object, which is an instance of the internal Power Query object model. The InternalStepSize variable is used to specify the size of the data chunks that are loaded during the data loading operation.
The function then updates the progress record with the current progress values retrieved from the Extension object. If any of the progress values are null, then the default values from the Progress record are used instead.
In conclusion, the Progress.DataSourceProgress function is a powerful tool that can be used to monitor the progress of data loading operations in Power Query. By including this function in your custom functions and scripts, you can provide feedback to the user and ensure that your data loading operations are running smoothly.
Understanding the M code behind the Progress.DataSourceProgress function is important for any Power Query user who wants to write custom functions and scripts. By studying the code and experimenting with different values, you can gain a deeper understanding of how Power Query works and how you can use it to transform and analyze your data.