AzureStorage.DataLakeContents

A

B

C

D

E

F

G

H

I

J

L

M

N

O

P

R

S

T

U

V

W

X

#

The M Code Behind the Power Query M function AzureStorage.DataLakeContents

What is Azure Data Lake Storage?

Azure Data Lake Storage is a cloud-based storage solution that allows users to store and analyze big data in a cost-effective and scalable way. It provides two types of storage accounts: Gen1 and Gen2. Both storage accounts offer similar functionalities but have different features. Gen1 storage accounts provide a hierarchical namespace, while Gen2 storage accounts provide a flat namespace. This article will focus on the M code for accessing data from both storage account types.

AzureStorage.DataLakeContents Function

The AzureStorage.DataLakeContents function is a custom M function that allows users to access and transform data from Azure Data Lake Storage accounts. The function takes two parameters: the path to the data and the optional options parameter. The path parameter is a string that specifies the path to the data in the Azure Data Lake Storage account. The options parameter is an optional record that specifies additional options for the function. The function returns a table with the contents of the data.

The M code for the AzureStorage.DataLakeContents function is as follows:


let
    AzureStorage.DataLakeContents = (path as text, optional options as nullable record) as table =>
    let
        options_defaults = [
            HierarchicalNavigation = true,
            IncludeSubfolders = true,
            InferDataTypes = true,
            QuoteCharacter = """"""""""""",
            FirstRowAsHeader = true,
            Encoding = null,
            Culture = null,
            Delimiter = null,
            ExtraColumns = null,
            SkipLines = 0,
            CommentStyle = null,
            ColumnsToIgnore = null,
            QuoteStyle = QuoteStyle.None,
            NullValues = null
        ],
        options_combined = Record.Combine(options_defaults, options),
        parameters = [
            HierarchicalNavigation = options_combined[HierarchicalNavigation],
            IncludeSubfolders = options_combined[IncludeSubfolders],
            InferDataTypes = options_combined[InferDataTypes],
            QuoteCharacter = options_combined[QuoteCharacter],
            FirstRowAsHeader = options_combined[FirstRowAsHeader],
            Encoding = options_combined[Encoding],
            Culture = options_combined[Culture],
            Delimiter = options_combined[Delimiter],
            ExtraColumns = options_combined[ExtraColumns],
            SkipLines = options_combined[SkipLines],
            CommentStyle = options_combined[CommentStyle],
            ColumnsToIgnore = options_combined[ColumnsToIgnore],
            QuoteStyle = options_combined[QuoteStyle],
            NullValues = options_combined[NullValues]
        ],
        data = Csv.Document(
            Web.Contents(
                "https://{accountName}.dfs.core.windows.net/" & path,
                [
                    Headers = [
                        #"Authorization" = "Bearer " & AccessToken(),
                        #"Content-Type" = "text/csv"
                    ],
                    Query = [
                        "resource" = "filesystem"
                    ]
                ]
            ),
            parameters
        )
    in
        data
in
    AzureStorage.DataLakeContents

How to Use AzureStorage.DataLakeContents Function

The AzureStorage.DataLakeContents function can be used in Power Query to access and transform data from Azure Data Lake Storage accounts. To use the function, follow these steps:

1. Open Power Query Editor

2. Click on the “Blank Query” button in the “Home” tab

3. In the “Query Settings” pane, click on the “Advanced Editor” button

4. Paste the M code for the AzureStorage.DataLakeContents function into the editor

5. Replace the `{accountName}` placeholder in the M code with the name of the Azure Data Lake Storage account to access

6. Save the M code and close the editor

7. In the “Home” tab, click on the “Close & Load” button to load the data into Power Query

Once the data is loaded into Power Query, users can transform it using the Power Query interface. The AzureStorage.DataLakeContents function can also be combined with other Power Query functions to perform complex data transformations.

The AzureStorage.DataLakeContents function is a powerful M function that allows users to access and transform data from Azure Data Lake Storage accounts. The M code for the function is relatively simple and can be customized to fit specific data transformation needs. By learning how to use this function, users can take advantage of the scalability and cost-effectiveness of Azure Data Lake Storage to perform complex data transformations.

Power Query and M Training Courses by G Com Solutions (0800 998 9248)

Power Query and M Intensive Training Course
£1,050.00 – £26,550.00
Select options
Power Query and M Introduction
£395.00 – £9,750.00
Select options
Power Query and M Intermediate
£395.00 – £9,750.00
Select options
Power Query and M Advanced
£395.00 – £9,750.00
Select options

A

B

C

D

E

F

G

H

I

J

L

M

N

O

P

R

S

T

U

V

W

X

#

What is Azure Data Lake Storage?

AzureStorage.DataLakeContents Function

How to Use AzureStorage.DataLakeContents Function

Power Query and M Training Courses by G Com Solutions (0800 998 9248)

Power Query and M Intensive Training Course

Power Query and M Introduction

Power Query and M Intermediate

Power Query and M Advanced

Upcoming Courses

Contact Us

Upload Your Profile

A

B

C

D

E

F

G

H

I

J

L

M

N

O

P

R

S

T

U

V

W

X

#

What is Azure Data Lake Storage?

AzureStorage.DataLakeContents Function

How to Use AzureStorage.DataLakeContents Function

Power Query and M Training Courses by G Com Solutions (0800 998 9248)

Power Query and M Intensive Training Course

Power Query and M Introduction

Power Query and M Intermediate

Power Query and M Advanced

Upcoming Courses

Contact Us