AzureStorage.DataLake

D

Back to top

T

Back to top

#

Back to top

The M Code Behind the Power Query M function AzureStorage.DataLake

What is Azure Data Lake Storage?

Azure Data Lake Storage is a cloud-based storage solution from Microsoft that is optimized for big data workloads. It is built on top of the Azure Blob Storage and provides a distributed file system that can store and process petabytes of data. With Azure Data Lake Storage, you can store any type of data, including structured, semi-structured, and unstructured data.

Understanding the AzureStorage.DataLake Function

The AzureStorage.DataLake function is a part of the Power Query M language that allows users to read and write data to and from Azure Data Lake Storage. It is a powerful function that enables users to perform a wide range of data manipulation tasks using M language.

The syntax for the AzureStorage.DataLake function is as follows:

AzureStorage.DataLake(accountName as text, optional options as nullable record) as function

The function takes two parameters:

– `accountName`: The name of the Azure Data Lake Storage account.

– `options`: An optional record that contains additional parameters for the function.

The M Code Behind the AzureStorage.DataLake Function

The AzureStorage.DataLake function is implemented using M language code that interacts with the Azure Data Lake Storage REST API. The M code behind the function is responsible for connecting to the Azure Data Lake Storage account, authenticating the user, and performing the requested data operations.

Here is a high-level overview of the M code behind the AzureStorage.DataLake function:

1. The M code starts by creating a URL string that points to the Azure Data Lake Storage account.

2. The code then creates a web request using the URL string and sets the necessary headers for authentication.

3. The web request is then sent to the Azure Data Lake Storage REST API, which returns a response.

4. The M code then parses the response and returns the requested data.

Here is the detailed breakdown of the M code behind the AzureStorage.DataLake function:

let

options = if options <> null then options else [Timeout=#duration(0, 3, 0, 0)],

url = "https://" & accountName & ".azuredatalakestore.net",

headers = [

#"Content-Type"="application/json",

#"Authorization"="Bearer " & AccessToken()

],

webRequest = Web.Contents(url, headers, options),

response = Json.Document(webRequest)

in

response

The M code starts by checking if any options were passed to the function. If no options were passed, it sets a default timeout of 3 minutes. It then creates a URL string that points to the Azure Data Lake Storage account by concatenating the account name with the REST API endpoint.

The code then creates a dictionary of headers that are required for authentication. The `”Authorization”` header is set to the user’s access token, which is obtained using the AccessToken() function.

The M code then creates a web request using the URL string and headers and sends it to the Azure Data Lake Storage REST API using the `Web.Contents()` function.

The REST API returns a response, which is in JSON format. The M code then uses the `Json.Document()` function to parse the response and returns the requested data.

The AzureStorage.DataLake function in Power Query M language is a powerful tool that allows users to interact with Azure Data Lake Storage. The M code behind the function interacts with the Azure Data Lake Storage REST API and performs the requested data operations. Understanding the M code behind the AzureStorage.DataLake function can help users to better utilize the function and perform more complex data manipulation tasks.

Power Query and M Training Courses by G Com Solutions (0800 998 9248)

Upcoming Courses

Contact Us

    Subject

    Your Name (required)

    Company/Organisation

    Email (required)

    Telephone

    Training Course(s)

    Your Message

    Upload Example Document(s) (Zip multiple files)