Web.Contents

D

Back to top

T

Back to top

#

Back to top

The M Code Behind the Power Query M function Web.Contents

In this article, we will take a closer look at the M code behind the Web.Contents function and explore some of the ways in which it can be used to extract data from websites.

Understanding the Web.Contents Function

The Web.Contents function is used to retrieve data from a website using a URL. The function takes two arguments – the URL of the website to retrieve data from and an optional set of options that can be used to customize the request.

The options argument is an optional record that can be used to specify a variety of settings for the request. For example, the options record can be used to specify custom headers, cookies, and user agent strings. It can also be used to specify the encoding to use for the response data.

When the Web.Contents function is called, it sends a request to the specified URL and waits for a response. Once a response is received, the function returns a binary value that represents the response data.

Using Web.Contents to Extract Data

One of the most common use cases for the Web.Contents function is to extract data from websites. To do this, users can use the Web.Contents function in conjunction with other Power Query M functions, such as Xml.Document or Json.Document, to parse the response data.

For example, let’s say we want to extract data from the following website:

https://jsonplaceholder.typicode.com/users/1

To do this, we can use the following M code:

let

url = "https://jsonplaceholder.typicode.com/users/1",

options = [Headers=[#"Content-Type"="application/json"]],

source = Web.Contents(url, options),

json = Json.Document(source)

in

json

This M code retrieves the data from the specified URL and uses the Json.Document function to parse the response data into a usable format.

Customizing the Web.Contents Request

As mentioned earlier, the options argument can be used to customize the request sent by the Web.Contents function. This can be useful in a variety of different scenarios.

For example, let’s say we want to retrieve data from a website that requires authentication. In this case, we can use the options argument to specify the username and password to use for the request.

let

url = "https://example.com/data",

options = [Headers=[Authorization="Basic " & Binary.ToText(Text.ToBinary("username:password"), 0)]],

source = Web.Contents(url, options),

xml = Xml.Document(source)

in

xml

This M code retrieves data from the specified URL and uses the authorization header to send the username and password for authentication.

The Web.Contents function is a powerful tool for extracting data from websites using Power Query M. By understanding the M code behind the function and how to customize the request using the options argument, users can extract data from a variety of different sources and transform it into a format that can be used for analysis.

Power Query and M Training Courses by G Com Solutions (0800 998 9248)

Upcoming Courses

Contact Us

    Subject

    Your Name (required)

    Company/Organisation

    Email (required)

    Telephone

    Training Course(s)

    Your Message

    Upload Example Document(s) (Zip multiple files)