How to Create Devices and Targets

An HTTP/S monitoring checks a single URL for availability, performance, proper content, and errors. It supports all popular types of requests, cookies, form submissions, custom headers, password-secured sites (basic HTTP/S authorization as well as cookie/script authorization mechanisms), and timeout thresholds.

HTTP/S monitoring validates security certificates, checks certificate authority, and performs certificate expiration checks. It can be configured to send you reminders when the certificate expiration date is approaching.

You can convert HTTP(S) request parameters to Context Parameters to pass in values, for example, retrieved from a response of another request within the monitoring device. You can set up context parameters for the URL, headers, request body, and prepare/post scripts. For details, see How to Use Context Parameters in HTTP(S) Requests.

Bulk Import

To create monitoring for many devices in one click, choose the Bulk Import option that is provided for the HTTP/S monitoring type on the Select a Monitoring Type page. For more details, please see the Bulk Import | Web Page monitoring, HTTP/S, and PING/ICMP devices article of the wiki.

Configuring a Request

URL

Enter the URL of the page you wish to perform the task on. It should be formatted as such: www.example.com. You can turn on a visually friendly input mode by clicking the Detailed switcher on the top of the section.

SSL/Certificate Check 

Secure Socket Layer SSL Certificate Check is a standard aspect of HTTP(S) tests.

The following additional options are available:

  • Authority: verifies whether a certificate chain contains a root certificate that is trusted, or not trusted.
  • Common Name (CN): validates that an address you navigate to matches the address certificate the address was signed to.
  • Date: verifies the certificate expiration date.
  • Revocation: validates that the certificate’s chain of trust doesn’t contain a revoked certificate.
  • Usage: verifies a certificate chain for the improper use of an intermediate certificate.
  • Expiration Reminder in Days: a reminder that notifies (as an error) about certificate expiration.
  • Client Certificate: client certificate name.

Request Type

In the Request Type field, you can select one of the most-commonly-used HTTP methods to send monitoring requests to a web page. If you need to send a payload with HTTP requests, provide it in the corresponding field (see the Request Body chapter for details). The payload can be specified and sent with all types of requests except Trace (RFC2616).

See also: Target Hostname or IP Address.

Time Validation Threshold (in seconds)

Enter the maximum number of seconds the system should wait for a response from the web page before ending the task and returning an error.

Maximum timeout value is limited to 70 seconds. If the threshold is not set, then the default 70-second timeout will be applied to the task.

URL Redirects 

If the Follow Redirects option is set to Yes, the system will follow the path of the URL that is sent with the 301 response and consider each redirect as a separate HTTP request. It enables you to follow the full redirect chain (all the links the request is redirected through) in the test results, including response times both for the initial URL and subsequent responses.

We recommend that you leave the Follow Redirects option activated if you need to test not only the initial URL, but all the URLs in the chain. For example, it can be useful to perform an SSL certificate check for each URL in a redirect chain.

In cases where you want to test an initial URL only, disable the Follow Redirects option.

Note that a default redirection limit is set at 10 redirects. If you want the system to execute a particular number of redirects (less than 10), you can specify the number of URLs you want to test in your redirect chain in the Prepare Script field:

string url;
url = "http://wtatour.com/";
currentTask.TaskMaxRedirectAttempts = N;

Where N is the number of redirect locations we want to follow. To follow no redirects, simply set the number of redirect locations to 0.

Content Validation

Content Validation Keywords are used to ensure that specific text or elements are present within the HTML source code of a web page. To select right keywords for web page content validation, access the webpage’s source code and choose the keywords or phrases that uniquely represent the content.

If you see text on a webpage in your browser but can’t find it in the HTML source code, the text might not be directly included in the raw HTML but dynamically generated by JavaScript, loaded from external APIs or embedded in IFrames. If you want to use this type of text in web page content validation, configure a Single Web Page device to set up performance monitoring in a real browser window.

To Access the HTML Source

Use tools or scripts to fetch the HTML source of the target web page. For example, open the webpage in your browser, right-click anywhere on the page and select “View Page Source”.

In the Keyword fields, you can specify one or more words or phrases that you wish to search for in the HTML source code of the target web page. If the expected keywords are not found, the task will return an error.

You can enter multiple strings into the keyword fields. The values can be separated by logical expressions:

{[("keyword1"&"keyword2")|!"keyword3"]}

Where:
{[ – keyword expression start;
]} – keyword expression end;
() – grouping brackets;
& – logical AND;
| – logical OR;
! – logical NOT;
“string” – a keyword.

A successful keyword expression must include the start and end brackets as follows:

{["keyword"]}

Authorization

The HTTP authentication protocol is used to allow users to access content on some websites.

The following authentication schemes are available:

  • Basic Authentication: This method encodes the username and password in base64 and sends them in the request header. It’s simple but not secure unless used with HTTPS.
  • Digest Authentication: This scheme hashes credentials using a nonce (a random value) before sending them over the network, providing better security than Basic Authentication by preventing replay attacks.
  • NTLM Authentication: A challenge-response mechanism developed by Microsoft, NTLM is used for securing credentials in Windows environments. It provides strong security by using multiple hashing and challenge-response protocols.

Once provided, login credentials will be passed along with the request header to the web server.

  • Username: contains a username for HTTP/S  authentication.
  • User Password: contains a password for HTTP/S authentication.

Do not confuse HTTP authentication with other authentication schemes such as Bearer Authentication that involves bearer tokens and OAuth 2.0 that uses access tokens.

Read the articles on Basic Authentication Username and Password and Monitoring OAuth 2.0-based APIs for more information.

Headers

The option enables adding any additional custom headers. For example, you can define the MIME type of the data sent along with the request in the Content-Type header:

Content-Type: text/html

If the Content-Type header is not specified for the request, the request will be sent with the default content type application/x-www-form-urlencoded.

The default User-Agent header is set to:

User-Agent: Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0; .NET CLR 1.1.4322; .NET CLR 1.0.3705; .NET CLR 2.0.50727; .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729) DMBrowser/2.1 (SV)

However, the user-agent string can be replaced with any other string. To do this, add a custom header with the name “user-agent” and the specific value needed.

Request Body

Dotcom-Monitor allows you to send payloads in HTTP(S) requests (except Trace requests). The content within the HTTP request body can be sent as “raw” data (JSON, XML, etc.) or static name-value collection (Form Data).

To work with a name-value collection, you can turn on the detailed input mode by using the Detailed switcher on the top of the section and provide request parameter names and values in the corresponding field.

To send “raw” data along with the request, such as a JSON object, enter your JSON payload in the input field. You can also dynamically change the request body. For example, if you need to send the current date and time as a part of your POST request or pass the current session ID in JSON payload to a remote server. Dotcom-Monitor enables dynamically changing HTTP request payload by using the Razor syntax and data masks.

  • Example. Dynamic JSON Body for HTTP Post Requests

    To better understand how Dynamic JSON body works in the HTTP request, let’s have a look at the following example. Suppose we need to submit an order on a website and the submission transaction includes three basic steps executed sequentially:

    1. Login
    2. Check-in
    3. Order Submission

    To set up a test with these steps executed sequentially, we need to create three HTTP tasks within one monitoring device (or load test, if load testing is taking place).

    Let’s assume that we need to send the current time and a unique GUID in the JSON with the HTTP request to check in with the application. Also, to submit an order, a user session ID generated upon login and an order time is required by the application.

    To implement this test, we first need to configure a login request with basic authentication parameters to the web application web server. Next, we need to configure an HTTP request to pass the actual check-in time and unique GUID along with a JSON body. For this example, we will enter the following string using the Razor syntax in the JSON body:

    { "CheckInTime": "@Model["CurrentTime"]", "GenGuid": "@Model["Guid"]" }

    Where @Model[“<Parameter Name>”] references a necessary context parameter name in the Razor expression.

    We must declare the context parameters and specify how the Post Data should be processed in the Prepare Script field:

    context.Guid = Guid.NewGuid().ToString(); // uniq random string
context.CurrentTime = DateTime.Now.ToUniversalTime().ToString("yyyy-MM-dd\\Thh:mm:ss") + ".0Z"; // get current time in UTC
ProcessPostDataByRazor(currentTask); // the call to process the Post Data content with the Razor engine

    The result HTTP request will look similar to this:

    03:15:23
POST https://www.dotcom-monitor.com/CheckIn
{ "CheckInTime": "2021-03-30T08:15:22.0Z", "GenGuid": "5c5e3d23-66fd-49d0-bd57-62c516aea7e7" }

    In the next step, we need to configure the HTTP request to submit an order. In order to do this, we will pass the order current time and session ID, along with the item’s model identification number, in the JSON body to the target endpoint. See the JSON body for this request below:

    { "OrderTime": "@Model["OrderTime"]",   "VIEWSTATE": "@Model["Session"]",  "ModelID": "2128506" }

    To pass a value of the current session ID variable, we need to retrieve it from the login page, called at the first login step, using the View State method. It can be coded in the prepare script. Additionally, to simulate a real user’s think time, we will set the order time variable with a three minute offset. Therefore, the Prepare Script field will contain the following strings:

    context.OrderTime = DateTime.Now.AddMinutes(3).ToUniversalTime().ToString("yyyy-MM-dd\\Thh:mm:ss") + ".0Z"; // order time + 3 min
context.Session = (Tasks["Login"] as Http).body["//INPUT[@ID='__VIEWSTATE']", "VALUE"]; // track state value from Login page 
ProcessPostDataByRazor(currentTask);

    The resulting HTTP request will look similar to this:

    03:15:45
POST https://www.dotcom-monitor.com/Order
{ "OrderTime": "2021-03-30T08:18:45.0Z", "VIEWSTATE": "<Server Generated ViewState>", "ModelID": "2128506" }

To learn how to configure an HTTP request with a dynamically changing payload, see How to Dynamically Change Payload in HTTP Request.

DNS Options

The DNS Options feature allows users to choose how domain name server (DNS) requests are conducted during a monitoring task.

To specify the mode of resolving hostnames, in the DNS Resolve Mode section, select one of the available modes. For more details on the feature configuration, see DNS Mode Options.

The Custom DNS Hosts section allows to set up the mapping of IP addresses to hostnames. IPv6 and IPv4 DNS resolution is supported.

To specify the mapping, enter the IP address and the hostname in the corresponding fields.

Examples:

192.168.107.246   example.com user.example.com userauth.example.com tools.example.com
192.168.107.246   example.com
192.168.107.246   user.example.com
192.168.107.246   userauth.example.com

See also: DNS Mode Options.

Prepare Script and Post Script

The fields can contain C# code, which can be used for specific POST, GET, URL data or for validating or publishing custom headers. Please see the Using Prepare Script and Post Script article or contact technical support for more details on usage.