User Directed Spidering with Burp is a great way to catch security vulnerabilities in a web application. A spider is a software that goes through your website, following every link and looking for the next page it needs to visit. This means it can’t get stuck when there are loops or missing links – which is what would happen with a regular browser like IE or Chrome.

Burp Spider:

Spidering is a browser security testing technique that requires using special tools in order to follow all possible paths through the application. These tools are designed in such a way that they can follow every visitor and every user action, as well as collect useful information while spidering.

The first use of spidering was developed by Netscape, which was able to follow every link on the web pages and capture all the information from them, just like a human being would do with his normal browser. This automated process of gathering data is done through a very intelligent software mechanism that analyzes Web pages for possible links and then creates new requests for them. These requests are sent to the web server of the website that has been identified and collected all the information an automated interpretation process can get.

Today, this spidering technique is commonly used by security experts and penetration testers to gather any information they are interested in, such as:

• A vulnerability assessment or vulnerability discovery.
• Vulnerability exploitation.
• An application testing with different user accounts.
• Information gathering from a remote site’s API (Application Programming Interface).

The main advantage of using this technique with burp is that it provides as much as possible detail about the application. This includes collecting HTTP data, URLs, and other forms of data for further analysis, testing, and research purposes after completing your assessment.

Burp Spider Options:

The spider tool of the burp suite is called “Intruder”. In order to use intruders, you have to configure them in the intruder tab of the Options section. The first thing that you need to do is specify a host, port, and protocol. After that, you have to configure the intruder’s payload (which will be used for attacking purposes). You can also choose which type of requests you wish to use: GET and POST requests with the multipart/form-data encoding. And at last but not least; the Intruder configuration consists of specifying Intruder payloads/attack payload content in the form of a string or file name or URL.

Key Points:

• The intruder tool uses HTTP requests and replies.
• The burp plugin will be automatically loaded into your browser after you start burp intruder, and it is also available in the Intruder tab of options.
• The intruder can be configured to use a payload file or a string as an attack payload.
• Burp Intruder Attack Payload Settings, this button allows you to get access to the following interfaces which can show you what happens when the user interacts with the webpage: Status – shows you if all sessions are connected, broken, or dropped. All Connections – displays all sessions under “All Connections”. Sessions – displays all sessions under “Sessions”.