Non-functional testing is concerned with the non-functional requirements and is designed to evaluate the readiness of a system according to several criteria not covered by functional testing. It enables the measurement and comparison of the testing of non-functional attributes of software systems.
Non Functional testing- its a part of checking the stability in the application, for instance how does the variables reflect if the data entered is not relevant to the data which specified. non functional testing involves adhoc testing process to check the stability. non functionality testing involves different ways meaning it can be testing the acceptance after changing the environment changing the pop up messages etc.
Non-functional testing is the testing of a software application for its non-functional requirements.
Under this category, evolution client will be tested to evaluate the ability to perform its required functions under stated conditions and set of operations for a specified period of time or number of iterations.
The scope of reliability testing is to execute basic functionalities provided by evolution client i.e. Mailer, Calendar, Task and Address Book operations. The type of account in which these operations will be performed will be Groupwise, Exchange and IMAP.
User:Sush: LDAP address book also needs to be considered, with account of any type.
a. 6 evolution clients will be installed on a SLP 10 test machine and will be configured in the following manner:
b. 2 evolution clients will be configured for a groupwise account
c. 2 clients will be configured for an exchange account
d. 2 clients will be configured for an IMAP account
All the 6 clients running on different test machines will be running simultaneously performing the following basic operations (test scenarios) :
a. Send a mail to different contacts randomly selecting from a list of email-ids.
b. Send a mail with or without attachments
c. Send a plain text mail with or without attachments
d. Send an HTML mail with or without attachments
e. Send a mail with huge text/HTML content spanning multiple pages
f. Open a large text/html mail with or without attachment
g. Reply to an existing mail
h. Forward an existing mail
a. Create a new meeting request
b. Add 1 or multiple invitees to a meeting request and send the appointment.
c. Add 1 or multiple attachments (different file formats) in the meeting request and send the appointment.
d. Accept an appointment present in the inbox.
e. Discard an appointment present in the inbox.
3. Address book:
a. Send a mail/meeting request autocompleting the ids from the address books.
b. Send a mail/meeting request autocompleting the ids to distribution lists.
c. Search for the contacts in remote address books and large local books.
This testing will be conducted for evolution to evaluate the time taken or response time of evolution to perform it's required functions (in mailer, calendar, address book and tasks) under stated conditions in comparison with different versions of evolution.
Scalability testing of the evolution client will be done together with the performance testing and will mainly involve the following:
1. Measuring the time it takes for evolution to load when the number of mails in the inbox are 100, 1000, 5000, 10000, 50000, 100000.
2. Measuring the time it takes for the address book to load when the number of contacts are gradually increased from 50 to 10000 and beyond (user:sush with most of the contacts having images).
3. Measuring the time it takes to perform a search when there are many number of mails present in the inbox.
4. Measuring the time it takes to perform a search in the address book (user:sush both when address book is marked for offline usage and not also for remote address books (LDAP, Exchange GAL, GW SAB) having more than 10,000 entries).
5. Measuring the time it takes to load the calendar entries from a remote server.
6. Measuring the time when the number of entries in the calendar is very high.
7. Measuring the time it takes to perform auto-completion.
Non Functional Testing suggests "HOW WELL" the product behaves. This HOW WELL definition is with reference to ease of use, reliability, performance, security etc.
Security Testing refers to testing of software in order to keep company data and resources secure from mistaken/accidental users, hackers, and other malevolent attackers. In this context, following are the major areas to focus on:
1. Database-driven websites are very common, and the parameters of a database query can often be seen in the URL of web pages. Here comes the time to test for Data Leakage. This refers to the fact that when information is passed in the query string i.e. in URL (e.g. using the "Get" method), an attacker may be able to access the database by iterating through the sequence of parameters passed in query string that forms part of the URL. For instance, a customer may somehow manage to alter the price of an order, or mark an unpaid order as paid. Special consideration needs to be given to this aspect while testing security related scenarios of websites.
2. While testing a website from a security perspective, make sure to test for SQL Injection Attacks. This is the name given to a vulnerability caused by poor input validation in an application, resulting in running unintended SQL scripts. For details on how to test for the mentioned vulnerability, refer to http://www.unixwiz.net/techtips/sql-injection.html.
3. Weakness in Control Panel of a shopping cart is another domain to focus on. In the Control Panel, roles and rights need to be clearly defined. Restricted access needs to be given to users e.g. in case of an admin control panel for a shopping cart, the right to view and alter the transactions needs to be restricted to a particular role and should not be given to multiple users. Only the super user should have access to alter the transaction details. Moreover, thorough testing is required to test the access rights definition feature. This is important because it may happen that a user with limited access may somehow manage to elevate his/her rights and gain the rights of any other user (say super user).
4. Incase of credit card transactions, if credit card numbers are to be stored in the database, a tester needs to ensure that a proper encryption algorithm is in place to encrypt those numbers and later on decrypt them.
Performance testing is testing that is performed to determine how fast some aspect of a system performs under a particular workload. This is specifically related to response times. With reference to this, following are the main points of considerations:
1.Web hits should cater for handling peak hit rate, not the average rate.
2.Response time should be catered with respect to download speed and complexity of the web page e.g. graphics displayed on the web page. A poorly-designed highly graphical dynamic website will be considered ‘slow’ and this element needs to be catered for seriously while testing performance of websites. This is specially required incase of broadband connections; since in this case pages that don’t appear instantly will perhaps never be seen at all.
3.Database indexing technology needs to be tested incase you have switched from a moderately populated database to an over populated database, e.g. if you move from a database with 50 rows in its tables to 50,000 rows, response time should not increase too much; rather it should be a matter of milliseconds. If however, response time increases too much this implies indexing has not been implemented on the corresponding columns, which ultimately is a performance issue and needs to be rectified.
4.The web application needs to be load tested to check the scalability of the website. To cater for this aspect of performance testing, refer to Load Testing.
Load testing refers to testing the system behavior and performance under stressful conditions.
As mentioned in the context of performance testing, load testing is done to analyze scalability of the website.
[Web application scalability is the ability to sustain the required number of simultaneous users and/or transactions, while maintaining adequate response times to end users].
Now the question arises, why is load testing so important? The reason is, most performance issues arise only when the server is stressed with a high user load. This means that you should perform load testing to know how many concurrent visitors your site can serve flawlessly. It can be difficult to organize such testing without the help of a group of real users. The best approach would be to use an automated load test tool which would serve the purpose of simulating many different users coming from different IP addresses, each with their own parameters: cookies, input data for various page forms, name and password, connection speed and their own specific path through the site. Such an automated tool may also help in testing the website at a constant load through out the entire test time or increase/decrease load after specified time intervals. Eventually detailed reports and graphs may be generated to depict test results with respect to performance issues, response time, bandwidth usage, errors generated etc.
For detail on website testing tools, refer to http://www.softwareqatest.com/qatweb1.html.
Usability testing refers to evaluating the ease with which users can learn and use a product (such as a website) for its intended purpose.
If usability testing uncovers difficulties, such as people having difficulty in understanding instructions, or interpreting feedback, then this aspect needs to be highlighted and design needs to be improved and re-tested.
Different usability evaluation techniques may be used. For instance, formal testing (such as lab testing, questionnaires etc) or informal testing (such as walkthroughs, peer reviews etc).
Following points may help in performing formal testing:
1.While testing for usability issues, the first and the foremost thing to note is check whether the company’s home page is displaying information regarding what the company does and its purpose.
2.It is a good practice to have the name and logo placed on every page and the logo can be a link to the home page (except on the home page itself, where the logo should not be a link: never have a link that points right back to the current page).
3.Each page should be structured so as to break long text into smaller units. e.g. grouping or subheadings can be used to help ignore large chunks of the page in a single glance.
4.Another important usability issue is to check for link titles, which provide users with a preview of where each link will take them, before they have clicked on it.
5.Test for broken links.
6.The behavior of websites with respect to supported resolution(s) can be tested to ensure that changing resolution does not distort the layout of web pages.
7.The layout of all the pages should be in accordance with the design specified. All pages need to be consistent with respect to the following:
a. Layout of controls
b. Color scheme of pages
c. Font style, size and case of forms and reports
d. Font style and size of text displayed on all the pages of website.
e. Font style, size and case of column headings incase of tables throughout the website
8. The navigation links in all the subsequent pages need to depict the entire path followed to reach a specific page. Clicking any of the links should lead to the corresponding page. Remember, if you are testing with the intent to improve the usability of the website, you should not have the feeling "Ok, where to go now?" or "Where did I start from?".
9. On all ‘Submit’ buttons, there should be a proper indication to the user corresponding to the action performed (success/failure).
10. Test for semantic and syntactic errors incase of text displayed on all the pages.
11. On long forms (e.g. while adding users, providing company information etc), a ‘Clear’ button has to be provided to reset all the fields.
12. It would be a good idea to provide a proper welcome message along with username when the user logs in.
13. If multiple records are to be displayed per page, there should be a limit on the maximum number of records displayed per page (e.g. 10 or 20). An alternative could be to ask the user to specify his/her own limit. Moreover, distribution of records in terms of page numbers may be provided to increase readability. (e.g. Page: 1 2 3 4).
14. Test for navigation within a form i.e. all the fields within a form should have proper tab-order.
15. Different colors may be used to distinguish visited and un-visited links.
Testing the application based on the clients and performance requirement. Non-Functioning testing is done based on the requirements and test scenarios defined by the client.
Non Functional testing- its a part of checking the stability in the appication, for instance how does the variables reflect if the data entered is not relevant to the data which specified. non functional testing involves adhoc testing process to check the stability. Non functionality testing involves different ways meaning it can be testing the accespatence after changing the environment changing the popup messeges ect.
Slow software doesn't please many people, and performance testing helps to find problems such as bottlenecks, race conditions, limits, etc. This section introduces performance testing and provides brief examples of some useful techniques for designing test cases.
Security testing is a hot topic currently, however only a small minority of software testers actually DO security testing. This section introduces the issues and mentions some useful techniques. References are provided at the end of the section will help you to get started.
Usability and accessibility have a significant effect on whether the software will be used, or not. Legislation and litigation is driving companies to make improvements to their software, albeit slowly in many cases. Usability testing is well-established with a wide range of techniques available. Read this section to learn more about usability and the techniques that are in use.
How much can you trust software to be there when you need it? Dependability testing helps to provide the answers. Read this section to learn more about the topic. This section includes some self-assessment questions.
Testing Miscellaneous Software Qualities:
This section covers a variety of topics including compatibility, interoperability, and portability testing techniques. This section includes some self-assessment questions.
It is testing of “how” the system works. Non functional testing may be performed at all test levels. The term non-functional testing describes the tests required to measure characteristics of systems and software that can be quantified on a varying scale, such as response times for performance testing. Types of Non-functional testing are performance testing, load testing, stress testing, usability testing, maintainability testing, reliability testing and portability testing.
"A non-functional requirement is a statement of how a system must behave, it is a constraint upon the systems behavior."
Non-functional requirements specify all the remaining requirements not covered by the functional requirements. They specify criteria that judge the operation of a system, rather than specific behaviors, for example:
"Display of the patient's vital signs must respond to a change in the patient's status within 2 seconds."
Typical non-functional requirements are:
Performance - Response Time, Throughput, Utilization, Static Volumetric
The term non-functional testing describes the tests required to measure characteristics of systems and software that can be quantified on a varying scale, such as response times for performance testing.
Types of Non-functional testing are performance testing, load testing, stress testing, usability testing, maintainability testing, reliability testing and portability testing.