Thursday, December 10, 2009

Daily Excerpt: Appendix D, Security Testing - Understanding A11y,an a11y testing Primer

OFF to Munich tomorrow but I want to get another Excerpt out from my new book, today I want to give a look at Appendix D. and specifically security testing and a11y

Appendix D – Different System Test types and why they matter to Accessibility Testing


The following lists of system tests are important to the Accessibility tester. This is a list of the most common types of system tests; please do be aware that there are many different types. If you are responsible for the accessibility testing for your organization it is very important for you to review all of the other QA test plans available to assist in the development of your test plan.

 
 
Security testing


Security testing - Security testing is a system test that validates the programs specific access / authentication logic works properly. Different levels of access may see different screen, different messages and different user interfaces.

The accessibility tester needs to be aware of these specific functions and how different user access levels may result in different output from the program. Additionally all prompts, warning and notifications must be tested to prove they are accessible.

For Example:

Every thirty days the user must change their password. If they have not done so an image is presented on the screen that says today is your last chance to change your password. The image unfortunately has no alternative text and the use has image x. The user continues and tomorrow cannot access the system. By being away of this security functionality the Accessibility tester can develop a proper test case.

Tuesday, December 8, 2009

Daily Excerpt: (The basics of a QA Test Plan) Understanding Accessibility, An accessibility testing primer

The Basics of a QA Test Plan


The central part of accomplishing any task is to develop a plan. The plan that that you develop should define what you are going to do and how you will know when it is completed. This may seem a bit basic and that is because it is basic. If the step of developing a plan is skipped and you begin accessibility testing you will be less than successful.

Test plans will include different types of system testing, some basic types are; Exploratory Testing, Functional testing, Stress testing, Boundary testing, Security testing, Graphical user Interface (GUI) testing, Browser Testing, and many other specific testing methods that exist under the system test banner. For the purpose of this section we will be covering exploratory testing and unit testing and the associated black-box and white-box testing that are a part of those processes. Appendix D will include a more exhaustive list and why the individual system test types matter in Accessibility testing.

One of the most common types of system tests run by accessibility testers and non structured engineering labs is the exploratory testing methodology. This method is not used because it is the best; it is used because many accessibility testers do not specifically come from a quality assurance background. One of the biggest problems with this type of testing is that it is all Black-Box Testing. Black-box testing addresses quality assurance testing from the perspective of being outside of the program or functionality. Knowledge of the program code is not required and in most cases it is not available. Rather, as the user “Kicks” the tires of the program they come up with test cases. These scenario based test cases are then used to build the test plan.

NOTE: Black-box testing cannot guarantee all program functionality and user interfaces are tested.

Example one:

A Developer develops a simple application, a web page form; this form is intended to accept address information and then submit the information into a database.

The tester, using the exploratory testing process opens the web page and enters the data in the form and submits the same. The user receives in response a JavaScript alert box indicating that the information was accepted. The problem is that the tester may now make the assumption that the form works. However they do not have enough information to come to that conclusion.

In the above example the simple form was tested but the testing was far from complete. The main reason is that the tester does not know all of the possible outcomes of the data entry. Knowing all of the possible outcomes is where white-box testing excels. White-box testing refers to testing a product where intimate internal knowledge of the program is specifically required. By evaluating the code the quality assurance engineer, with programming skills, evaluates the data flow, branching and user path analysis to develop the test plan.

Note: While white-box testing is applicable at most parts of the testing process, it is best used as part of unit testing. The unit test refers to the testing, generally by the programmer or designer, of the smallest units of the program that they are working on. This unit may be a screen, page, function or procedure. The test assures that the bit being tested conforms to functional design.

In our example above, the black-box testing was done without access to the code or the knowledge of possible paths. The exploratory tester may not know that if the user enters into the form that they are from outside of the United States that a new form will come up with specific options for non US residents. So as you can see the more complete testing methodology would combine both black-box testing and white-box testing. In order to develop and complete a mature black box testing plan, you would need to know all of the expected outcomes and functions of the program-typically derived from white-box testing plans. This however does present a problem. If you are outsourcing the Accessibility testing (we will cover this more later in this book), you need to provide the accessibility tester the white-box derived testing plan that includes unit testing and you should expect them to develop and deliver in turn their exploratory testing plan.

Now that we have discussed the two main methods for developing test cases we can discuss how they bond together into a test plan. Test Plan refers to a group of test cases that represent a baseline requirement to provide for a systematic testing of a program to validate that it conforms to design specifications and that it is suitable for its intended use. Besides the individual test cases there needs to be a complete definition to the test plan. All test plans should have, in some form, at least the ten following sections in a manner that fits the company requirements.

I. Introduction – The introduction describes what the plan is and what types of system testing will be completed in the plan. Using our example above again: We would say; that the test plan has two levels of testing, exploratory testing and unit testing.

II. Test Subject – The test subject defines what will be tested. Using our example: We would say that the address form was the test subject.

III. Scope – The scope section clearly details what will be tested and what will not be tested as part of the test plan. Using our example: The test plan will test the address entry form for proper entry including both exploratory testing and specific unit testing. The scope does exclude testing the database server and all web browsers except for Internet Explorer

IV. Testing Approach – The testing approach explains how the testing team will attack the process of testing. Using our form example: The testing will include both unit and exploratory testing. The accessibility unit testing will be derived from the unit test plans delivered to the accessibility tester and the exploratory test cases will be completed by a separate quality assurance engineer.

V. Testing Tools – The testing tools section details all tools that will be used as part of the testing process. In our example above: We may say that the tester will complete the form using the JAWS screen reader.

VI. Defect Classification – The defect classification section is very important because it provides guidance and severity acknowledgement that allows leadership to make informed decisions on next steps. Example: Severity of defects will be determined based on their impact on the user and or program and they will be numbered one through five:

1) Do not ship, program fails

2) Severe, only one sub-section fails

3) Moderate, some functionality may be lost

4) Minor, some graphical user issues

5) Cosmetic: Spelling errors and such (note: too many severity fives will become a severity one)

VII. Success Criteria – The success criteria clearly details if a test case passes or fails. In general there are four possible test case outcomes: Pass, Fail, Not Applicable, Not tested/Incomplete.

VIII. Staff and/or Training required for completing the test plan.

IX. Test Cases – The test cases covered by the plan.

X. Specific Environment / Operating System Requirements – This details the environment where testing will be performed. Example: Windows Seven Professional, Internet Explorer (IE) 8.0, 32 MB of Ram, > 200 gigabytes (GB) of free hard drive disk (HDD) space.

Sunday, December 6, 2009

Daily Excerpt: (Outsourcing Accessibility testing) Understanding Accessibility, An accessibility testing primer

Outsourcing Accessibility Testing
With the specific Subject Matter Expert (SME) requirements that are an integral part of accessibility testing, many companies choose to outsource the site and/or application accessibility testing to a third party testing company. While there is nothing new with outsourcing, it is important to use caution. This section will cover some tips, tricks and suggestions for you to consider before selecting the organization to which you will outsource your compliance testing.

 
What do you need?
The first step is selecting your outsourcing partner, has nothing to do with the partner. The first step is for you to develop and to define the scope. While this may seem very basic, it is the lynchpin to a successful project. The scope of a project can include many things, and the basic items are reviewed below.

 
First there is the work to be done. This details what will be tested and what the same will be tested against, specifically what standards will be used as part of the testing. All expectations should be set as part of the scoping of the project for what is being tested and what the schedule will be for all the deliveries. To fully understand the task you need to be able to define the size of the project. If it is a basic website you are talking about you should know all of the pages/content. If you are dealing with a web based application or dynamic site it is important to map out all of the possible screens and testing points. In the previous sections we discussed developing your test plans; this is what you would be using to come up with the scope of the testing project. Tip; if the outsourcing partner you are considering does not ask for any existing test plans at this stage you have just received a red flag. It is not the end of the world; you are hiring someone to test your application. If they are qualified they should know they need to build a test plan or use your existing plan. After defining the scope you are ready to proceed to the next step.

 
Evaluation of the Testing Company
We have all seen at some time or another a slick TV pitchman that wants us to buy an amazing cleaning product that will clean anything including stains that have been on surfaces for ten years. After they explain that they will give you three for the price of one and all you need to do is pay for the shipping. Some of us have even purchased these products; I once purchased the amazing super back cleaner, yes, I made a mistake! However, in my defense, if you get a slick sales person who tells you exactly what you want to hear you may jump in before evaluating the product. I lost about 20 US Dollars, but when you consider an outsourced project you have cost related to: Preparation, Management, Testing, Project Management and then any other travel or associated materials.

 
The first step in evaluating your potential outsourced testing partner will be to review the provider as you would review the hiring of a new full-time employee. Now this is not perhaps the entire company but it does not hurt to ask for the resumes of the project team and then to hire the technical team lead/manager. Once you have the list of potential companies down to a few providers the real work will then begin. You should do the following:

 
  • Ask for three references on the company’s services where one of the references is related to the exact type of project that you have.
  • Ask for a detailed capabilities document
  • And when it comes to Outsourcing Accessibility testing you need to ask for examples of completed deliveries.
  • You need them to deliver, in writing the names of the team members and their skill sets.

 
At this point you should be engaged in a dialog that when completed will make you feel comfortable with your selection.

 
Does Experience Matter?

During the evaluation process you need to consider experience. Think of it; if you need a new windshield and you have to select who will be repairing it, do you want someone who is a windshield expert in every way but has never installed a windshield? Of course not! The same is true in selecting a quality assurance tester. They can be the brightest Accessibility SME but if they have no testing experience and you are hiring a tester should you hire them? Simply put, no. The last thing you want to be is a guinea pig for someone’s new service experiment. When outsourcing accessibility testing it is important to find a proper testing fit. Interview the potential candidate company.

 
  • Do they have experience with applications like yours?
  • What are their specific quality assurance credentials?
  • What steps will they use to develop a test plan?

Let’s say you have an application that allows people to go online, select a book, add it to a shopping cart and then pay for it as a final step. In this case, if the outsourcing company has never tested a retail site or handled any sort of credit card processing would they be a good fit for you? They may be, but you will find yourself having to manage the entire test team. This may be fine but you just added something to the scope.

 
The Number One Cause of Outsourcing Failure

You would think that companies that do testing would be good at defining and accepting scope and in most cases you would be correct. However, there are many companies so anxious to “win the business” that they will take any job even if they are not qualified. You may find companies that have no real experience so when they underestimate the job; they immediately start demanding more money, or even worse, those that intentionally underpriced the job with the intent of increasing the price once it is too late for you to find another option. While you might think that after defining scope you are protected from these kinds of issues. But that is not the case. You need to also put in a process that deals specifically with change requests and essentially change in scope. Tip, ask the vendor to provide you with their boiler plate scope change request document, again if they have no experience with this it will fall to you to define this and this should be defined as part of the contract documents. Some central aspects of a change request document that you should be concerned with include:

 
  • Who has authority to request a change
  • What is required in a change request, details, scope, contract
  • What are the outsource companies responsibilities
  • What are the companies responsibilities
  • Is the vendor required to suggest changes to the project if they think it would be beneficial and if so what is the process
With the change request process ironed down and the scope ironed down you are now poised to have a successful project. There is just one step left, selecting the vendor. Appendix B lists some questions that can help you select a vendor and in the end you will also have to consider your own budget for the project.

 
Final Thoughts

Regardless of the project size, when soliciting proposals you cannot simply accept the lowest bid, even the government has processes to prevent them from having to go with the lowest bid. The proposal price should be the last thing you look at. You need to evaluate all samples carefully. Evaluate the proposal, is it clearly and professionally written? Do the project plans have spelling errors or do they indicate no real experience of managing resources or a technical team. Then make your decision based on quality, experience and cost. Beware of people that use Microsoft project like it is Microsoft Excel and then they do not even use the spell check feature. If after reviewing any vendor and you are unsure as to their capabilities you should feel comfortable asking them to prepare a sample report for you. And finally, start small; there is nothing wrong with giving a few different companies a small part of the project to see if they are capable and professional. It may seem like a paid trial of the company, but it is something you will not regret as this will give you actual experience with the companies capabilities and their style. Additionally tie all payment to project deliverables, avoid having to give more than a 10% project startup fee on any project regardless of the size. And remember get everything in writing. Good luck!

A Horror of an Office is now available on LuLu, Print and Download

A Horror of an office Print
A Horror of an Office download


252 pages

Saturday, December 5, 2009

Security testing - Excerpt from Rob Yonaitis' new Book Understanding Accessibility, An accessibility testing primer

Security testing


Security testing - Security testing is a system test that validates the programs specific access / authentication logic works properly. Different levels of access may see different screen, different messages and different user interfaces.

The accessibility tester needs to be aware of these specific functions and how different user access levels may result in different output from the program. Additionally all prompts, warning and notifications must be tested to prove they are accessible.

For Example:

Every thirty days the user must change their password. If they have not done so an image is presented on the screen that says today is your last chance to change your password. The image unfortunately has no alternative text and the use has image x. The user continues and tomorrow cannot access the system. By being away of this security functionality the Accessibility tester can develop a proper test case.

Wednesday, November 11, 2009

Who needs Valid ID ATTRIBUTES or one HTML tag anyways?

I was checking out all my twitter follows today and I noticed one link by John Foliot, a person whom I respect.

RT @johnfoliot: & If your (X)HTML is valid & your ARIA is valid, your document is valid, so don't worry about it. http://ow.ly/BdiE  (@jared_w_smith) #a11y
This link is an interesting argument around ARIA and Standards an argument that I am not personally for or against. However, I am concerned with what will be the unintended consequences of institutional violation of long time standards. Let’s take the conversation a bit further. What I read the conversation to be saying is (and I may be wrong):
“If the page still renders and the same page provides valued information to some group and since there is no harm and there is value in the standard not being complied with then the standard, which is not valid, should not be marked invalid - so instead of failing the page let’s just give a warning.”
To the readers of this blog post, I have to ask, have you ever cursed a software company for extending capabilities of their browser or web based software because it was outside of standards, do you have blogs that say right on top, best viewed with a standards compliant browser. Should the text on your blogs template read “best viewed with standards that I agree with and with exceptions that a respected group of people said are fine?” How do you suggest that I teach this deviation?

Let us take a walk on the web, shall we? So I travel out to a public web site, for a company that I founded in 1996, and I see they are registering people for a big seminar, an accessibility roundtable with distinguished experts, this is good right?

Let us look at this page: https://www.hisoftware.com/SP2010roundtable.htm  .

  • Let us ignore that it does not validate for a minute (because the importance of validation is in question, isn’t it)
  • Let us also ignore that it has two HTML Tags (same pesky validation issue that does not impact rendering)
  • Let us forget the 52 Errors (same pesky validation issue)
  • Let us also forget that this page is a tribute to “Spacer.Gif”
Instead, let us forget these things because our argument is that there is no harm therefore no foul. To repeat the argument posed: if the page renders properly and meets the site or application developer technology baseline, sorry to drag that specter out, someone has to say it (I know I am taking it a step further and I will show you why in a moment, please be patient), then why not issue a warning and call this page valid! Let us look a little further: on this same page, put forward by the distinguished experts (in this case excuse the pun) we find an interesting error that would have been caught by AccVerify (IMHO – the gold standard). The error is with the ID’s on the form. IDs are case sensitive (same pesky validation issue); therefore, what do you think happens in the following case”
//Please excuse the stripped HTML for blog purposes.

label for='name'

Name/label td width="50%" height="25" class="bodyText" align="left"

input id="Name
Now in IE 8 if you click on that label (Name) the input field is selected, now why not, the attribute requirement is silly, case sensitive? Why not change all validators to warn, most miss it anyway?

Now in the Latest; Firefox, Opera, and Safari (please forgive the lack of versions) nothing happens when tabbing or selecting. So why am I even looking at different browsers? I should not have to look at different browsers if this page was developed to standards. Is this not the purpose of the standards to begin with, standards are not for fun, are they?
Look at this table:

table role="presentation" width="100%" border="0" align="center" cellpadding="0
Is there not some valid attribute that can be used on this static HTML page to let a user of “AT” know that this table is for layout only? Shouldn’t the people that cry out for the need of standards compliance be the ones that defend them versus making excuses as to why it is OK to break them?

Why not just add stuff, separate from the standards and let AT Vendors take advantage of the new stuff! ON HTML5: why fight about accessibility attributes or features in the standard? Just put them in and User Agents and screen readers will take advantage of all the new stuff, even if they are not in the standard!

I know I take this a bit too far, but I have to remember my Dante; what was it Canto XXIII, Eighth Circle, Sixth Pit? OK I Digress, I have to ask my readers - when I go out and mentor young engineers, and they say “It works in IE, Firefox and Opera, which is all I code for” - what should I say to them? Should I tell them that the standards are arbitrary and that if it renders for their technology baseline, that they have selected for their project, then they are good? What do you think?

Download PDF of the Post: http://www.yonaitis.com/WhoneedsValid.pdf

Tuesday, October 27, 2009

Rob Yonaitis, HiSoftware founder, writes his 1st Fiction Novella

A Horror of an office by Rob Yonaitis
download complete book:


A Horror of an Office
A Novella by Robert B. Yonaitis

Synopsis
I am working on my new novella “A Horror of an Office” this is a fictional tale that is meant to be funny while combining urban legend and other fictional tales to provide a worst case office situation. The company name is Blue Marine Software.

Characters

William – The Founder of the company is a Scientist with Degrees in Computer Science and Aeronautical Engineering, specializing in Aeronautics and Space Studies. William is an amateur artist and a Pilot who has authored several books. William dedicates his free time to different local charities and is a supporter of the arts.

Kent – The CEO hired by William before he stepped down from control of the company to focus on his inventions and improve his flight skills. A man of short stature that has to put his name on everything that he did not write and then frames it and hangs it in the office. He can be seen at any moment sitting in his office staring at his pc with a blank expression on his face. The type of a man that would pay a marketing company to pose a photograph to make him appear to be the tallest when he was the shortest.

TC – The CFO who always works with Kent (Partners in Crime). Whenever he talks he tells the young boys in the company, his favorites, that if they work hard they can get a watch or car like his. At company parties he brags how this company paid for his watch, this company paid for his car, and this company paid for his pants. TC can always be counted on to lie to investors or shareholders!

Johnnie – Johnnie is the VC that recommended Kent. Johnnie can normally be seen with a glass of Vodka in his hand and if there is no vodka in the office TC will send out one of his boys to the package store to get it fast. Johnnie has no idea how much money Kent and TC Steal from him nor does he know how often they lie to him.

Lilly – Lilly is the salesperson that cannot sell and has one claim to fame, she did not testify against TC and Kent at the last company that they were at together “Moon Crater Software.” Lilly has been the salesperson of the quarter for three straight quarters, not because she could sell but because Kent assigns other peoples sales to her as payback.

Sub-Stuff
The company is partnered with the world’s largest software company “Acme Software” and Kent is doing all he can to steal from them in every way possible. The story takes place in 2012 and as I have chapters ready I will post them. I look forward to all feedback.