Applications Services Blog
Get the latest thought leadership and information about the role of Applications Services in an increasingly interconnected world at the HP Blog Hub.

Six steps for removing subjectivity from testing system requirements


teaser.jpgBy: James (Jim) R. Hughes, Global Business Analysis Capability Leader, Hewlett Packard Company


Author’s note: The Focus on Fundamentals blog series addresses the challenges that limit applications solution delivery success. See the list of previous topics in this series at the end of this blog.


Good requirements are the foundation of every successful project. If we cannot specify what an application must do, then it will be difficult for developers to build software which meets the requirements. In addition, it has long been known that correcting a defect during requirements analysis is orders-of-magnitude less costly than correcting the defect if it is detected late in the development life-cycle or when the application is in production. Therefore, we should apply discipline to ensuring that our requirements are complete and accurate—in other words, we should test them.


Testing system requirements is different from testing software. When we test software we can test it against a set of requirements to ensure that it produces the results expected. However, when we test requirements we have not yet produced a fleshed-out baseline against which to compare the requirements—at best, we have a set of higher level business requirements or scope statements against which we can map the system requirements. So we need to utilize a different strategy for testing requirements. This blog will consider only how to test a set of narrative requirements which are used to specify what a system should do. We will not consider how to assess the quality of Use Cases or User Stories (although they can also be tested).


There are many factors which could be assessed to test requirements. However, in our Business Analysis Capability at HP, we determined that many factors are correlated and redundant. We settled on the following basic criteria to test requirements statements:


  • Clear – An individual requirement is clear (unambiguous) if it addresses a single need, does not include multiple scenarios, has no ambiguous terms, and has no implied or hidden assumptions.
  • Complete – A requirement is complete if it has no missing information (including attributes such as priority, release, risk level, constraints, etc.), has precise calculations (where applicable, such as in a business rule), and can be interpreted only one way.
  • Consistent – A requirement is consistent if it does not contradict any other requirement or compromise the possibility of fulfilling another requirement, and complements or completes a set of related requirements.
  • Verifiable – A requirement is verifiable if has an acceptance criterion and its fulfillment can be tested using one of the standard system testing approaches to produce an unambiguous result.
  • Solution Independent – A requirement is solution independent if it states what is required but not how the requirement should be met, and does not impose unnecessary or non-essential design or implementation constraints.
  • Traceable – A requirement is traceable if it can be mapped clearly to a business requirement or scope statement.

We use these criteria to test the quality of a set of requirements during requirements review and validation, which takes place at the end of the detailed requirements phase but before the requirements are delivered to the customer for sign off. We also use them to assess the quality of a set of requirements provided by a customer or third-party. We produce a Requirements Quality Index (RQI) which allows us to present an objective assessment of the quality of each requirement and of the set of requirements; thereby eliminating subjective opinions about the quality of the requirements from the review process.


This approach to verifying the quality of requirements moves testing earlier in the software development life cycle and helps to reduce development costs.


Other blogs by Jim Hughes:


Related links: 


About the Author


Image for Blog.jpgJames (Jim) R. Hughes, Global Strategic Capability Leader, Hewlett Packard Company

Jim has been with HP for 33 years and currently leads a global Strategic Capabilities Management team, with a specific focus on Business Analysis and Configuration Management. Jim also manages a team within the US State, Local, and Education division of HP. He was a member of the IIBA committee that created a Business Analyst Competency Model and he participated in the development of IEEE standards. Jim graduated from Harvard University, the University of British Columbia, and Ottawa Theological Hall. He currently lives in Toronto, Canada.

Srinivasan Desikan(anon) | ‎07-19-2013 05:16 PM

Excellent points Jim. In specific, I very much liked what you said  in "Testing system requirements is different from testing software".  I agree fully here. It is lot more easier to test a software than testing a requirement. Typically, it requires a seasoned BA and experienced Testers to work together.


I also have a point to add to your traceability factor.  A requirement should have both upward and downward traceability. On upward, it should map to scope or business requirement and for downward traceability t should map to design,code, test cases and defects.  If not design & code atleast each requirement should have traceability to test cases , to analyse the impact/priority of defects.  When in question, priority of defect is same as the priority the requirement it impacts. Makes life so easy for test engineers  

James_Jim_R | ‎07-22-2013 02:33 PM

Good point Srini. I guess, I assume, but should spell out, that traceability is bi-directional. As you say 'upward' for scope management. Downward is also for impact assessment. If you change a requreiment you want to be able to determine what objects instantiate that requirement (code, test cases, user documentation, etc.).

Thanks for reading. :smileyhappy:

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the community guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
About the Author
Jim has been with HP for 33 years and currently leads a global Strategic Capabilities Management team, with a specific focus on Business Ana...
About the Author(s)
  • Business Development leader for HP’s Testing Practice in Americas and TaaS Global Capability Leader
  • Anthony works in the ES Apps Enterprise APJ . He is a self-motivated and well-disposed individual. He is a certified SAP FICO consultant. He has over twenty years industry experience in business transformation and Information Technology consulting (SAP). With excellent, interpersonal and communication skills, Anthony is able to influence and motivate at all levels. An empathic team player, He derives fulfilment in contributing towards others' development. Anthony possesses good analytical and problem-solving skills. He is a quick thinker, remains calm under pressure, handles multiple tasks, and accomplishes objectives with excellent time management. He is flexible and accustomed to meeting high standards and deadlines. Anthony possesses practical knowledge of business processes and concepts and has a keen eye for detail. He is aware of the importance of an effective system and quality management.
  • Bryan Coapstick is the Director of Mobile Innovation at HP. In this role, he is responsible for driving strategic mobility initiatives and key industry partnerships to enable clients the ability to leverage the mobile channel to effectively reach their customers. Bryan’s expertise lies with several Fortune 200 companies in both public and private sectors that focus on business technology and emerging strategies. Bryan demonstrates innovative leadership though public speaking, panel discussions and blogging on the various ways mobility is changing our daily lives.
  • Benjamin Romberg has over twenty five years experience in the IT industry, the last twelve of which have been as a testing professional. He has experience as a test analyst, performance tester, automation specialist, test lead, test manager, test consultant and test executive. He has worked on all types of testing engagements ranging from small web based applications through to core system upgrades. Benjamin holds a Bachelor’s Degree in Genetics, a Post Graduate Degree in Software Engineering and an Executive Masters in Business Administration. Benjamin now leads HP’s Testing and Quality Assurance Practice in Asia Pacific and Japan.
  • Over the past 5 years, I have participated in the different stages of delivering Microsoft Enterprise Applications to customers as large as 40.000 concurrent users across Europe with a focus on the public and call center sector. During this period, I worked in numerous technical (lead) and business consultant roles. Using this experience, I am currently a certified Microsoft Dynamics CRM 2011 expert which specializes in guiding enterprise customers and implementation teams through the process of implementing Microsoft Dynamics CRM 2011.
  • Craig has been with HP for 10 years working on the solutioning side as a Solution Architect. Primarily works with Microsoft technologies and has for the better part of 16 years. This past year Craig has worked mostly with Windows AZURE and the mobility side of Windows Phone 7/8 and Android.
  • Master Business Consultant and Distinguished SE with 35+ years of IT experience, specializing in SOA and Integration.
  • Daniel Amor (EMEA, AMOD, Cloud, Portals, Web Apps and SOA domain expert): Daniel has designed, won, led and delivered large-scale, complex applications-related projects by acting as a trusted advisor to clients. Daniel conducts regular training and client-facing sessions, and he has authored six books and numerous articles in European and US-based magazines and newspapers on IT change. He is a regular lecturer and he speaks on Portals, E-Business and Cloud.
  • Global HP Microsoft Business Applications Practice. Microsoft Dynamics CRM subject matter expert. Based in Manchester, United Kingdom. @DarrenCRM on Twitter
  • Solutions Architect working at HP since 2002 with focus on Applications Modernization, located at Rio de Janeiro, Brazil
  • Managing Consulting specialising in Business Transformation and Value Management for Defence, Healthcare, Public Sector, Criminal Justice and Telecommunications.
  • Guido Arndt is since May 1997 business consultant and project manager @ Hewlett Packard Enterprise Services. Main focus is strategic and organizational alignment of logistics, sales and production processes in the automotive and manufacturing industry as well as business performance projects in finance and telecommunication industry. Guido has experience as change agent and transformation manager in people-driven merger & aquisition projects focusing government, consulting firms, manufacturing plants and consumer brands.
  • Horia Slușanschi serves as an Agile Coach leading the HP Agile Mentoring Office and the HP Software Engineering Profession. He is passionate about helping teams and leaders to find joy in work in the pursuit of customer delight.
  • CISSP, Open CA Master Architect Technical Consultant with 28+ years of IT experience specializing in Application Security, Solution Architecture, Portals, Social Business, and Integration
  • Jim has been with HP for 33 years and currently leads a global Strategic Capabilities Management team, with a specific focus on Business Analysis and Configuration Management. Jim also manages a team within the US State, Local, and Education division of HP. He was a member of the IIBA committee that created a Business Analyst Competency Model and he participated in the development of IEEE standards. Jim graduated from Harvard University, the University of British Columbia, and Ottawa Theological Hall. He currently lives in Toronto, Canada.
  • SAP Technologist from APPS GD SAP Practice - Manila covering SAP NetWeaver EP, SAP Mobility and general SAP solutions.
  • WW product marketing manager for HP Applications Modernization and Transformation Consulting Services. 35+ years of IT technology and services experience.
  • Laurence has over 35 years experience in the IT industry and has spent 27 years delivering knowledge management (and before that became fashionable, Artificial Intelligence and knowledge-based systems) transformation projects to clients in Government, Industry, Manufacturing and Telecomms industries. He has developed and implemented knowledge management (KM) strategies for HP's clients and for business areas in HP (for BPO and for our Consulting business) and believes that KM strategy is essential for an effective KM practice.
  • . . Service Systems . .
  • Lori Lewis leads Global Digital and Integrated Marketing for the Applications and Business Services division in HP’s Enterprise Services. The organization is focused on providing clients with enterprise applications services and solutions to solve some of their most pressing business problems. For over 2 decades, she has championed innovative communication approaches to drive and grow market share and build motivated, empowered teams. She is a compelling, passionate ambassador of the art and science of marketing and the amazing results it can achieve when done with integrity and passion. Ms. Lewis’ credentials, track record, and enthusiasm are essential ingredients her team relies on as they work together to establish HP as the undisputed leader in information management and analytics services.
  • Microsoft Dynamics CRM solution architect, C# developer, MBA, husband, father, Auburn man. Follow me on Twitter @lucas_is.
  • CRM enthusiast, avid techie
  • More than 10 years of experience in developing effective and innovative marketing and communication strategies. Recently I have studied the effects of social media and disruptive technology on business, marketing, and culture. Follow me on Twitter: @micaelaraimondi
  • Agile Champion and coach. Passion in software development using lean and agile techniques. Software is a people business!!
  • Have 11 years of experience in SAP Consulting, Solutioning and Account management.
  • I have over 20 years of I.T. experience predominantely in applications development. In addition to my I.T. experience I spent 7 years as the Director of Store Services at Canada's third largest grocery retailer.
  • Shafraz Nasser is the Lead Microsoft CRM Architect with HP Enterprise Services in the South Pacific Microsoft Dynamics Practice. His focus is on selling, architecting and overseeing delivery of Microsoft Dynamics CRM based implementations within New Zealand.
  • Shefali Dua is leading Worldwide BPaaS Strategy & Solution Innovation team at HP Enterprise Services as the Solution Innovation Manager. Her focus is currently assisting in planning and delivery of industry solutions for verticals like social media, big data, mobility and security on the HP Enterprise Cloud offerings for CRM Technologies.
  • Steve Petruno is a Product Manager for HP’s Global Oracle Services with over 24 years experience in the Information Technology industry, project Management, Human Resources, Payroll and Benefits, ERP implementations, upgrades, acquisitions and divestitures.
  • I have worked for HP as a business consultant for seven years, providing process, strategy and business transformation advice in the Financial Services and UK Government sectors. As part of this I have used Lean and Continuous Improvement techniques to assess and improve business performance, ensure regulatory compliance and improve environmental sustainability.
  • Tom is the Global Product Marketing lead for HP's Cloud Applications Services. Tom has over 20 years of experience in information technology and a proven track record for implementing innovative client solutions in several industries including government, healthcare, and telecommunications. He thrives on developing services and solutions which result in business process improvement through greater systems efficiency.
  • Vijay Seetharaman is an Enterprise Architect within HP Enterprise Services - Industry Transformation Consulting practice, focusing on Cloud and Mobility

Follow Us