Applications Services Blog
Get the latest thought leadership and information about the role of Applications Services in an increasingly interconnected world at the HP Blog Hub.

Test design contest (Part 1) -- The trend of software test design in Japan

By:  Tsuyoshi Yumoto, Test Consultant, Hewlett Packard Japan


Japan map.jpgThe biggest conference for software testing in Japan is JaSST (Japan Symposium on Software Testing) held by the Association of Software Test Engineering (ASTER) which is a NPO that promotes software testing in Japan. JaSST is held eight times a year in various locations throughout Japan. The largest of these conferences was held in Tokyo this March with 1,600 visitors. Conference events included introducing the latest testing trends in Japan, speeches by well-known test researchers and consultants, and the introduction of new published papers.


One of the events at JaSST Tokyo 2014 was the "Test Design Contest - Final Match, 2014” also hosted by ASTER. I was on the panel of judges for this contest and would like to tell you about it in this blog.


In this contest, the competing teams performed test analysis and design using the test basis that the host of the contest provided. Test basis means all of the documents from which the requirements of a system under test can be inferred, and on which the test cases are based. The contest consists of regional matches and a final match. The competing teams submitted artifacts about one month before the regional matches. Regional matches involved evaluation of an analysis method, a design method, and the quality of derived artifacts, and then a presentation. The teams that win the regional matches move onto the final match. The final match also involves evaluation of the derived artifacts and a presentation.


A similar contest was held in the past in which teams competed on the number of defects found in testing, but there were several challenges with this approach. Participants of the contest were unable to see:

  • if the test analysts had the right perspective to analyse the test basis,
  • if the artifacts of test analysis were well arranged for the following phase of test design, and
  • if the test design techniques were applied appropriately based upon the analysis.


At the contest, the judges only focused on the number of defects found by the teams, and not on the artifacts of test design. This time, at the Test Design Contest, artifacts of test analysis and design are the main focus of the judges. This is what makes the contest unique and interesting.


I am not familiar with the culture of other countries, but in Japan, test analysis and test design artifacts are seldom produced in development projects. Engineers who receive test case documents will not know how the test cases were designed and from where they were derived. Test designs then tend to remain within the designer’s head. By showing test designs to each other at the contest, we would like the participants to increase awareness that software testing is an engineering activity which requires an analysis and a design. It also provides an educational opportunity for software testing engineers regarding software test design that can improve their technical skills. These are the overall aims of the contest.


A record high of 27 teams participated in this year’s contest. Seven teams made it through the qualifying rounds and individually presented their test designs to the judges in the final round.


This time, the test basis subject was vending machine software. Each of the seven teams at the final had their own unique ideas; there was a team which designed tests focused on product and testing stakeholders, a team that designed tests by investigating quality characteristics and defects in the past, and a team that started a test design after arranging the functions of the product.Though the test object is the same, there are so many ways to approach it. The quality of the test design of the teams which made it to the final round was of a very high level, and the upward trend of the maturity of test design in Japan was clearly evident.


The artifacts of the teams that made it to the final round are available at this report site.This website is only in Japanese, so I will explain the product features of each team briefly in my next blog.


Related links:

About the Author


Yumoto - cropped.jpgTsuyoshi Yumoto, Test Consultant, Hewlett Packard Japan

Yumoto has been working within IT for over 20 years with a focus on professional testing. He consults with numerous clients on testing process improvements and maturization. He is also a recognised tool expert, having successfully deployed dozens of testing tools such as capture replay, test management, automation and performance tools. He is a member of ASTER and the Japan Software Testing Qualifications Board (JSTQB).

Leave a Comment

We encourage you to share your comments on this post. Comments are moderated and will be reviewed
and posted as promptly as possible during regular business hours

To ensure your comment is published, be sure to follow the community guidelines.

Be sure to enter a unique name. You can't reuse a name that's already in use.
Be sure to enter a unique email address. You can't reuse an email address that's already in use.
Type the characters you see in the picture above.Type the words you hear.
Showing results for 
Search instead for 
Do you mean 
About the Author

Follow Us
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.