Hierarchical Testing – The Evolution of the Test Plan

The discussion of complex increases in complexity….
December 23, 2017
Planning, Scheduling and Executing – A Deeper Look [Part 1]
December 25, 2017

Hierarchical Testing – The Evolution of the Test Plan

In the beginning there was the computer and all of the programs ran there. The source code simply appeared, was compiled, and executed. Nobody was really sure how the software came to be except that there were the requirements and they had to be met. In this world, a separate group of people would evaluate the condition of the system by executing the test plan. If it passed, then the blessing was given; if not, then there was rejection and a return back to the analysts and programmers.

Around 35 years ago, this began to change. New methodologies and tools to support them began to appear.  Among the first was Source Control; the ability to track changes over time, compare versions,  and even revert back to a previous state.  The ability to correlate the “work to be done” with actual and specific changes to the code began to appear.  Instead of massive changes that took years to implement, shorter cycles began to become common. And yet, the test plan and the processes around it tended to remain largely unaffected.

Fortunately this has been starting to change over the past decade. As organizations move from “Gantt Chart” type schedules (which never were accurate) to “Backlog” driven development, there are many opportunities to make the formal testing cycles significantly more effective.

The remainder of this post will use terminology from Microsoft Team Foundation Server [TFS] and Team Services [VSTS] along with Scrum [such as Product Backlog Item], but only because some set of terms needed to be chosen. The concepts are universal and independent of platform, language, development tools, et. al.

We will start with a quick review of how work is planned using backlog based approaches.

Initially the “big items” are identified, these may take an extended period to fully achieve; they are commonly important to upper-management; some may not be started until significantly after they have been identified.  These items are referred to as “Epics”.

Each epic can be refined down to a number of smaller elements. At this level, there is an increment in functionality that is of interest to the consumer [customer, business user]. The degree to which each epic is stable and known. Typically epics of higher priority (therefore to be focused on sooner) are handled in a much more detailed manner than those that are further out. These items are referred to as “Features”.

Each feature can be refined down to even smaller increments such that each provides an atomic increment of value. As an example, an epic of “Establish e-Commerce” may have a feature of “Shopping Cart on Web-Site”. Individual items could then be “Add Item to Cart”, “Remove Item from Cart”, “Change Quantity of Item in Cart”, “View Catalog Page for Item in Cart”. During exercises that focus on this type of work breakdown, it is not uncommon to identify upwards of 20 distinct items within the feature. These items are referred to as Product Backlog Items [PBI].

We can now return to looking at testing. Each of the Product Backlog Items can (and should) be associated with one or more Test Cases as part of the Acceptance Criteria. The PBI is not “done” until it has been demonstrated that all of the Test Cases pass.  A first pass (general description) of each Test Case is created at the same time as the PBI is created, additional Test Cases may be added throughout the cycle as the PBI is further refined. At some point during the development the Test Case is likely to contain specific actions and expected results which can be performed manually or automated.

If the set of Test Cases is complete, then *any* code which passes all of the test cases is acceptable [passing, working, deployable, releasable] code. If it is determined that there is something wrong with the code from functional perspective, it is indicative of a missing (or incomplete) Test Case.

As more work is defined and completed, the number of Test Cases grows. Multiple reasons for grouping the tests for organizational purposes begin to appear, and these groupings are Test Suites. Suites may be nested (a Suite can contain one or more Suites).

For each key Release, a Test Plan is created which comprises of the relevant Suites.  Typically this is a clone of the Test Plan used in the previous Key Release, with the addition of new Suites and Test Cases [but also with the potential for removal of things that no longer apply].  When all of the items in the Test Plan pass, the system has been validated.

When this approach is taken, the Test Plan has been transformed into a dynamic artifact which is active for the entire duration of a release cycle. Developers working on code [because they are delivering a PBI – the only real reason code should be written] know exactly what the tests (Acceptance Criteria) for there work is, at the time they are working on it. With rich automation, the Test Plan can be executed nightly against the prior day’s work to mitigate risks and provide an updated view of progress.


  1. suachuathanhhoa.com chuyên cung cấp dịch vụ sửa chữa tại nhà ở thanh hóa

  2. StephenSet says:

    viagra cost viagra non prescription viagra

  3. Recurrent episodes of genital herpes Recurrences are usually less painful and shorter in duration than the first episode.
    does generic viagra work does generic viagra work

  4. Jamesteado says:

    cialis dosage 40 mg dangerous cialis daily use of cialis

  5. TravisEngip says:

    generic viagra india buy viagra generic viagra online for sale

  6. EddieNeade says:

    erectyle dysfunction best canadian online pharmacy pump for ed

  7. JamesInomi says:

    cvs viagra buy viagra viagra professional

  8. JamesInomi says:

    where can i buy viagra over the counter viagra canada buy viagra online canada

  9. Different viruses, bacteria or even parasites
    can cause gastrointestinal symptoms. https://genqpviag.com cipla generic viagra

  10. RaymondGok says:

    Online Casino USA 2020 https://slot-profit.com/ No Deposit Bonuses for US Players 2020. Bonus 20 Free Spins Welcome Bonus 400% up to 4000$. Start with a $20 Free Chip.Welcome Bonus 250% up to 1000$

  11. Carlosbaine says:

    Преимущества онлайн казино очевидны для всех, кто хотя бы раз пробовал играть в интернет казино. Подборка лучших бездепозитных бонусов и фриспинов за регистрацию в онлайн казино 2020 года – https://all.casino-profit.pro/freespins.html

  12. Carlosbaine says:

    Список лучших бездепозитных бонусов за регистрацию с выводом для новых игроков из России и с других стран СНГ. Вам не обязательно тратить время на поиск бонусов без депозита или фриспинов за регистрацию на других сайтах и форумах. Подробную информацию о лучших бездепозитных бонусах казино РФ, Вы найдёте на сайте – https://all.casino-profit.pro/casino/rub.html

  13. RaymondGok says:

    Профессиональная Реклама Серых тем, Казино онлайн, Casino online, Gaming. Звоните: Telegram @evg7773

  14. Carlosbaine says:

    Обеспечиваю Заработок производителям, благодаря рекламе в Pinterest https://youtu.be/b_i8uomkv4U от 7000 до 100 000 usd в месяц в Etsy

  15. RaymondGok says:

    Besame Mucho (Lyrics), Latino music https://youtu.be/9Qyvqf5-nZs Сover by Elona Krasavtseva. Pop opera songs. Music arrangement of Andrea Bocelli. Elona Krasavtseva is an award-winning singer and songwriter from Moscow, Russia. She sings classical, traditional, and original songs in different European languages

  16. RaymondGok says:

    Как и обещал, запускаю конкурс на 10 призов по 0,001 BTC среди подписчиков! Подписывайся на канал https://t.me/neweconomytime – узнай условия и удачи!

  17. Carlosbaine says:

    Балясины для лестниц от производителя! https://balyasiny-optom.ru/ Изготовим балясины и ножки для мебели любой формы: точеные, граненые, витые, резные, комбинированные (дерево + металл) из сосны, дуба, бука, ясеня. Токарные станки с ЧПУ, быстрая и качественная обработка. Шлифовка под покраску

  18. Carlosbaine says:

    Элементы лестниц оптом, кухни на заказ, двери из массива дуба – https://www.ekolestnica.ru На сайте большой выбор изделий из дерева (дуб, бук, ясень, береза, сосна): балясины для лестниц, перила для лестниц, ступени для лестниц, двери из массива дуба, мебельный щит! На рынке более 15 лет, отгружаем товар в любые регионы!

  19. RaymondGok says:

    С 2012г. гарантированно раскручиваю продажи handmade в Etsy с помощью Pinterest. Это дает Заказчикам заработки от 7000 до 100 000 usd в месяц https://youtu.be/TT31RsoY5x0 Ведем аккаунты через пинтерест также в Ebay, Amazon, Shopify и др. Ручная работа, Цена 500 – 1000 usd за месяц

  20. Josephviono says:

    Элементы лестниц оптом, кухни на заказ, двери из массива дуба – https://www.ekolestnica.ru На сайте большой выбор изделий из дерева (дуб, бук, ясень, береза, сосна): балясины для лестниц, перила для лестниц, ступени для лестниц, двери из массива дуба, мебельный щит! На рынке более 15 лет, отгружаем товар в любые регионы!

Leave a Reply

Your email address will not be published. Required fields are marked *