The Reference Ontology on Software Testing (ROoST) is partially integrated to SEON, representing the activities, artifacts and stakeholders involved in the Software Testing Process, considering only dynamic tests. Since the testing process is a technical process in the software development, ROoST shares concepts with other SEON networked ontologies.
Networked ontologies used by ROoST:
Ontology | Relation | Integration Level |
SPO - Software Process Ontology | Most of ROoST concepts extend concepts from SPO. | High |
Figure 1 presents the packages of the ROoST Modularization.
Figure 1. ROoST Modularization.
Figure 2 presents the conceptual model of the Testing Artifacts subontology.
Figure 2. Testing Artifacts conceptual model.
Testing activities handle with several artifacts, which can present different nature and composition. Among the documents produced by testing activities the following detach: Test Plan, Test Suite, Test Case, Test Result and Test Analysis Report. Test Plans describe how the tests should be performed, providing a roadmap for the testing activities. Test Suites are collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours. Test Cases are most important testing artifacts. They describe the input data, expected results, steps and general conditions for testing some situation regarding a Code To Be Tested. Some test cases are implemented by some Test Code, such as Test Driver, Stub or Test Script. Test Results documents the execution of a test case, and includes the Test Case Actual Result, and optionally a Test Incident Report. Finally, Test Analysis Reports documents the analyses done from a set of Test Results.
Figure 3 presents the conceptual model of the Testing Stakeholders subontology.
Figure 3. Testing Stakeholders conceptual model.
The main stakeholders involved in testing activities include Test Manager, Test Case Designer and Tester. These are roles that can be played by people, teams or even by organizations.
Figure 4 presents the conceptual model of the Testing Techniques subontology.
Figure 4. Testing Techniques conceptual model.
For designing test case, several testing techniques apply. These testing techniques are classified according to Testing Technique Types, such as black-box, white-box, model-based and mutation testing techniques. Some Testing Techniques Types applies only to certain testing levels. For instance, white-box testing typically applies to unit testing.
Figure 5 presents the conceptual model of the Testing Process and Activities subontology.
Figure 5. Testing Process and Activities conceptual model.
Testing Process is a Performed Process with the purpose of conducting dynamic software testing related activities, from planning to results analysis. It is composed of two main activities: Test Planning and Level-based Testing, performed respectively by the Test Manager and the Tester stakeholders. The former activity creates a Test Plan as a roadmap for conducting the following testing activities (such as testing design and execution). The Level-based Testing can be classified according to the testing level into: Unit Testing, Integration Testing and System Testing (among other). Level-based Testing activities are composed of the following testing activities: Test Case Design, Test Coding, Test Execution and Test Result Analysis. Thus, the testing activities can be performed according to the different testing levels (e.g. a Test Case Design focusing on Integration Testing). Concerning artifacts produced and used: (i) Test Case Design uses a Test Case Design Input (e.g. a requirements or a design document) for designing a Test Case containing the information for running the test; (ii) Test Coding creates Test Code implementing the defined Test Cases; (iii) Test Execution uses the Test Code for effectively testing the established Code To Be Tested, producing Test Results relative to the Test Cases; and, finally, (iv) Test Result Analysis produces a Test Analysis Report by examining the Test Results and determining whether the tests have been successful in identifying defects.
Figure 6 presents the conceptual model of the Testing Environment subontology.
Figure 6. Testing Environment conceptual model.
The testing environment is the collection of the Test Hardware and Software Resources that are used to perform the testing activities of a Project. Among the software tools that are used to support testing activities, there are Test Management Tools and Incident Management Tools.
The following table shows the definitions for ROoST concepts.
Concept | Definition |
Black-box Testing Technique
|
Testing technique that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary)
|
Code To Be Tested
|
Portion of Code (software module) to be tested by a Test Case.
|
Incident Management Tool
|
Software product that manages and maintains lists of issues as needed by an organization (adapted from Wikipedia).
|
Incident Report
|
Report describing the occurrence, nature, and status of an incident (adapted from ISO/IEC/IEEE 29119-1:2013 Software and systems engineering--Software testing--Part 1: Concepts and definitions, 4.18).
|
Integration Testing
|
Level-based Testing focusing on larger components, ensuring that a collection of units functions as desired.
|
Level-based Testing
|
Composite Performed Activity consisting of testing activities and classified by the different levels they can be performed.
|
Model-based Testing Technique
|
Testing technique in which the test cases are derived from a model that describes the system under test.
|
Mutation Testing Technique
|
Testing technique in which two or more program mutations are executed using the same test cases to evaluate the ability of the test cases to detect differences in the mutations (ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary)
|
Stub
|
Skeletal or special-purpose test code substituting a software module, used to test a module that calls or is otherwise dependent on it (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering - Vocabulary)
|
System Testing
|
Level-based Testing focusing on the behavior of the entire system, ensuring that it is in conformance with its requirements.
|
Test Analysis Report
|
Document containing the analysis of Test Results and how they succeed.
|
Test Case
|
Document containing the input data, expected results, steps and general conditions for testing some situation regarding a Code To Be Tested.
|
Test Case Actual Result
|
The actual result obtained from the execution of a test case.
|
Test Case Design
|
Simple Performed Activity for designing the Test Cases to be executed, supported by input artifacts.
|
Test Case Design Input
|
Some Artifact defined as an input for the Test Case Design.
Ex.: Requirements Document, Design Document, Code. |
Test Case Designer
|
Stakeholder responsible for designing test cases and analysing test results.
|
Test Case Expected Result
|
The result expected from the execution of a test case.
|
Test Code
|
Code produced for implementing a Test Case.
|
Test Coding
|
Simple Performed Activity for implementing the Test Cases as Code artifacts (Test Code) to be used during Test Execution.
|
Test Driver
|
Software item used to invoke a module under test and, often, provide test inputs, control and monitor execution, and report test results (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary).
|
Test Execution
|
Simple Performed Activity for effectively executing the Test Cases, by running the Test Code and producing the Test Results.
|
Test Hardware Resource
|
Hardware Resource that comprises the Testing Environment of a Project.
|
Test Incident Report
|
Report describing any event that occurs during the testing process which requires investigation (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary)
|
Test Input
|
Descriptions of the parameters and values to be used in a test case, in order to produce an expected result.
|
Test Management Tool
|
Software Product used to store information on how testing is to be done, plan testing activities and report the status of testing activities (adpated from Wikipedia).
|
Test Manager
|
Stakeholder responsible for planning and managing the testing activities.
|
Test Plan
|
Document describing how the tests should be performed, providing a roadmap for the testing activities.
|
Test Planning
|
Composite Performed Activity for planning how the testing activities should be performed, and documenting it in a Test Plan.
|
Test Result
|
Document containing the actual results and identified issues relative to a Test Case execution.
|
Test Result Analysis
|
Simple Performed Activity for analyzing Test Results to determine whether or not tests have been successful in identifying defects.
|
Test Script
|
Short program written in a programming language used to test part of the functionality of a software system. Test scripts written as a short program can either be written using a special automated functional GUI test tool or in a well-known programming language (Wikipedia).
|
Test Software Resource
|
Software Resource that comprises the Testing Environment of a Project.
|
Test Suite
|
A collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours (Wikipedia).
|
Tester
|
Stakeholder responsible for coding and executing test cases.
|
Testing Environment
|
The set of resources (Hardware and Software Resources) that are used to perform testing activities of a Project.
|
Testing Level
|
Levels in which testing activities are performed, such as Unit, Integration and System testing.
|
Testing Process
|
Specific Performed Process for planning and executing the dynamic testing activities for the software in development.
|
Testing Technique
|
Technique for designing test cases. It can be Black-box, White-box, Model-based or Mutation testing techniques, among others.
|
Testing Technique Type
|
Major types of testing techniques such as black-box, white-box, model-based and mutation testing techniques.
|
Unit Testing
|
Level-based Testing focusing on the unit or the individual components that have been developed, ensuring that the unit functions correctly in isolation.
|
White-box Testing Technique
|
Testing Technique in which the tests are derived from an examination of the structure of the code to be tested (adapted from ISO/IEC/IEEE 29119-1:2013 Software and systems engineering--Software testing--Part 1: Concepts and definitions)
|
<<subkind>> Black-box Testing Technique |
<<role>> Code To Be Tested |
Test Execution (0..*) uses (1..*) Code To Be Tested
Test Case (1..*) tests (1..1) Code To Be Tested
<<subkind>> Incident Management Tool |
<<subkind>> Incident Report |
<<event>> Integration Testing |
<<event>> Level-based Testing |
Level-based Testing <>-- Test Case Design
Level-based Testing <>-- Test Coding
Level-based Testing <>-- Test Execution
Level-based Testing <>-- Test Result Analysis
Level-based Testing (1..*) uses (1..1) Test Plan
Level-based Testing uses Test Hardware Resource
Level-based Testing uses Test Software Resource
Level-based Testing (0..*) instanceOf (1..1) Testing Level
Testing Process <>-- Level-based Testing
<<subkind>> Model-based Testing Technique |
<<subkind>> Mutation Testing Technique |
<<subkind>> Stub |
<<event>> System Testing |
<<subkind>> Test Analysis Report |
Test Analysis Report (0..*) analyzes (1..*) Test Result
Test Result Analysis (1..*) creates (1..1) Test Analysis Report
<<subkind>> Test Case |
Test Case (1..*) tests (1..1) Code To Be Tested
Test Case (1..*) <>-- (1..*) Test Case Expected Result
Test Case (1..*) <>-- (1..*) Test Input
Test Case (0..*) built applying (0..1) Testing Technique
Test Case Design (1..1) creates (1..*) Test Case
Test Suite (0..*) <>-- (2..*) Test Case
Test Result (0..*) relative to (1..1) Test Case
Test Coding (0..*) uses (1..*) Test Case
Test Execution (0..*) executes (1..*) Test Case
Test Code (0..*) implements (1..*) Test Case
<<subkind>> Test Case Actual Result |
Test Case Actual Result (0..*) compared to (1..1) Test Case Expected Result
Test Result (1..*) <>-- (1..*) Test Case Actual Result
<<event>> Test Case Design |
Test Case Design (1..1) creates (1..*) Test Case
Test Case Design (1..*) uses (1..*) Test Case Design Input
Test Case Design adopts Testing Technique
Level-based Testing <>-- Test Case Design
Test Case Designer is in charge of Test Case Design
<<rolemixin>> Test Case Design Input |
Test Case Design (1..*) uses (1..*) Test Case Design Input
<<role>> Test Case Designer |
Test Case Designer participates in Test Planning
Test Case Designer is in charge of Test Case Design
Test Case Designer is in charge of Test Result Analysis
<<subkind>> Test Case Expected Result |
Test Case Actual Result (0..*) compared to (1..1) Test Case Expected Result
Test Input (1..*) is expected to produce (1..1) Test Case Expected Result
Test Case (1..*) <>-- (1..*) Test Case Expected Result
<<subkind>> Test Code |
Test Code (0..*) implements (1..*) Test Case
Test Coding (1..1) creates (1..*) Test Code
Test Execution (0..*) uses (1..*) Test Code
<<event>> Test Coding |
Test Coding (1..1) creates (1..*) Test Code
Test Coding (0..*) uses (1..*) Test Case
Level-based Testing <>-- Test Coding
Tester is in charge of Test Coding
<<subkind>> Test Driver |
<<event>> Test Execution |
Test Execution (1..1) creates (1..*) Test Result
Test Execution (0..*) uses (1..*) Test Code
Test Execution (0..*) executes (1..*) Test Case
Test Execution (0..*) uses (1..*) Code To Be Tested
Level-based Testing <>-- Test Execution
Tester is in charge of Test Execution
<<role>> Test Hardware Resource |
Level-based Testing uses Test Hardware Resource
Testing Environment (1..*) <>-- (1..*) Test Hardware Resource
<<subkind>> Test Incident Report |
Test Result (0..*) <>-- (0..*) Test Incident Report
<<subkind>> Test Input |
Test Input (1..*) is expected to produce (1..1) Test Case Expected Result
Test Case (1..*) <>-- (1..*) Test Input
<<subkind>> Test Management Tool |
<<role>> Test Manager |
Test Manager is in charge of Test Planning
Test Manager participates in Test Result Analysis
<<subkind>> Test Plan |
Level-based Testing (1..*) uses (1..1) Test Plan
Test Planning (1..*) creates (1..1) Test Plan
<<event>> Test Planning |
Test Planning (1..*) creates (1..1) Test Plan
Test Manager is in charge of Test Planning
Test Case Designer participates in Test Planning
Testing Process <>-- Test Planning
<<subkind>> Test Result |
Test Result (0..*) relative to (1..1) Test Case
Test Result (0..*) <>-- (0..*) Test Incident Report
Test Result (1..*) <>-- (1..*) Test Case Actual Result
Test Analysis Report (0..*) analyzes (1..*) Test Result
Test Result Analysis (0..*) uses (1..*) Test Result
Test Execution (1..1) creates (1..*) Test Result
<<event>> Test Result Analysis |
Test Result Analysis (0..*) uses (1..*) Test Result
Test Result Analysis (1..*) creates (1..1) Test Analysis Report
Level-based Testing <>-- Test Result Analysis
Test Manager participates in Test Result Analysis
Test Case Designer is in charge of Test Result Analysis
<<subkind>> Test Script |
<<role>> Test Software Resource |
Level-based Testing uses Test Software Resource
Testing Environment (1..*) <>-- (1..*) Test Software Resource
<<subkind>> Test Suite |
Test Suite (0..*) <>-- (2..*) Test Case
<<role>> Tester |
Tester is in charge of Test Execution
Tester is in charge of Test Coding
<<collective>> Testing Environment |
Testing Environment (1..*) <>-- (1..*) Test Hardware Resource
Testing Environment (1..*) <>-- (1..*) Test Software Resource
Testing Environment (0..1) defined for (1..1) Software Project
<<2ndOT>> Testing Level |
Level-based Testing (0..*) instanceOf (1..1) Testing Level
Testing Technique Type (0..*) applies to (0..*) Testing Level
<<event>> Testing Process |
Testing Process <>-- Test Planning
Testing Process <>-- Level-based Testing
<<subkind>> Testing Technique |
Testing Technique (0..*) instanceOf (1..1) Testing Technique Type
Test Case Design adopts Testing Technique
Test Case (0..*) built applying (0..1) Testing Technique
<<2ndOT>> Testing Technique Type |
Testing Technique Type (0..*) applies to (0..*) Testing Level
Testing Technique (0..*) instanceOf (1..1) Testing Technique Type
<<event>> Unit Testing |
<<subkind>> White-box Testing Technique |