SEON: Networked Ontology Specification

Reference Ontology on Software Testing (ROoST)

1. Ontology Description

The Reference Ontology on Software Testing (ROoST) is partially integrated to SEON, representing the activities, artifacts and stakeholders involved in the Software Testing Process, considering only dynamic tests. Since the testing process is a technical process in the software development, ROoST shares concepts with other SEON networked ontologies.

2. Related Ontologies

Networked ontologies used by ROoST:

3. Ontology Models

Figure 1 presents the packages of the ROoST Modularization.

Figure 1. ROoST Modularization.

3.1. Testing Artifacts

Figure 2 presents the conceptual model of the Testing Artifacts subontology.

Figure 2. Testing Artifacts conceptual model.

Testing activities handle with several artifacts, which can present different nature and composition. Among the documents produced by testing activities the following detach: Test Plan, Test Suite, Test Case, Test Result and Test Analysis Report. Test Plans describe how the tests should be performed, providing a roadmap for the testing activities. Test Suites are collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours. Test Cases are most important testing artifacts. They describe the input data, expected results, steps and general conditions for testing some situation regarding a Code To Be Tested. Some test cases are implemented by some Test Code, such as Test Driver, Stub or Test Script. Test Results documents the execution of a test case, and includes the Test Case Actual Result, and optionally a Test Incident Report. Finally, Test Analysis Reports documents the analyses done from a set of Test Results.

3.2. Testing Stakeholders

Figure 3 presents the conceptual model of the Testing Stakeholders subontology.

Figure 3. Testing Stakeholders conceptual model.

The main stakeholders involved in testing activities include Test Manager, Test Case Designer and Tester. These are roles that can be played by people, teams or even by organizations.

3.3. Testing Techniques

Figure 4 presents the conceptual model of the Testing Techniques subontology.

Figure 4. Testing Techniques conceptual model.

For designing test case, several testing techniques apply. These testing techniques are classified according to Testing Technique Types, such as black-box, white-box, model-based and mutation testing techniques. Some Testing Techniques Types applies only to certain testing levels. For instance, white-box testing typically applies to unit testing.

3.4. Testing Process

Figure 5 presents the conceptual model of the Testing Process and Activities subontology.

Figure 5. Testing Process and Activities conceptual model.

Testing Process is a Performed Process with the purpose of conducting dynamic software testing related activities, from planning to results analysis. It is composed of two main activities: Test Planning and Level-based Testing, performed respectively by the Test Manager and the Tester stakeholders. The former activity creates a Test Plan as a roadmap for conducting the following testing activities (such as testing design and execution). The Level-based Testing can be classified according to the testing level into: Unit Testing, Integration Testing and System Testing (among other). Level-based Testing activities are composed of the following testing activities: Test Case Design, Test Coding, Test Execution and Test Result Analysis. Thus, the testing activities can be performed according to the different testing levels (e.g. a Test Case Design focusing on Integration Testing). Concerning artifacts produced and used: (i) Test Case Design uses a Test Case Design Input (e.g. a requirements or a design document) for designing a Test Case containing the information for running the test; (ii) Test Coding creates Test Code implementing the defined Test Cases; (iii) Test Execution uses the Test Code for effectively testing the established Code To Be Tested, producing Test Results relative to the Test Cases; and, finally, (iv) Test Result Analysis produces a Test Analysis Report by examining the Test Results and determining whether the tests have been successful in identifying defects.

3.5. Testing Environment

Figure 6 presents the conceptual model of the Testing Environment subontology.

Figure 6. Testing Environment conceptual model.

The testing environment is the collection of the Test Hardware and Software Resources that are used to perform the testing activities of a Project. Among the software tools that are used to support testing activities, there are Test Management Tools and Incident Management Tools.

4. Concepts Definition

The following table shows the definitions for ROoST concepts.

Concept	Definition
Black-box Testing Technique	Testing technique that ignores the internal mechanism of a system or component and focuses solely on the outputs generated in response to selected inputs and execution conditions (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary)
Code To Be Tested	Portion of Code (software module) to be tested by a Test Case.
Incident Management Tool	Software product that manages and maintains lists of issues as needed by an organization (adapted from Wikipedia).
Incident Report	Report describing the occurrence, nature, and status of an incident (adapted from ISO/IEC/IEEE 29119-1:2013 Software and systems engineering--Software testing--Part 1: Concepts and definitions, 4.18).
Integration Testing	Level-based Testing focusing on larger components, ensuring that a collection of units functions as desired.
Level-based Testing	Composite Performed Activity consisting of testing activities and classified by the different levels they can be performed.
Model-based Testing Technique	Testing technique in which the test cases are derived from a model that describes the system under test.
Mutation Testing Technique	Testing technique in which two or more program mutations are executed using the same test cases to evaluate the ability of the test cases to detect differences in the mutations (ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary)
Stub	Skeletal or special-purpose test code substituting a software module, used to test a module that calls or is otherwise dependent on it (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering - Vocabulary)
System Testing	Level-based Testing focusing on the behavior of the entire system, ensuring that it is in conformance with its requirements.
Test Analysis Report	Document containing the analysis of Test Results and how they succeed.
Test Case	Document containing the input data, expected results, steps and general conditions for testing some situation regarding a Code To Be Tested.
Test Case Actual Result	The actual result obtained from the execution of a test case.
Test Case Design	Simple Performed Activity for designing the Test Cases to be executed, supported by input artifacts.
Test Case Design Input	Some Artifact defined as an input for the Test Case Design. Ex.: Requirements Document, Design Document, Code.
Test Case Designer	Stakeholder responsible for designing test cases and analysing test results.
Test Case Expected Result	The result expected from the execution of a test case.
Test Code	Code produced for implementing a Test Case.
Test Coding	Simple Performed Activity for implementing the Test Cases as Code artifacts (Test Code) to be used during Test Execution.
Test Driver	Software item used to invoke a module under test and, often, provide test inputs, control and monitor execution, and report test results (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary).
Test Execution	Simple Performed Activity for effectively executing the Test Cases, by running the Test Code and producing the Test Results.
Test Hardware Resource	Hardware Resource that comprises the Testing Environment of a Project.
Test Incident Report	Report describing any event that occurs during the testing process which requires investigation (adapted from ISO/IEC/IEEE 24765:2017 Systems and software engineering-Vocabulary)
Test Input	Descriptions of the parameters and values to be used in a test case, in order to produce an expected result.
Test Management Tool	Software Product used to store information on how testing is to be done, plan testing activities and report the status of testing activities (adpated from Wikipedia).
Test Manager	Stakeholder responsible for planning and managing the testing activities.
Test Plan	Document describing how the tests should be performed, providing a roadmap for the testing activities.
Test Planning	Composite Performed Activity for planning how the testing activities should be performed, and documenting it in a Test Plan.
Test Result	Document containing the actual results and identified issues relative to a Test Case execution.
Test Result Analysis	Simple Performed Activity for analyzing Test Results to determine whether or not tests have been successful in identifying defects.
Test Script	Short program written in a programming language used to test part of the functionality of a software system. Test scripts written as a short program can either be written using a special automated functional GUI test tool or in a well-known programming language (Wikipedia).
Test Software Resource	Software Resource that comprises the Testing Environment of a Project.
Test Suite	A collection of test cases that are intended to be used to test a software program to show that it has some specified set of behaviours (Wikipedia).
Tester	Stakeholder responsible for coding and executing test cases.
Testing Environment	The set of resources (Hardware and Software Resources) that are used to perform testing activities of a Project.
Testing Level	Levels in which testing activities are performed, such as Unit, Integration and System testing.
Testing Process	Specific Performed Process for planning and executing the dynamic testing activities for the software in development.
Testing Technique	Technique for designing test cases. It can be Black-box, White-box, Model-based or Mutation testing techniques, among others.
Testing Technique Type	Major types of testing techniques such as black-box, white-box, model-based and mutation testing techniques.
Unit Testing	Level-based Testing focusing on the unit or the individual components that have been developed, ensuring that the unit functions correctly in isolation.
White-box Testing Technique	Testing Technique in which the tests are derived from an examination of the structure of the code to be tested (adapted from ISO/IEC/IEEE 29119-1:2013 Software and systems engineering--Software testing--Part 1: Concepts and definitions)

Detailed Concepts