version 1.0

The Human-Computer Interaction Ontology (HCIO)

core ontology from HCI-ON

1. Ontology Description

The Human-Computer Interaction Ontology (HCIO), a reference ontology that aims to provide a common conceptualization of the Human-Computer Interaction (HCI) phenomenon and serves as a core ontology of HCI-ON. Given that a human-computer interaction involves communication between user and interactive computer system, HCIO is composed of sub-ontologies to deal with the interaction participants and with the interaction itself.

Given that a human-computer interaction involves communication between user and interactive computer system, HCIO is composed of sub-ontologies to deal with the interaction participants and with the interaction itself.

2. Related Ontologies

Networked ontologies used by HCIO:

Ontology	Relation	Integration Level
UFO - Unified Foundational Ontology	The HCIO concepts are derived from UFO (mostly social entities from UFO-C).	High
SysSwO - System and Software Ontology	Most HCIO concepts extend concepts from SysSwO.	High

Ontologies (re)used by HCIO.

3. Ontology Models

Figure 1 presents the packages of the HCIO Modularization.

Figure 1. HCIO Modularization.

3.1. Interactive Computer System

Figure 2 presents the conceptual model of the Interactive Computer System subontology.

Figure 2. Interactive Computer System conceptual model.

The Interactive Computer System sub-ontology addresses what an interactive computer system is and its elements, including the user interface. It is mainly an extension of SysSwO (SEON Core Layer).

Interactive Computer System is a subtype of Computer System, and like the latter, it combines hardware and software. Concerning hardware, an Interactive Computer System is (the Computer System) composed of a set of Computer Machines and peripheral devices (Hardware Equipment) connected to them. The striking feature of an Interactive Computer System is that it has a User Interface, a complex Object that is composed of Input Equipment and Output Equipment connected to the Computer Machine. If an Output Equipment is part of a User Interface of an Interactive Computer System, then this Interactive Computer System should be constituted of the Computer Machine that connects the Output Equipment. The same applies to Input Equipment. These constraints are addressed by the following axioms:

A1. ∀ oeq: Output Equipment, ui: User Interface, ics: Interactive Computer System partOf(oeq, ui) ∧ partOf(ui, ics) → (∃ cm: Computer Machine partOf(cm, ics) ∧ connectedTo(oeq, cm))

A2. ∀ ieq: Input Equipment, ui: User Interface, ics: Interactive Computer System partOf(ieq, ui) ∧ partOf(ui, ics) → (∃ cm: Computer Machine partOf(cm, ics) ∧ connectedTo(ieq, cm))

When a Hardware Equipment is both an Input Equipment and an Output Equipment it is said an IO Equipment. Thus:

A3. ∀ he: Hardware Equipment Input Equipment(he) ∧ OutputEquipment(he) → IOEquipment(he)

Regarding software, an Interactive Computer System has a set of Software Systems (Interactive Software System) loaded in its Computer Machines (Loaded Interactive Software System Copies). The programs that constitute these systems are instances of Program. Some of them deal with aspects related to the user interface and, thus, are instances of User Interface Program. Thus, an Interactive Computer System has a User Interface and the copies of the programs loaded in its computer (Loaded User Interface Programs Copy) that handle its User Interface. The following axiom applies:

A4. ∀ ui: User Interface, ics: Interactive Computer System, lissc: Loaded Interactive Software System Copy partOf(ui, ics) ∧ has(ics, lissc) → ∃ luipc: Loaded User Interface Program Copy includes(lissc, luipc) ∧ handles(luipc, ui)

Interactive Computer Systems can form another Interactive Computer System, which is said a Complex Interactive Computer System.

3.2. User

Figure 3 presents the conceptual model of the User subontology.

Figure 3. User conceptual model.

The User sub-ontology focuses on the user and its possible actions when interacting with an interactive computer system.

User is the role played by a Person that participates in a human-computer interaction. Such participation is said a User Participation, which can be either intentional (Intentional User Participation − or simply User Action) or unintentional (Unintentional User Participation).

In terms of UFO, User Participation is an Agent Participation. Intentional User Participation (User Action) is an Action Contribution, i.e., an intentional participation of a User. Intentional User Participations (User Actions) are caused by Intentions (User Intentions) that inhere in the User. As an Intention, User Intention has a Goal (more specifically a User Goal) as its propositional content. User Action, as an intentional participation, is performed by the User in order to achieve a User Goal. User Actions performed by a User can only be caused by User Intentions that inhere in that User. This constraint is addressed by the following axiom:

A5. ∀ user: User, ua: Intentional User Participation (User Action), uint: User Intention participationOf(ua, user) ∧ causedBy(ua, uint) → inheresIn(uint, user)

Considering its mereological structure, a User Participation can be an Atomic User Participation or a Complex User Participation, which is composed of others User Participations. In a Complex User Participation, all the User Participations are participations from the same User. Thus:

A6. ∀ cup: Complex User Participation, user: User, up: User Participation participationOf(cup, user) ∧ partOf(up, cup) → participationOf(up, user)

In another classification, which considers the nature of participations and is orthogonal to the ones discussed above, User Participations are classified into two disjoint types: User Initiated Participation and User Interpretation. User Initiated Participation refers to an act performed by the user making an input in the system. User Interpretation, in turn, regards interpreting a state of the system. When a User Initiated Participation is intentional, it is said a User Initiated Action.

3.3. HC Interaction

Figure 4 presents the conceptual model of the HC Interaction subontology.

Figure 4. HC Interaction conceptual model.

The HC Interaction sub-ontology links concepts from the other two sub-ontologies to define what a human-computer interaction is.

A Human-Computer Interaction is an interaction between a User and an Interactive Computer System. Thus, a Human-Computer Interaction is composed by User Participations and an Interactive Computer System Participation (which can be composed of several Program Copy Executions), indicating the events performed by both parties in a specific interaction. It is important to notice that the events that compose a Human-Computer Interaction (i.e., the User Participation and the Program Copy Executions that compose the Interactive Computer System Participation) can have different granularities, depending on the needs of a particular domain.

As said before (see User Sub-ontology), in terms of UFO, User Participation is an Agent Participation. Interactive Computer System Participation, in turn, is an Object Participation. It is a Complex Event, since it aggregates all events performed by the Interactive Computer System in the context of a single Human-Computer Interaction. It is an Object Participation because, being the Interactive Computer System an Object, its participation is always unintentional.

A User Initiated Participation is performed using one or more Input Equipment. As a result of a User Initiated Participation, a User Input Resulting State is achieved. User Input Resulting State is a Situation representing the data entered by the user before any program execution. This situation triggers Program Copy Executions. Moreover, we can say that User Initiated Participation directly causes Program Copy Execution. Program Copy Execution brings about a Computing Resulting State (internal computer state), which, in turn, can trigger other Program Copy Executions. Thus, a Program Copy Execution can directly cause new Program Copy Executions.

Some Program Copy Executions can bring about a special type of Computing Resulting State, the one that is perceivable by user, said User Observable State. A User Observable State, thus, triggers User Interpretation. Therefore, we say that Program Copy Execution directly causes User Interpretation. In this context, the following axiom applies:

A7. ∀ ui: User Interpretation, uos: User Observable State, pce: Program Copy Execution, uirs: User Input Resulting State, uip: User Initiated Participation triggers(uos, ui) ∧ bringsAbout(pce, uos) ∧ triggers(uirs, pce) ∧ bringsAbout(uip, uirs) → causes(uip, ui)

It is important to say that (A7) does not constrain who are the users involved in the User Initiated Participation and in the User Interpretation, i.e., the User in the User Initiated Participation may be different from the one involved in the User Interpretation.

A User Observable State triggers User Interpretation and is presented by one or more Output Equipment. Thus, Output Equipment supports User Interpretation, i.e.:

A8. ∀ oe: Output Equipment, uos: User Observable State, ui: User Interpretation presents(oe, uos) ∧ triggers(uos, ui) → supports(oe, ui)

In a Human-Computer Interaction, both the Input and Output Equipment involved in the interaction should be part of the User Interface of the Interactive Computer System that participates in the interaction.

Since User Initiated Participation typically leads to some processing inside the system that results in an output to the user who can interpret it, we say that User Initiated Participation (indirectly) causes User Interpretation. User Interpretation may cause the User to act again. Thus, User Interpretation may directly cause (in terms of UFO) User Initiated Participations.

When a User Participation is intentional, it means that the User performed it considering some goal (User Goal). Interpretations performed by the user (User Interpretations) may evaluate User Goal achievement. This is a case of goal-driven behavior, in which a cycle of actions (User Participation) are repeated until the user achieves his/her goal (User Initiated Participation (indirectly) causes User Interpretation and User Interpretations (directly) causes User Initiated Participations).

The HC Interaction sub-ontology main purpose is to define what human-computer interaction is. In view of what was discussed, in summary, a Human-Computer Interaction is a Complex Event composed of User Participation and Interactive Computer System Participation. User Participation can involve both User Initiated Participations and User Interpretations. Interactive Computer System Participation regards the set of Program Executions performed by the Interactive Computer System in the interaction.

4. Concepts Definition

The following table shows the definitions for HCIO concepts.

HCIO concepts definition.
Concept	Definition
Atomic User Participation	Single User Participation. E.g.: user touching her finger to unlock her mobile phone by using her fingerprint; user clicking a button.
Complex Interactive Computer System	Interactive Computer System composed of others Interactive Computer System E.g.: IoT in a smart home. src.: adapted from (ISO 9241-210, 2019) (ISO 9241-11, 2018).
Complex User Participation	Participation composed of two or more User Participations. E.g.: user typing her password and then answering security questions to log in the e-mail system.
Human-Computer Interaction	An exchange of information (i.e., communication) between a User and an Interactive Computer System via the User Interface. E.g.: user touching her finger to unlock her mobile phone by using her fingerprint and the mobile phone system showing a message informing that the fingerprint was not recognized; user entering a room, in a smart home, which automatically turns on the light. src.: adapted from (ISO 9241-210, 2019).
Input Equipment	Hardware Equipment used to receive and transmit information or data from a User (input). E.g.: keyboard, mouse, sensors. src.: adapted from (Dix et al., 2004).
Intentional User Participation (User Action)	User participation intentionally performed. E.g.: user touching her finger to unlock her mobile phone by using her fingerprint; user clicking the send button to send an e-mail.
Interactive Computer System	Computer System that combines User Interface, Computer Machine and Loaded Software System Copies, used to process, transform, store, display or transmit information or data by receiving input, and communicating output to Users. E.g.: a desktop computer, its monitor, keyboard, mouse and software systems loaded in the computer (operational system, browser, text editor, etc.); smart watch; smart mobile phone. src.: adapted from (ISO 9241-210, 2019).
Interactive Computer System Participation	The participation of an Interactive Computer System in a Human-Computer Interaction. E.g.: the participation of the mobile phone system in the human-computer interaction in which the user touches her finger to unlock her mobile phone by using her fingerprint and the mobile phone system shows a message informing that the fingerprint was not recognized; the participation of the smart watch in the human-computer interaction in which the user uses the smart watch to monitor her heart pulse.
Interactive Software System	Software system that includes, among other programs, User Interface Programs. E.g.: a home banking system and its graphical programs that deals with user interface aspects.
Loaded Interactive Software System Copy	Loaded Software System Copy that is the materialization of an Interactive Software System. E.g.: the copy of the home banking system loaded in the computer of a certain user.
Loaded User Interface Program Copy	Loaded Program Copy that is the materialization of a User Interface Program. E.g.: the copy of the programs that deal with interface aspects of the home banking system loaded in the computer of a certain user.
Output Equipment	Hardware Equipment used to transmit or display information or data, communicating output to Users. E.g.: Monitor, speaker, etc. src.: adapted from (Dix et al., 2004).
Unintentional User Participation	User participation unintentionally performed. E.g.: a user walking and unintentionally giving information about number of steps to a monitoring system loaded in her smart watch; user entering a room, in a smart home, which automatically turns on the light.
User	A Person who interacts with (or is expected to interact with) an Interactive Computer System. E.g.: a person who uses a home banking system, smart watch or smart mobile phone. src.: adapted from (ISO 9241-210, 2019).
User Goal	The propositional content of something the User wants to achieve (User Intention). E.g.: the goal of buying fly tickets to go on vacation.
User Initiated Action	Intentional User Action of inputting data or information using an Input Equipment. E.g.: user touching her finger to unlock her mobile phone by using her fingerprint; user clicking the send button to send an email.
User Initiated Participation	Event in which a User inputs data or information using an Input Equipment. E.g.: user touching her finger to unlock her mobile phone by using her fingerprint.
User Input Resulting State	A situation achieved as the result of User’s input (data or information) received by means of an Input Equipment. E.g.: the user fingerprint captured by the mobile phone system; the button clicked.
User Intention	An Intention that inheres in a User, causing him to perform intentional user actions. E.g.: the intention to travel on vacation.
User Interface	All components of an interactive system (software or hardware) that provide information and controls for the user to accomplish specific tasks with the interactive system. E.g.: mouse, keyboard, monitor, sensors and the software that handle them. src.: adapted from (ISO 9241-210, 2019; ISO 9241-112, 2017).
User Interface Program	Program that deals with aspects related to the User Interface. E.g.: the program responsible for the graphical user interface of the component that captures the user fingerprint to unlock a mobile phone.
User Interpretation	Event in which a User interprets the system output. E.g.: user interpreting a message from the mobile phone system informing that the captured fingerprint was not recognized; user sensing her smart watch vibrate; user hearing the notification sound of new message.
User Observable State	A situation (an Observable State) perceivable by Users, typically achieved as the result (output) of a Program Copy Execution, communicated or presented by an Output Equipment. E.g.: a message shown in the mobile phone screen informing the user that the captured fingerprint was not recognized, smart watch vibrating; sounds emitted by the mobile phone.
User Participation	Event in which the user participates in a Human-Computer Interaction. E.g.: user walking and unintentionally giving information about number of steps to a monitoring system loaded in her smart watch; user touching her finger to unlock her mobile phone by using her fingerprint; user interpreting a message from the mobile phone system informing that the captured fingerprint was not recognized.

5. Detailed Concepts

Human-Computer Interaction Ontology (HCIO) detailed concepts.

HCIO::Atomic User Participation

Atomic User Participation

Specializes:

HCIO::User Participation

Definition:

Single User Participation.
Example: user touching her finger to unlock her mobile phone by using her fingerprint; user clicking a button.

Human-Computer Interaction

Specializes:

UFO::Complex Action

COM::Measurable Entity

Definition:

An exchange of information (i.e., communication) between a User and an Interactive Computer System via the User Interface.
Example: user touching her finger to unlock her mobile phone by using her fingerprint and the mobile phone system showing a message informing that the fingerprint was not recognized; user entering a room, in a smart home, which automatically turns on the light.
Source: adapted from (ISO 9241-210, 2019).

Relations:

Human-Computer Interaction comunicates user input User Interface Human-Computer Interaction (1..1) <>-- (1..1) Interactive Computer System Participation Human-Computer Interaction <>-- (1..1) User Participation HCI Quality Characteristic (1..*) inheres in (1..*) Human-Computer Interaction HCI Evaluation (0..*) evaluates (1..*) Human-Computer Interaction User Interface comunicates system output Human-Computer Interaction

HCIO::Input Equipment

Input Equipment

Specializes:

SysSwO::Hardware Equipment

Definition:

Hardware Equipment used to receive and transmit information or data from a User (input).
Example: keyboard, mouse, sensors.
Source: adapted from (Dix et al., 2004).

Relations:

Input Equipment (1..*) is done using (0..*) User Initiated Participation Interactive Computer System (0..*) <>-- (1..*) Input Equipment User Interface (0..*) <>-- (1..*) Input Equipment

HCIO::Intentional User Participation (User Action)

Intentional User Participation (User Action)

Specializes:

HCIO::User Participation

UFO::Action Contribution

Definition:

User participation intentionally performed.
Example: user touching her finger to unlock her mobile phone by using her fingerprint; user clicking the send button to send an e-mail.

Relations:

Intentional User Participation (User Action) (0..*) caused by (1..*) User Intention

HCIO::Interactive Computer System

Interactive Computer System

Specializes:

SysSwO::Hardware Equipment

SysSwO::Computer System

COM::Measurable Entity

Definition:

Computer System that combines User Interface, Computer Machine and Loaded Software System Copies, used to process, transform, store, display or transmit information or data by receiving input, and communicating output to Users.
Example: a desktop computer, its monitor, keyboard, mouse and software systems loaded in the computer (operational system, browser, text editor, etc.); smart watch; smart mobile phone.
Source: adapted from (ISO 9241-210, 2019).

Relations:

Interactive Computer System <>-- User Interface Interactive Computer System <>-- Hardware Equipment Interactive Computer System (0..*) <>-- (1..*) Input Equipment Interactive Computer System <>-- User Input Resulting State Interactive Computer System (1..1) has (1..*) Loaded Interactive Software System Copy Interactive Computer System (1..*) has (0..*) HCI Quality Characteristic Complex Interactive Computer System (0..*) <>-- (2..*) Interactive Computer System HCI Evaluation (0..*) evaluates (1..1) Interactive Computer System Interactive Computer System Participation (0..*) participation of (1..1) Interactive Computer System

HCIO::Interactive Computer System Participation

Interactive Computer System Participation

Specializes:

Specializes:

HCIO::User Participation

Definition:

User participation unintentionally performed.
Example: a user walking and unintentionally giving information about number of steps to a monitoring system loaded in her smart watch; user entering a room, in a smart home, which automatically turns on the light.

User Initiated Action

Specializes:

HCIO::User Initiated Participation

HCIO::Intentional User Participation (User Action)

Definition:

Intentional User Action of inputting data or information using an Input Equipment.
Example: user touching her finger to unlock her mobile phone by using her fingerprint; user clicking the send button to send an email.

User Intention (0..*) inheres in (1..1) User User Goal (1..1) propositional content of (1..*) User Intention Intentional User Participation (User Action) (0..*) caused by (1..*) User Intention

HCIO::User Interface

User Interface

Specializes:

HCIO::User Participation

User Participation

Specializes:

UFO::Agent Participation

COM::Measurable Entity

Definition:

Event in which the user participates in a Human-Computer Interaction.
Example: user walking and unintentionally giving information about number of steps to a monitoring system loaded in her smart watch; user touching her finger to unlock her mobile phone by using her fingerprint; user interpreting a message from the mobile phone system informing that the captured fingerprint was not recognized.

Relations:

User Participation (1..*) participation of (1..1) User Human-Computer Interaction <>-- (1..1) User Participation Complex User Participation (0..*) <>-- (2..*) User Participation

References

Dix, A., Finlay, J. E., Abowd, G. D., & Beale, R. (2004). Human-Computer Interaction (3rd ed., p. 834). Pearson Prentice-Hall, Inc. [bibtex]

ISO. (2017). Ergonomics of human-system interaction — Part 112: Principles for the presentation of information (Standard No. 9241-112:2017; p. 20). International Organization for Standardization. https://www.iso.org/standard/64840.html [bibtex] [url]

ISO. (2018). Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts (Standard No. 9241-11:2018; p. 36). International Organization for Standardization. https://www.iso.org/standard/63500.html [bibtex] [url]

ISO. (2019). Ergonomics of human-system interaction — Part 210: Human-centred design for interactive systems (Standard No. 9241-210:2019; p. 33). International Organization for Standardization. https://www.iso.org/standard/77520.html [bibtex] [url]