User Evaluation of the OpenAIS Pilot Installation

The Internet of Things is rapidly entering the lighting domain. An Internet of Lighting is expected to have significant benefits for its users, such as advanced automated lighting and personal lighting control.

The OpenAIS project developed an IP-based lighting architecture and realized a large-scale pilot implementation of an IoT lighting system with advanced sensor-driven controls and user control in a real-life office. Two user control apps were developed and deployed in the open office, individual offices, and meeting rooms: a smartphone application for personal lighting control and an app for dedicated room control tablets. This article reports on an extensive study that evaluated peoples' use and experiences with the lighting system.

The results show that people appreciate personal lighting control and that they adjusted the lighting regularly using both the phone application and the control tablets. Furthermore, lighting control was experienced differently in different workplaces. In general, the level of lighting appraisal increased when more control was available. We argue that the flexibility will be a key successfactor for improving the lighting experience, that human-in-theloop control strategies need balanced automated system behavior and user control, and interfaces should support shared lighting control by including the social context.

Introduction

Following the trends of rapid penetration of Solid-State Lighting (SSL) and the Internet of Things (IoT), lighting in buildings is becoming connected, creating an Internet of Lighting (IoL). This transformation opens up new opportunities and value propositions for the broad group of stakeholders in the lighting value chain. Moreover, office buildings are transforming into open-plan office floors with flex-working policy, increasing office workers' need for personalization of the workplace. With the office lighting domain transitioning from an industrial to a knowledge paradigm, it becomes essential to take the human experience as a focal point: value for users, customers, and all stakeholders involved. A major opportunity of IoL is the data it can generate. Performance data can inform manufacturers and facility managers, and energy data can inform building owners of their carbon footprint, to name a few. Moreover, the flexibility of IoL allows building owners to reduce their CapEx by installing basic lighting systems and update them based on the building users' needs throughout the lifecycle of the system. For office workers, IoL can allow for personal lighting control at their workplace. Personal lighting control is known to decrease energy consumption, increase the level of work comfort, appraisal of the work environment, and job satisfaction.

This article presents the results of an extensive user evaluation of personal lighting control of a state-of-the-art IoT lighting system, as part of the final evaluation of the project. A large-scale pilot was realized in a real-life office in Eindhoven, the Netherlands. Two user applications were developed and deployed: A lighting control phone app allowed people to scan a QR code and control the lighting at their workplace, and tablets dedicated to room control presented people with lighting scenes and dimming controls for the meeting rooms.

Pilot Installation

The solution was validated with a full-scale office lighting system in a real office in Eindhoven, the Netherlands. Figure 1 shows a map of the office floor. The modern office is primarily open-plan with flexible workplaces. Next to 120 desks in the open space, the workplace offers a variety of workplaces: several open meeting cubicles in the open office, 24 individual offices, and ten meeting rooms. Initially, before the pilot, the office was equipped with a tube-lighting system providing 350 lx on desk level with 2700 K. People used regular lighting switches to turn on and off groups of around 20 luminaires. In total, 302 people work on the office floor (average age of 47 years, 10% male), but occupancy varies per day.

The implementation

The installation comprised 400+ luminaires with presence sensing, daylight regulation, and user control. The suspended tube-lighting was replaced using the original lighting grid. The open office area and meeting rooms were equipped with Zumtobel and Signify suspended luminaires (Figure 1B & D), and the individual offices were equipped with recessed luminaires by Signify (Figure 1A). The corridors were equipped with Zumtobel pendant luminaires (Figure 1C).

The new lighting system provided 500 lx on desk level with 4000 K, and was commissioned in two phases. First, after basic commissioning, luminaire groups of 2 to 4 luminaires were controlled through presence-sensing with an on-state (500 lx), backgroundstate (350 lx), and an off-state with delay times of 15 minutes. Corridor areas used presencebased controls with "corridor holding" logic. Luminaire groups were configured according to the light plan during the commissioning phase of the installation. Second, after advanced commissioning, the apps made personal lighting control available in the open office, individual offices, and meeting rooms. Luminaires close to the windows applied daylight regulation. When someone made a lighting adjustment with a user app, the automatic control (presence & daylight regulation) was disabled for that area until the sensors in that area detected an absence. Only then, automatic control would take over using the defined delay times.

Figure 2 shows a schematic overview of the implementation at the pilot location. A separate IPv6 network was set-up for the lighting system, next to the regular IPv4 network that was already in place. To maximize the availability of user apps to the office workers, the OpenAIS Webserver was accessible from the regular network, allowing the user apps to connect with the OpenAIS system using the existing corporate and public Wi-Fi SSID.

Figure 2: System overview of the OpenAIS pilot installation components and connections

System:
•    Architecture - The architecture uses LWM2M as its basis and entirely relies on standard internet technology with IPv6-based communication, UDP as its transport layer, and the CoAP protocol. The Group Communication protocol (OGC) was developed to allow for high speed, low-latency, secure and reliable communication between large sets of devices. More details can be found in the reference architecture report [5]
•    LWM2M server - The LWM2M server receives RESTful requests from the webserver and translates these to CoAP messages that are directly usable by the control objects
•    Webserver - A webserver allowed third-party applications to interface with the OpenAIS pilot solution using a predefined user control API. Its primary function was to translate incoming lighting adjustment messages from user apps to RESTful requests for the LWM2M server. Next to this, the webserver managed user authentication. The webserver stored all messages from and to user apps in the message database for research purposes
•    Commissioning Tool - To allow users to control the lighting system, stickers with QR codes on all the desks in the open office and the individual offices (Figure 3) linked to the ID of the control objects and devices on that location in the QR database. The webserver relayed a QR-code message form a user app to the commissioning tool, which returned object ID in JSON format

User apps

The Eindhoven University of Technology developed two types of user apps to control the light in the open offices, individual offices and meeting rooms. The apps were compatible with any luminaire in the area, no matter the type or vendor.

The phone app was available in the Google Play store and the Apple Appstore for people to install it on their smartphone (Figure 3 left). After opening the app and scanning a QR-code on the desk, the app showed a control screen with sliders to adjust the intensity of the luminaire(s) on the location between 0% and 100% of the light intensity. The number of sliders shown in the control screen depended on the number of luminaires present at the user's location. In general, the first slider could be used to change the intensity of the complete luminaire group at once, while the second slider allowed for controlling the luminaire closest to the user's desk. In case there was only a single luminaire (in most individual offices) the app showed only one slider. The webserver kept all apps in sync, so after a lighting adjustment, all apps logged in to the same location received an update from the webserver and updated their control page to reflect the current lighting conditions.

The ten meeting rooms in the office were equipped with tablets for room control (Figure 3 right). The control screen shows several buttons that allow for applying predefined scenes, customized to the meeting room (e.g., on, discussion, presentation). A slider allows for dimming the scene. A QR-code on the bottom of the screen allows people to control the luminaires in the room individually using the phone app.

Figure 3: User apps: (left) the phone app, and (right) the room control tablet in a meeting room

User Evaluation

The goal of the study was to gain insights into people's experiences with the lighting systems and to gather rich scenarios of using the provided user apps. Therefore, we applied a qualitative approach to the data collection by conducting semi-structured interviews with open questions to encourage people to provide rich feedback in the form of empirical data. An accompanying questionnaire structured to the interview. This questionnaire allowed participants to rate and note their appraisal of the lighting system (between 1 and 10), and included statements like "I feel in control of the lighting at my workplace" and "When I adjust the lighting, I feel limited by the presence of others" that participants scored using Likert scales. The study method is described in more detail in a separate article [1].

Firstly, before the installation, surveys were distributed in the office, allowing people to score the original tube-lighting system with light switches on several aspects and to rate the system between 1 and 10. A total of 19 completed surveys were completed before the installation took place. Secondly, the office workers evaluated the basic commissioned solution through interviews and questionnaires eight weeks after the installation. Finally, they assessed the advanced commissioned system after five weeks of use through interviews and questionnaires. 26 participants voluntarily signed up for the study (average age of 47 years, 19 female & 7 male), resulting in 25 interviews about the basic system, and 23 interviews about the advanced system. The group included 4 iPhone users, 22 Android users, and 1 Windows phone user. Participants spent 1-5 days of the week in the office, had a variety of job types, e.g., doctor, secretary, manager, and project-assistant, and did not have a fixed workplace. During the study, participants worked on their day-to-day jobs. Meanwhile, the webserver logged all user interactions with the phone apps and the room controls. As people filled in the first surveys anonymously, we cannot say whether they are the same people as the participants of the further evaluation of the system.

Results

The results of the study are presented as follows: First, we discuss how people experienced the system in its basic commissioning state. Next, we present quantitative results that display the lighting appraisal for the three data inquiries, and the logged data from the phone apps and room control tablets. The section concludes with insights from the qualitative data about how people experienced the two different user apps in the different types of workplaces.

Basic configuration of the system

It took some time getting used to the change in lighting intensity and color temperature of the new luminaires during the installation of the system. Nevertheless, this was a temporary issue, as people were very positive about the lighting during the final interview as it felt more "professional", "productive", and "activating". Overall the automatic control with presence sensors was preferred to controlling the lighting with the switches. Almost every participant expressed a wish for controlling the light intensity on their desk, and many expressed a need for tunable color especially in less formal areas. Unlit areas in the offices were only noticed by those who first arrived early in the morning or left the office last. Participants were positive about the energy-saving approach. Moreover, the automatic offswitching even helped some people to estimate whether a meeting room was vacant.

Lighting appraisal

Figure 4 shows the lighting appraisal scores from the questionnaires for the original lighting system and the two states of the OpenAIS system. In general, the appraisal for the lighting system increased after the new lighting system was installed, and again after people interacted with the user apps. While only lighting appraisal showed statistically significant improvements in comfort, the qualitative analysis showed a high appreciation for personal lighting control.

Figure 4: Lighting appraisal scores for the old lighting system and the two OpenAIS solutions. The rating scale ranges from 1 to 10

User app usage

People made a total of 3937 lighting adjustments over the 3-month period: 93 people made 2712 lighting adjustments with the phone apps and 1225 lighting adjustments with the room control tablets (that is about 24 adjustments per day). Figure 5 shows a heat map of the lighting adjustments that people made throughout the office floor. Table 1 provides an overview of the number of lighting adjustments. In general, people like to have control over the lighting at their workplace, and the feedback on the user apps was very positive. Surprisingly, participants mentioned that they expected to be using the apps even more than they actually did, even though they used the phone app and room controls quite often. The phone apps allowed them to increase the light intensity at locations and times that there was less natural lighting, and to create spotlight-like settings when they wanted to concentrate on their work.

Table 1: Number of lighting adjustments by phone apps and room controls in the open office, meeting rooms and individual offices

Figure 5: Heat map of user interactions with the OpenAIS system with the phone apps (red) and the room controls (blue) over a 3-month period

User control in an individual office

Personal lighting control was valued in individual offices in particular, as people felt that this was their workplace and they would not disturb anyone by making lighting adjustments. In total 43 people made 826 lighting adjustments in the individual offices. Especially the individual offices with more than one luminaire were valued as it allowed people to adjust light distribution as well. Participants mentioned that, before lighting control, some individual offices felt dark and remote as there was little natural lighting. Interestingly, people were able to make those workplaces more pleasant by increasing the lighting intensity and were, therefore, less reluctant to work in those offices.

User apps on the phone

Although the phone app was easy to use, people indicated that controls ready at hand (e.g., tangible lighting controls on the desk, or the room control tablets) might be more appreciated. Benefits of dedicated lighting controls compared to a phone app include that they can be easier to access by everyone, with less effort. The interactions in the meeting rooms illustrate the effect: The room control tablets (that were accessible for anyone in the office and ready at hand) were used significantly more (1225 lighting adjustments), compared to 307 lighting adjustments made with the phone app by 16 people. The OpenAIS standard, which utilizes standard IoT principles, allows third-party developers to easily integrate any user app, whether it is on a phone or a dedicated tangible device, as it allows for the integration of a vast already existing collection of microcontrollers and protocols.

Autonomous system behavior

The combination of autonomic, sensor-driver controls and user control caused some issues every once in a while. The presencedriven controls sometimes overruled a user setting, and the luminaires would revert their initial on-state. These situations, although sporadic, were experienced negatively and sometimes, when it happened twice or more times to one person, people started to lose interest in personal lighting control. Some participants even mentioned feeling less in control with the OpenAIS system as the system can never be truly turned off anymore, and they felt like the system was able to ignore user control if it wanted to.

IoT lighting systems can offer a solution for this, as it allows for easy updating control software over the air throughout the system's lifecycle. The OpenAIS solution can continuously improve controls and fine-tune autonomous behavior to make it fit better to the occupants' wishes, and in this way can minimize the number of negative encounters.

Luminaire groups

Not every participant agreed with the pre-defined luminaire group, especially in the open office where the lighting grid did not always match the office layout. In particular, people generally used one of the corridor areas in the east wing for receptions and other informal occasions and wished to dim the lighting during those occasions. However, user control was not possible. As with most lighting installations, the luminaire groups were defined according to the lighting plan during the commissioning phase.

The flexibility of an IoT lighting solution allows for a relatively effortless recommissioning. Moreover, the OpenAIS standard allows flexible and layered grouping, which makes it easy to change luminaire groups on the fly according to the users' wishes.

User control in the open office

During the interviews, people were divided about personal lighting control in the open office, as generally, nobody wanted to disturb colleagues with their lighting adjustments. Despite their concerns, people used the phone app extensively in the open office (1579 lighting adjustments by 49 people). Occasionally people started a discussion with every colleague that might be affected by the lighting change, but they mentioned that this was often too much effort and decided not to interact at all. Nobody knew who made a lighting adjustment when it happened since the lighting control app was hidden in someone's phone, while in a meeting room, the control tablet and the person interacting were visible from anywhere, serving coordination of shared lighting control amongst everyone in the area.

Figure 6: Open office in the White Lady pilot site

Concluding Remarks

OpenAIS has successfully implemented a fully-functional multi-vendor IoT lighting system in a large-scale pilot in a real-life office. Two novel user apps were designed, fully integrated into the OpenAIS system, and deployed in the office for three months: the phone app allowed office workers to change lighting intensity at their flexible workplace anywhere in the office, and tablets with room controls allowed for adjusting the lighting in meeting rooms.

The solution was evaluated extensively with 26 office workers. The results from assessing the original tube-lighting system, a basic installation of the system (with presence controls), and an advanced installation of the system (with daylight regulation and user control), show a significant increase in lighting appraisal as the amount of user control increased and illustrated a growing need for personal lighting control in the office.

In general, people working in the office were very positive about having the possibility to adjust the lighting in their workplace, even though some had expected to use the apps more than they actually did. The room control tablets in the office were highly appreciated. The phone app was especially useful in individual offices. People expressed concerns about disturbing others with lighting adjustments in the open office. The study showed that the design of the user app and control tablets influenced the shared use of the lighting system, which is in line with previous studies [2,3]. With this in mind, we want to bring particular attention to the importance of the design of interfaces for lighting control, as it is up to the developers to consider the social context in which the interfaces are used, and to design them accordingly so that they fit the context and can allow for shared use of the lighting system.

Another main insight from this work is that, to unlock the benefits of personal lighting control, human-in-the-loop control strategies need to balance autonomous system behavior and user control. This also shows the importance for IoT lighting systems to allow for updating firmware and control software throughout a system's lifecycle, to adapt a system's behavior and control strategies to the people working in the office.

Moreover, we want to highlight that the integration of IoT to lighting in the office brings changes to the entire lighting value chain. Lighting systems no longer deliver value to the customer only but create value for the users (office workers, installers, and commissioning agents), the customers (building owners, tenants), and all stakeholders involved, including new stakeholders (e.g., third-party developers). This is especially apparent in the opportunities that the flexibility of IoT systems brings. A variety of stakeholders can stay involved with the use of the system throughout its lifecycle and optimally update control strategies, luminaire grouping, and user control to make it fit the users' needs. We want to urge the office lighting value chain to take a holistic perspective and to develop the lighting systems of the future with the human experience as the focal point.

Acknowledgments:
This article is the result of three and a half year close cooperation between the partners in the OpenAIS project. We want to thank all team members for their contribution and collaboration. We also want to thank the people of the office building for their cooperation, and the participants of the study for their openhearted accounts. The project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 644332.

References:
[1]  The impact of the Internet of Lighting on the Office Lighting Value
       Network, T.C.F. van de Werff, H.A. van Essen and J.H. Eggen, Journal
       of Industrial Information Integration, January 2018
[2]   OpenAIS Integrating Lighting in the Internet of Things, Frank van Tuijl
       and Ben Pronk, Led Professional Review #67, May 2018
[3]   OpenAIS Deliverable: 2.7 - Final reference architecture of OpenAIS
       system: http://www.openais.eu/user/file/openais_final_reference_architecture_(d2.7)_v2.0-pub.pdf
[4]   Selected Scenarios and Use Cases, John Sayer, LPS 2016:
       http://www.openais.eu/user/file/openais-lps2016-selected_scenarios_and_use_cases-sayer(jci).pdf
[5]   OpenAIS Deliverable 1.1, Selected Scenarios and use cases:
       http://www.openais.eu/user/file/openais-lps2016-selected_scenarios_and_use_cases-sayer(jci).pdf

The OpenAIS Consortium & the OpenAIS Project
The OpenAIS project started in 2015 as a response to the emergence of IoT as a leading force in the digitalization of buildings and homes. OpenAIS is a European Community supported project that is partially funded through the Horizon2020 program. The project team started from the assumption that further developments in IoT-infrastructure would drive a revolution in connected lighting solutions, moving these to open (IoT) standards and off the shelf Internet technology. Additionally, the project definition included the vision that the ubiquitous lighting infrastructure would be an ideal platform to integrate multiple IoT-devices and deliver additional functionality beyond lighting. Such a development would revolutionize the lighting business, moving it away from vertical silos and proprietary (and closed) solutions towards the use of (open) IoT- ecosystems and standards. As has happened in many "digitized" domains, this transition would, in the view of the project consortium members, also greatly shake up the entire value chain and stimulate demands for openness and interoperability by professional customers keen to avoid lock-in.
The OpenAIS project runs from 2015 to mid-2018 and is coordinated by Signify (formerly known as Philips Lighting). The OpenAIS consortium includes partners from all segments of the lighting industry and its adjacencies:
facility management, installation, lighting manufacturing, technology suppliers and two academic partners. As a carrier case for the project Professional Indoor Office lighting in Europe has been chosen.
The OpenAIS partners are Signify (formerly known as Philips Lighting B.V.), Zumtobel Lighting GmbH, Tridonic GmbH & Co KG, Johnson Controls Systems and Service Italy SRL, Dynniq Belgium N.V., NXP, ARM Ltd, Eindhoven University of Technology and TNO-ESI