ParmoSense: A Scenario-based Participatory Mobile Urban Sensing Platform with User Motivation Engine

Rapid proliferation of mobile devices with various sensors have enabled Participatory Mobile Sensing (PMS). Several PMS platforms provide multiple functions for various sensing purposes, but they are suffering from the open issues: limited use of their functions for a specific scenario/case and requiring technical knowledge for organizers. In this paper, we propose a novel PMS platform named ParmoSense for easily and flexibly collecting urban environmental information. To reduce the burden on both organizers and participants, in ParmoSense, we employ two novel features: modularization of functions and scenario-based PMS system description. For modularization, we provide the essential PMS functions as modules which can be easily chosen and combined for sensing in different scenarios. The scenario-based description feature allows organizers to easily and quickly set up a new participatory sensing instance and participants to easily install the corresponding scenario and participate in the sensing. Moreover, ParmoSense provides GUI tools as well for creating and distributing PMS system easily, editing and visualizing collected data quickly. It also provides multiple functions for encouraging participants' motivation for sustainable operation of the system. Through performance comparison with existing PMS platforms, we confirmed ParmoSense shows the best cost-performance in the perspective of the workload for preparing PMS system and varieties of functions. In addition, to evaluate the availability and usability of ParmoSense, we conducted 19 case studies, which have different locations, scales, and purposes, over 4 years with cooperation from ordinary citizens. Through the case studies and the questionnaire survey for participants and organizers, we confirmed that ParmoSense can be easily operated and participated by ordinary citizens including non-technical persons.


Introduction
Mobile devices come equipped with various sensors including GPS, inertial sensors, environment sensors, camera, microphone and so on. The rapid widespread of such mobile devices have enabled Participatory Mobile Sensing (PMS) [1,2,3]. PMS systems are based on crowdsourcing technology, where data in a wide geographical area can be collected efficiently and at low cost by leveraging sensors on mobile devices carried by ordinary citizens. Many applications can utilize collected urban data to bring various benefits to our daily lives. For instance, because PMS systems use common devices, they can be easily used to collect data for urban analysis [4], office management [5], healthcare [6], and education [7,8]. In addition, since PMS systems can be applied to any region in which people stay or pass, they are very effective for collecting geospatial data over a wide area. In urban environment for example, data such as illuminance of the road at night [9], noise levels in the city [10,11], and air pollution degrees [12,13] can be collected.
PMS is a sensing mechanism based on the voluntarism of general people. In other words, the sustainability of the system is a critical challenge in real-world operations. As an idea to enhance this sustainability, the mutual linkage with the local community where ecosystems are already formed can be considered. For example, the civic cooperation that people work with government, universities, companies, etc. to promote community development spreads globally. Especially in recent years, CivicTech which combines ICT and civic cooperation is gathering attention, e.g., FixMyStreet [14]. In our study, we focus on PMS systems which can be used in CivicTech communities.
To realize PMS systems in the real world for broad urban environment analysis, we believe that a platform that can be easily and quickly customized by organizers to perform a variety of sensing tasks, and that is easy to set-up and run on the participants' smartphones, is essential. However, when we investigated the functions implemented in existing PMS platforms [15,16,17,18,19,20,21,22,23], we found two main challenges in using these platforms for broad urban environment analysis: C1: Limited support of essential functions and C2: Difficulty of system construction and operation.
Regarding C1, existing platforms tend to focus on specific sensing purposes, e.g., urban transport data sensing, and therefore support limited functions. Because the purpose of sensing differs among organizers of urban sensing, the necessary functions, i.e., sensing function, incentive mechanism, task request control, and data processing method, will also differ depending on the purposes. Thus, in the ideal PMS platform, flexibility to adapt the platform to perform sensing for various purposes is mandatory. Moreover, motivating participants is an important aspect of PMS since participatory sensing relies on voluntary participation of ordinary citizens [24], but we found that these platforms do not implement it enough. And regarding C2, The platforms require a high level of technical skill for users. For example, some platforms require programming skills for organizers and data processing skills for participants. In order to open the door of participatory sensing to non-technical users, it is necessary to ensure that PMS systems can be easily constructed and operated by both organizers and participants.
In this study, we designed and built a novel PMS platform called ParmoSense, for easily and flexibly collecting urban environmental information for various purposes by overcoming the challenges mentioned above. To achieve this, we employ two features: modularization of functions and scenario-based PMS system description. We provide various functions essential for PMS systems such as sensing functions, motivating functions for participants, and processing functions for collected data, and allow organizers to combine these modularized functions freely through a GUI web application. We call a combination of these modularized functions a scenario. Once a scenario has been created, participants can download it onto the ParmoSense Client application and run it without doing any further setup or processing tasks. Thus, participants can contribute to many different sensing tasks without installing multipleapplications or performing complicated tasks which require technical skills.
To evaluate the superiority of ParmoSense, we compared a performance with existing PMS platforms. First, we confirmed that ParmoSense provides a higher variety of functions than the existing PMS platforms, which solves challenge C1. We also found that ParmoSense eases organizers to prepare PMS systems. It belongs to the lowest preparation workload group among those platforms, which solves challenge C2. From both perspectives, therefore, ParmoSense shows the best cost-performance. In addition, to evaluate the availability and usability of ParmoSense in the real-world, we conducted the 19 practical case studies with ordinary citizens including non-technical people. We confirmed that ParmoSense can deal with various sensing targets, organizers, and participants, in real environments.
Our contributions in this paper are as follows: i. We designed a PMS platform, named ParmoSense, which allows general people to easily operate PMS systems with scenario-based system construction regardless technical skills.
ii. We organized functionality requirements through the survey of existing PMS platforms, and implemented all functions as combinable modules.
iii. We confirmed that ParmoSense shows the best cost-performance in the perspective of varieties of functions and preparation workload through the evaluation on comparison with nine existing platforms.
iv. We confirmed the availability and usability of ParmoSense through the 19 practical case studies in the real world, and interviews with participants and organizers of sensing tasks. The rest of this paper is organized as follows: In Section 2, we survey the existing PMS platforms and systems, and organize required functions for PMS systems and skill requirement for participants and organizers. In Section 3, we describe the concept and the architecture of ParmoSense and provided functions on ParmoSense. To evaluate ParmoSense, we compare the functionality and performance with existing PMS platforms in Section 4, and conduct the 19 practical case studies with general people including non-technical persons in Section 5. Section 6 concludes this paper, and we will discuss limitation and future challenges of ParmoSense.

Related work and challenges
This section is devoted to clarifying what kind of functions are necessary for PMS systems and what kind of skills are required for users (organizers and participants) of PMS systems. We first organized the necessary functions into three categories: sensing functions, motivating functions and processing functions. Then, we investigated the functions implemented in existing PMS platforms [15,16,17,18,19,20,21,22]. The results are summarized in Table 1. The skills required by organizers and participants in existing PMS platforms are shown in Table 2.

Sensing functions
Sensing is an essential part of PMS systems. We define sensing functions as those that allow the organizer to specify what kind of sensors to use, and how to collect sensing data from the urban environment. There are two different ways of sensing: implicit (F1) and explicit (F2) sensing.
Implicit-sensing (F1): Uses sensors embedded in mobile devices. It is mainly used for collecting urban environmental data without actively involving the participant, i.e., implicit sensing.
Explicit-sensing (F2): Used for collecting data generated by human behavior, e.g., photos, voice, and questionnaires. It is used for collecting urban environmental data through directly involving participants, i.e., explicitly, and locally.
AWARE [15] provides a platform for both implicit and explicit sensing. For implicit-sensing, the organizer can choose which smartphone sensors to use from a web UI. They can also configure the detailed settings such as sensing interval. For explicit-sensing, AWARE allows the organizer to distribute questionnaires manually.
Ohmage [21] supports explicit-sensing, by allowing participants to post reports by themselves. Several report formats are accepted such as single/multiple selections, free text, multimedia (e.g., photo, sound) and so on. Ohmage also supplements collected data through implicit-sensing. Specifically, it can be used to record the transportation status (e.g., still, walk, run) of a participant.
The other conventional platforms however, tend to focus more on either implicit-or explicit-sensing (as shown in Table 1), and provide limited functionality for the other form of sensing. As mentioned before, the two sensing methods have many differences such as data type that can be collected, and the data's features. Thus, with these platforms, it is difficult to supplement the collected data due to severe restrictions in sensing functions.
In this paper, in order to realize a flexible sensing platform, we aim to provide functions for both sensing methods comprehensively and support the organizers' ability to easily choose and combine functions.

Motivating functions
Since PMS relies on voluntary participation of ordinary citizens, it is essential to not only focus on acquiring users, but also on motivating them to continue participating in the sensing tasks, i.e., to support user retention and activation [25].
Motivating functions allow the organizer to define the methods for motivating participants. Some of the conventional platforms use outside stimuli to support participant motivation and engagement. We classify these outside stimuli as follows: Request (F3): This is urging behavior by explicitly requesting participation. There are many methods of sending requests. The most common method is the static/dynamic request, which includes providing a task list, issuing notifications and so on. Other methods such as Audition [17], and Reverse-auction [26,27] which purposely restrict the rights of contribution and make participants scramble to contribute, have also been used.
In addition, the willingness-based participant selection [23], which selects the participant for requesting based on an estimation of the participant's willingness for sensing task, has been used.

Reward (F4):
In this method, participants are compensated for their contribution through monetary and non-monetary incentives [28,29]. The monetary incentive encourages participants to contribute to the system by giving money-related value such as in-app currency/points, discount coupons, and gifts. Participants can directly get explicit value as a consideration for their contribution. It often marks high effectiveness, however, there are problems related to sustainability of system management [24]. To reduce costs for rewards, the Auction mechanism [30], and the following non-monetary incentive mechanism can be used. The non-monetary incentive gives experience (a kind of fun) as compensation for participant's contribution [31,32,33]. There are several kinds of experiences such as interaction with other participants, and gamification. The interaction with other participants induces social facilitation effects that stimulate the participant to contribute more actively [34] for getting praise from others, e.g., getting more like, more comments. Gamification is the mechanism which introduces game elements into a conventional system, and it has been shown to contribute to motivation of participants, and reduction of monetary rewards [35].
Feedback (F5): This method urges behavior by providing feedback such as a visualization of participants' contribution on a map, graph or timeline. Sometimes the contributions of other participants are also included in the visualizations, and sometimes they are excluded.
MinaQn [19] uses a recruitment mechanism to grant participants the right to participate in urban planning (contribution to society) as non-monetary incentive. In addition, the platform also visualizes a summary of the participant's contribution to increase their willingness to continue contributing.
Medusa [17] is a platform which utilizes monetary incentives effectively. Medusa can acquire participants by using recruitment, which provides money as compensation, and through audition. Furthermore, to retain participants, Medusa adopts the concept of reverse incentive (obligation/responsibility of executing tasks), where workers pay to organizers for the privilege of performing the task. This can help prevent participants from quitting the system in the middle of sensing tasks.
GP-Selector [23] is a platform that employs a participant selection algorithm by various conditions suitable for sensing tasks. It includes location-based constraints (e.g., geofences), capability-based constraints (e.g., available sensor), and willingness prediction.
In these conventional platforms, functions to motivate participants have a number of shortcomings. For example, with Request functions (F3), in order to increase the number of successful requests, it is necessary to consider the notification timing and the target participants, but this is not supported. Similarly, with Reward functions (F4), we need to consider not only monetary incentives, but also Gamification. In this paper, we consider how to design motivating functions which incorporate the concepts of interruption through notification and gamification.  [15] As needed *a -Required -Sensus [16] As needed *b ---Medusa [17] Required Required --Funf [18] --Required Required MinaQn *c [19] ----KOKOPIN app *c [20] ----Ohmage [21] --Required -OpenDataKit [22] As needed *a Required Required Required GP-Selector [23] As needed *d ---ParmoSense ----*a Development by programming is needed for extending functions. *b Database server is required for data collection. *c These platforms assume the use of ordinary citizens or administrative officers, hence, technical skills are not required. *d XML-based script coding is needed for making complex participant selection constraints.

Processing functions
In general, organizers intend to analyze or visualize urban environmental data collected with PMS. Hence, PMS platforms must support easy and quick access to this data. In the processing functions, the organizer defines methods of data processing to be used. The following functions are implemented in conventional platforms: Data editing (F6): This involves data cleansing and labeling, as pre-processing for detail data analysis. Funf [18,36] and OpenDataKit [22] are designed as platforms that mainly focus on data cleansing and visualizing, as well as exporting. Additionally, these platforms support data processing in a variety of environments such as in the cloud and on the smartphones used for data collection (endpoint devices). Thanks to the various processing functions in the platforms, many research projects in the world utilize them. Although most of them also support basic functions, there are differences in functions supported by other conventional platforms [15,19,21].
In this paper, we implement all functions (F6-F8) as in Funf [18,36] and OpenDataKit [22]. During implementation, we considered how to make the required technical skills for organizers and participants lower. Specific skills needed to operate existing platforms are described in the section below.

Required skills for operation and use
In PMS, it is assumed that the organizer may be from a non-technical profession/background, e.g., they may be an administrative officer or urban planner, and ordinary citizens participate in the data collection. Thus, it is necessary to re-consider the skills required by the PMS platforms for the organizers and participants. AWARE [15], Medusa [17], and OpenDataKit [22] have high extendability like a framework, but a high-level of programming skill is required for system construction (R1). In addition, most of the systems that require system development also require organizers to have the skills necessary to release applications on official stores such as Google Play and AppStore (R2). With web-based platforms such as MinaQn [19] on the other hand, deploying the applications is quite easy. However, continuously attracting users to the web application is required (R2).
App/Function management skills (R3) and Data processing skills (R4) are other skills that are required from participants, where: App/Func. management skills (R3): Managing applications and functions such as installation of applications and setting of functions to accomplish sensing tasks. Data processing skills (R4): Processing data collected using the participant's device before uploading.
Since AWARE [15] and Ohmage [21], OpenDataKit [22] have many functions as a platform, the functions are divided into multiple applications and provided to participants. Also, Funf [18] requires participants themselves to set up the sensors (e.g., sensing interval) for each smartphone. Therefore, to construct the sensing environment that the organizer intended, knowledge and skills related to applications and functions are required for participants (R3).
Funf [18] and OpenDataKit [22] adopt a mechanism where they ask participants to perform data processing and cleansing. Such processing requires knowledge and skill to judge whether data is good or bad, so the burden on participants is substantial (R4).
Overall, conventional platforms require various skills for both organizers and participants to construct and operate the system. To realize PMS systems that can be used by non-technical people, these problems must be addressed. In this study, we propose a new platform to resolve these problems.

Scenario-based participatory mobile sensing platform
We design and implement a novel participatory urban sensing platform, ParmoSense, to solve the problems mentioned in Related work section. ParmoSense aims to solve the following key challenges: C1: Limited support of essential functions needed for PMS systems (listed in Table 1) C2: Difficulty of system construction and operation ParmoSense allows organizers and participants to operate or contribute to PMS systems without complex procedures or technical skills.

ParmoSense Basic Principles
ParmoSense is based on the following three basic principles.

Principle #1 Modularized functions
ParmoSense must achieve two contradictory requirements, easiness of system construction and diversity of available functions. Therefore, we employ the idea of modularizing functions inspired by existing research [15,21]. ParmoSense provides the sensing, motivating and processing functions in Table 1), and these can be combined to form a PMS system. Principle #2 Standardized PMS system Conventional platforms require a high level of technical skills, e.g., programming skills for organizers to customize the platform for a specific sensing task. In order to deal with multiple purposes flexibly, ParmoSense is composed as a combination of modularized functions as well as the detailed settings of each function. We unify the combination and settings as a scenario. A Scenario contains such information as the scenario name, description, sensing targets, sensing area, period, motivation method, etc. Through GUI-tool in ParmoSense, the organizer can generate the scenario easily. Based on the created scenario, ParmoSense automatically configures both a server system and a client application by distributing necessary information. Since scenarios can be created using any combination of functions and settings, logically, any kinds of PMS system can be built with ParmoSense. Another important advantage of the scenario-based system is that users can participate in various PMS projects through one client application, whereas in the past, each PMS project required a dedicated application.

Principle #3 Customizable motivation engine
The most effective way of motivating participants depends on the purpose of sensing. To realize sustainable urban environmental sensing, ParmoSense has a Motivation Engine with a variety of motivation algorithms.
The Motivation Engine provides the following functions for motivating participants: • Motivation based on the behavior of an individual: Granting incentives according to contribution, Visualization of contribution • Motivation for all participants, regardless of contribution: Providing competition mechanisms such as rankings, Sharing of experiences among participants Additionally, by considering temporal/spatial information, e.g., the current time and the current position of the participant, it is possible to control the actuation timings of these functions. The optimal motivating algorithm according to the purpose of the organizer can be incorporated into the PMS system by combining and customizing these functions.

ParmoSense system overview
The architectural design of ParmoSense is shown in Fig 1. ParmoSense consists of three parts that have the following roles: ParmoSense Dashboard: A web application for organizers that can be used to create and distribute scenarios of PMS systems, and to process collected data.

ParmoSense Client:
A client application for participants that can run various scenarios. By downloading and installing a scenario, it behaves as the corresponding sensing application.
ParmoSense Server: A central system for integrated management of the scenarios created by organizers and automatically constructing a virtual server system for each scenario. It can collect sensing data from ParmoSense Client and generate feedback based on the analysis of the collected data. i. Distributing Phase: The phase for distributing the scenario created by the organizer to the participants via ParmoSense Server.
ii. Sensing Phase: The phase for giving feedback (e.g., reward) to the contribution of the participant such as uploading sensing data by participants.
iii. Processing Phase: The phase for editing, cleansing and visualizing the collected data.
Since ParmoSense is based on Principles #1, #2 mentioned above,organizers can distribute sensing applications by simply exchanging scenarios with participants in Distributing Phase. It is therefore not necessary for each participant to manage the applications by himself. Furthermore, thanks to Principle #3, in Sensing Phase, the feedback for motivating participants is created by the Motivation Engine using the collected data, and automatically provided to participants.

ParmoSense system architecture
The concrete system configuration of ParmoSense is shown in Fig 2. In the following subsections, we will describe the ParmoSense Dashboard used by organizer, the ParmoSense Client used by participants, and the ParmoSense Server in more detail.

ParmoSense Dashboard
An organizer carries out every operation, e.g., management of a PMS scenario, processing of collected data on the ParmoSense Dashboard, a web application. It consists of Scenario tools and Data tools shown in Fig 2 (a) 1 and 2 respectively.
Scenario tools (Fig 2 (a) 1 ) provide many operations such as creating, editing and deleting PMS scenarios, and browsing, activating and stopping scenarios. Fig 3 (a) shows the user interface for editing scenarios. The organizer can describe a scenario using the three kinds of functions, sensing, motivating and processing functions, mentioned in Related work section, without programming, through the GUI (Principles #1, #3 ). The scenario defined in Scenario tools is automatically converted to JSON format, and transferred between each part of ParmoSense.
When the scenario editing is completed, the virtual server system is automatically built depending on the scenario, and deployed by Scenario manager (Fig 2 (a) 3 ). At the same time, the QR code for downloading the scenario to ParmoSense Client is automatically generated . Fig 3 (b) shows the user interface for browsing/managing the scenario created by organizers. Scenarios currently in progress/stopped are indicated by blue/gray respectively, and these statuses and the scenario settings can be changed by using the GUI.
Data tools (Fig 2 (a) 2 ) provide the functions for processing and visualizing data aggregated into Data manager (Fig 2  (a) 4 ). The user interface for editing the collected data is shown in Fig 3 (c). The organizer can improve the quality of the data by editing/excluding inappropriate data from the collected data. The organizer can also add labels to the data. The processed data can be exported in the form of JSON, CSV, RDB etc. The user interface for visualizing the collected data is shown in Fig 3 (d). The organizer can check the data in two ways: overlaying them on a geographical map, or sorting them in a time-series order.

ParmoSense Client
The participant performs all sensing tasks of ParmoSense through the ParmoSense Client smartphone application. ParmoSense Client runs on smartphones with Android OS or iOS, and it can be installed from general application stores (Google Play, AppStore). Since the behavior of PMS system on ParmoSense is defined by a scenario (Principle #2 ), it can behave as various sensing applications by installing scenarios to ParmoSense Client.  i. Log in on ParmoSense Client via Google Authentication. ii. Scan the scenario QR code by using their smartphone camera (Fig 4 (a)). An organizer can get QR codes of scenarios from ParmoSense Dashboard, and print it out for participants. iii. The participant confirms participation in sensing.
This procedure makes it easy to install scenarios. All these are done via the Web API shown in Fig 2 (a) A . Participants can participate in multiple scenarios. The scenarios that have been installed and performed in the past are listed, as shown in Fig 4 (b). This makes it easy for a participant to join the same scenario again. Scenario manager plays the role of storing the scenario created on ParmoSense Dashboard, and constructing and managing Scenario instance in accordance with the scenario. The Scenario instance is an instance executed on a server which communicates with a ParmoSense Client. By automatically constructing this Scenario instance for each scenario and forming Scenario instances, various scenarios in ParmoSense can be created without programming. Scenario manager monitors the operation status of the Scenario instance, and the organizer can start or stop the operation. It also detects unexpected troubles in the Scenario instance, and it stops or restarts them. The data collected by participants is aggregated in Data manager, and this data is used for calculating the participants' score, visualizing on the map and so on.  based on the scenario that an organizer created. Scenario instance communicates with ParmoSense Client via MQTT as described above. If a participant sends (publish) the sensing data, the corresponding Scenario instance receives (subscribe) the data collected by participant's smartphone sensors, and processes using modularized functions (e.g., analysis of data, calculation of ranking) which are described in the scenario. According to these results, response data is generated and published to all participants who should be informed.

Functions support
ParmoSense comprehensively supports functions investigated in Table 1. In this section, we outline the support status of each function.
Implicit-sensing (F1): ParmoSense supports data collection from sensors embedded in smartphones. The types of supported sensors are shown below: • Position sensor (e.g., GPS) • Environmental sensors (e.g., light sensor, barometer) • Inertial sensor (e.g., accelerometer, gyroscope) • External sensor device (heart-rate sensor) It also supports data collection of BLE scan logs of peripheral devices (e.g., iBeacon, other smartphones). For all these sensors, detailed configuration such as measurement interval and enabling/disabling of background measurement can be set on Scenario editor of ParmoSense Dashboard by the organizer.
Explicit-sensing (F2): ParmoSense supports various kinds of data collection methods that cannot be collected by sensors embedded in a smartphone. One of them is photo uploading. When taking photos and uploading them, participants can provide additional data such as explanatory texts of the photo taken, GPS position, and other data obtained by Implicit-sensing. Questionnaires are also provided. Different question types such as binary questions (YES/NO

Contents of work
Preparation Workload Done by w 1 development (programming) 0, 8 *a organizer w 2 development (GUI editing) 0, 4 *e w 3 publishing to app store 0, 8 *b w 4 installing application 0, 1 *c , 2 *d participant w 5 configuring functions 0, 2 *e *a calculated with LOC of source code. *b calculated based on time for becoming available in general stores. *c installation of single application (reference time). *d installation of multiple application. *e calculated by comparing with w1, w3 and w4 relatively. question), multiple-choice questions (up to 4 options), and questions that require photo uploads and explanatory text, are supported. These questions can be chained together to support a step-by-step questionnaire.
Static/Dynamic request (F3): ParmoSense supports both static and dynamic requests for soliciting contributions on sensing tasks. In PMS for urban environments, there are many requests based on geographical information. Static requests place each task as a checkpoint on a map as shown in Fig 4 (c), which allows participants to easily find the tasks to be performed. For dynamic requests, we support informing participants of task requests through notifications as shown in Fig 5 (a). Organizers can generate and request tasks from specific participants, for example, by setting that any person who is detected entering a certain area is notified. Location information can be obtained using region monitoring technology such as Geofence and iBeacon.
Monetary/Non-monetary reward (F4): ParmoSense supports both monetary and non-monetary rewards to compensate participants for their contributions. The monetary rewards supported are discount coupons that can be used at restaurants, cafe and so on, as shown in Fig 5 (b). Since the discount coupon is linked with purchasing behavior, it is effective for motivating participation in scenarios such as sightseeing. For non-monetary rewards, ParmoSense supports the gamification mechanism. This includes awarding points for a contribution, competition mechanisms such as comparing a participant's degree of contribution to that of other participants, and virtual level-up elements by repeating the contribution. Also, depending on the demand for data collection, monetary/non-monetary rewards can be biased. For example, the expensive reward can be set for a low upload rate task or high priority task.

Feedbacks by visualization (F5):
ParmoSense also supports feedback not included in game features, for example, visualization of own contributions, and data/experience sharing. Visualization of own contributions is of two types: (i) plotting the pins of contributions on the map and (ii) scoring contributions with the non-monetary rewards detailed above. In addition, a data sharing function to share/visualize data such as participants' experiences and what different participants viewed, heard, or sensed (environmental conditions) is provided. For example, ParmoSense can plot sensing data uploaded by other participants on the map and share them on a timeline as shown in Fig 5 (c).
Editing of collected data (F6): ParmoSense helps organizers to pre-process collected data before detailed analysis. For example, it provides functions such as cleansing unnecessary data, selecting data to be used, and labeling the data. Also, since these data processing functions are designed to protect the original data, it can be restored at any time.
Browsing of collected data (F7): ParmoSense provides visualization tools for instant and easy confirmation of the collected data. There are many types of tools such as a tool for plotting data collected by all or each participant on the map, and a tool for displaying all data as a list. These tools can be used at any time even while sensing tasks are in progress.

Evaluation
ParmoSense has tackled two challenges to be solved: "limited support of essential functions in existing PMS platforms (C1)," and "difficulty of system construction and operation (C2)." In this section, we use the easiness of system construction and operation and the number of functions provided as the metrics, and we evaluate the performance of ParmoSense compared with conventional platforms.

Workload for system operation
We evaluated the difference in the development cost and the operation cost of ParmoSense compared to conventional platforms [15,16,17,18,19,20,21,22,23]. We used the workload cost for starting operation of the system (Preparation Workload) as the metric of comparison. The definition of Preparation Workload is as follows: Preparation Workload (W ) ✓ ✏ Preparation Workload (W ) is the total time required from the start of developing the PMS system to the start of sensing tasks, which is calculated by the sum of sub tasks (Eq. 1).
Where w 1 ∼ w 5 is the relative estimated time required for each task, as shown in Table 3. As reference, we defined the time required to install one application from a general application store, e.g., Google Play or AppStore as w x = 1.

✒ ✑
The estimated Preparation Workload on each platform is shown in Table 4 and the x-axis of Fig 6. The filled circle (•) represents the case where the simplest system on each platform was constructed. AWARE [15], Medusa [17], OpenDataKit [22] and GP-Selector [23] can be extended by programming as necessary. However, w 1 will increase linearly with the development of functionality.
Therefore, ParmoSense belongs to the group which needs low Preparation Workload, and can be operated as easily as other platforms in the same group. Also, conventional platforms [15,17,22,23] suffer from the problem where the time required for expansion tends to be long when trying to extend the systems' functionalities. In contrast, ParmoSense is designed to cover the necessary functions in advance, and thus, although Preparation Workload required is equivalent to other platforms, it achieves higher functionality.

Variety of functions
We evaluated the diversity of functions that ParmoSense provides, compared to conventional platforms [15,16,17,18,19,20,21,22,23]. As a metric for comparison, we use the following fulfillment status of functions (Function Score): Function Score (S) ✓ ✏ Table 1 shows all the functions supported by each of the existing PMS platforms. The score of each function s 1 ∼ s 8 is determined by the implementation status of each function (F1 ∼ F8) of Table 1, and Function Score (S) is calculated by the sum of them (Eq. 2).
The estimated Function Score on each platform is shown in Table 4 and the y-axis of Fig 6. While conventional platforms have moderate scores (S = 4 ± 1), ParmoSense has a higher score (S = 8) due to comprehensively supporting all the functions provided in existing platforms, and incorporating motivating functions, which are missing on all existing platforms. Furthermore, the advantage of ParmoSense will be even higher if we consider the effect brought by the combination of these functions. For example, ParmoSense can provide combinations of multiple motivating functions, such as gamification, rewards, and interruptions, which can not be provided on other existing platforms, and which may be more effective than using a single motivating strategy/function, since they cater for more participant preferences.

Case studies
We conducted 19 case studies with ParmoSense over four years. Prior to the evaluation, we released ParmoSense to general application stores (Google Play, AppStore). We collaborated with various organizers to design and build 19 scenarios, and then deployed them via the client application. Members of the general public who had downloaded the application acted as participants.
Our aim in doing these case studies was two-fold. First, we wanted to validate that our implementations of the sensing, processing and motivating functions on ParmoSense would work well in real world PMS tasks. Second, we wanted to determine how effective the functions were in motivating participation and in supporting organizers to collect required data and to extract the required information. Such knowledge would allow us to improve the functions, or to recommend additional functions to be included in PMS systems. Table 5 shows the details of each scenario (start date, period, number of participants, and embedded functions). In following subsections, we discuss the sufficiency of ParmoSense functions in each scenario.

Overview of case studies
We categorize case studies into four groups according to the type of sensing tasks involved and provide an overview of each group in this section. We then describe how well the ParmoSense functions performed for each scenario.
Urban-data collection during workshops (S1-S6): S1-S6 are scenarios for workshop-style events, such as a Mapping-party [44,45,46], which is widely used in organizations such as OpenStreetMap. The overview of each scenario is as follows: Scenario S1-S2 (Mapping-party) These scenarios were aimed at collecting unmapped geographical data. In this event, we collected information on trees (names and positions) in our university campus.

Scenario S3 (FixMyStreet)
This scenario was for collecting dynamic geographical data such as road breakages, graffiti, and street lamp failures by imitating the mechanism on FixMyStreet [14].
Scenario S4-S6 (Urban-planning) These scenarios involved surveying existing POIs such as local buildings, public facilities, and nature, for urban planning.
Through the deployment of these scenarios, we obtained the following knowledge: • Because participants were willing to attend the event by themselves, we confirmed that the set of motivating functions on ParmoSense are enough for getting sufficient contributions from the general public.
• The following functions of processing functions worked effectively: -Data labeling function -Cleansing function for unnecessary data -Automatic mosaic function by face recognition -Data export function Urban-data collection during sightseeing (S7-S11): S7-S9 are scenarios for sightseeing PMS, e.g., with an information sharing tool for tourists, or an urban sensing system mimicking a tourist guide. The overview of each scenario is as follows: Scenario S7 (Experience-sharing) This scenario involved sharing discovered sightseeing spots among a group of tourists. To encourage positive posting among participants, we used points as the motivating function (F4).

Scenario S8 (Tourist-guidance)
This scenario involved collecting data such as photos and behavior logs from tourists, while providing sightseeing information through a virtual tour guide [41]. We provided a data editing function (F6) for organizers to edit collected data after the event.
Scenario S9-S11 (Multi-type-requests) The scenario was for comprehensively collecting environmental data of sightseeing spots, by requesting it in various ways from tourists. To encourage active contribution of participants, we provided all the available motivating functions such as static/dynamic request (F3), points and coupons (F4). Fig 7 shows Screenshots of ParmoSense Client which are providing motivating functions including the dynamic points controls based on the demand for sensing tasks. Fig 8 visualizes participants' trajectories when they use ParmoSense which has/does not have motivating functions (F4). The area marked with dashed lines shows motivating functions (weighted points) effectively work to change user's behaviors by dynamically weighting points which can be earned. It suggests motivating functions in ParmoSense could contribute to solving the spatio-temporal coverage issue of general PMS systems. The detail analysis of these case studies (S9 and S10) are provided in our other papers [42,43]. Nov. 22nd, 2016 3 days 14 S9 [42] Nov. 26th, 2017 1 day 30 S10 [43] Jul. 29th, 2020 1 day 10 S11 Oct. 5th -Nov. 4th, 2020 1 day/person 108 S12 Jan. 21st, 2016 10 days 12 S13 Apr. 20th, 2017 14 days 83 S14 Mar. 12th, 2016 1 day 10 S15 Nov. 13th, 2016 1 day 48 S16 Feb. 25th, 2017 1 day 25 S17 June 11th, 2017 2 days 18 S18 July 24th, 2017 4 hours 100 S19 Sept. 24th, 2017 6 hours 200  Through these case studies, we observed the following on the effectiveness of the implemented functions for scenarios belonging to this category: • In the case of sightseeing, motivating functions were essential because tourists participated in the scenario opportunistically.
• Collecting factors, e.g., points and sharing information with each other, were more effective than the competitive factors, e.g., score and ranking, at motivating participant's continuous participation because "sightseeing" was their primary purpose and collecting factors were more relevant. Figure 9: Result of human-behavior investigation in Scenario S15.
Urban-data collection in daily life (S12-S13): The data suitable for collecting by PMS are often "continuous" and "long-term" data, because PMS is a sustainable sensing mechanism to realize comprehensive spatio-temporal data collection without any infrastructure due to the use of the many mobile devices dispersed in the city. S12 and S13 were scenarios for collecting such long-term and continuous data. The overview of each scenario is as follows: Scenario S12 (Static-request) This scenario was for periodically collecting information that changes day by day, e.g., bulletin board information at a facility, and temperature of a place. To encourage participation, we set a limit on the number of participant contributions resulting from static requests (F3) that would be accepted, which is similar to the Audition mechanism [17]. Scenario S13 (Dynamic-request) This scenario was for conducting questionnaires linked to location information by interrupting participants. Push notifications were used, and participants' location and behavior were taken into consideration, to provide dynamic requests (F3). No maps, rewards, or visualizations were included.
Through the case studies involving these scenarios, we concluded the following regarding the effectiveness of Par-moSense functions intended for urban-data collection in daily life: • In the case of S12, due to the use of static requests through placing pins on a map, the achievement of sensing depended on the active and continuous contribution of participants themselves. The following functions worked efficiently when motivating contribution: -Virtual rewards (e.g., point, ranking, level) -Monetary incentives -Visualization of contributions of the participant and of other participants • Restricting the number of contributions on static requests was effective for encouraging continuous participation because it prevented point inflation, but it also caused the leaving of some participants. • In the case of S13, we found that contribution of participants improved when the timing of requests was based on participant's location and behavior, e.g., participants were more likely to respond to requests if the location of the sensing task was near to their present location.
Human-behavior investigation on events (S14-S19): ParmoSense can not only collect urban environmental data, but also data of "people" existing in the city. S14-S19 were scenarios created for investigating human behavior during various situations: events, daily life, sightseeing and so on. The overview of each scenario is as follows: Scenario S14-S16 (Stamp-rally) These scenarios was for investigating the behavior of people who participate in the electronic "stamp rally" which is a process where visitors are instructed to go around to certain places, events etc. location [47]. A physical (electronic) stamp is put in a predetermined place, and when a visitor arrives, a stamp is stuck on a sheet (application). Reward is given based on the number of stamps accumulated. The rewards include coupons, prizes, points and ranking as non-monetary/monetary rewards (F4). We provided a data browsing function for visualizing participants' behavior (F7). Fig 9 shows examples of visualization of participants' behavior.
Scenario S17 (Sightseeing) This scenario was for collecting participants' movement data linked with the location information, to investigate the participants' behavior tendencies during sightseeing. To continuously sense tourists' behavior, we used an implicit sensing function that ran in the background (F1).
Scenario S18-S19 (w/external-sensor) These scenarios was used to collect participants' movement and heart rate data linked to location information, to be used to construct a database for human behavior analysis. We added a function to connect a wearable sensor to external sensors (F1) via BLE, for collecting data.
Through these case studies, we learned the following about PMS for human-behavior investigation: • To analyze the behavior of a person, there is a need to visualize behavior logs in various ways. We found that the functions provided by ParmoSense such as GPS trace, and Chord diagram listings worked effectively.
• Since these scenarios were aimed at sensing of human behavior affected by the specific function, it was necessary to suppress interference of elements, which is except for the function to be evaluated, to people. We confirmed that ParmoSense can minimize the number of unnecessary functions because of module-based design.
• It is sometimes necessary to collect data with high frequency over a long term (S18 and S19 were scenarios that required acquiring the 9-axis sensor data at 100 Hz over several hours). In this case, we found that functions such as continuous data acquisition in the background worked efficiently.

Survey and discussion
In the following sections, we summarize and discuss the results of the subjective surveys held in each case study. We asked each participant and organizer questions about the usability and performance of ParmoSense.

Overview
Participants in nine scenarios (S2, S4-S6, S9, S13-S16) were given the questionnaire. These scenarios had many ordinary citizens participating, and include at least one motivating function. The total number of participants who answered the questionnaire was 201. About 70% of participants were in their 20s, however, various age groups ranging 10s to 80s were targeted. Attributes of participants are listed below: • Citizens living in the local area, men and women of all ages (S2, S4, S15-S16) • Undergraduate students majoring in architectural engineering in their 20s (unfamiliar with the information science) (S5, S6) • Graduate students majoring in information science (S9, S13) We also distributed questionnaires to the organizers of five scenarios (S2, S4-S6, S9). The skills of the organizers varied from those with high IT skills like IT engineer to those with low IT skills like students majoring in fields other than IT. The attributes of the organizers are listed below: • Employee of regional public facility (S2) • IT engineer (S4) • Undergraduate students majoring in architectural engineering (S5, S6) • Graduate students majoring in information science (S9)

Survey of participants
To evaluate the usability of ParmoSense Client, we asked participants the questions which are listed in Table 6. we asked Q1 ("Was ParmoSense Client easy to use?") with a 5-point Likert scale (5: very easy to use -1: very hard to

Q3
Was the event using ParmoSense fun? (

Q4
Why did you think so about the answer to Q3? (except S13) (Open-ended question) -

Q5
What factors will encourage you to participate in experiments? (Non-participants of S13) (Open-ended question) use), and then asked for the reason in Q2. The total number of answers was 196. The breakdown of answers is shown in Table 6. The average score was 3.4 and about 40% of participants answered "easy to use." Participants who answered "easy to use" had the following to say: • "We could operate intuitively because of simple application design." (S4, S6, S16) • "Map visualization function is helpful for understanding the collected data having position information." (S4, S5, S6) Those who answered "hard to use" gave the following comments: • "It was difficult to use ParmoSense application because I was not used to smartphones." (S4, S15) • "ParmoSense application was not suitable for long-term use because this application consumes battery more than expected." (S4, S5, S6) • "Unused functions should be invisible." (S13) Overall, we found that using ParmoSense was easy for participants who use smartphones on a daily basis regardless of age. In the case of local events where elderly persons also attended, some seniors felt that it was difficult to use a smartphone. However, this is not a particular problem of ParmoSense. As the use case diversifies, the information displayed on the screen also diversifies. Thus, to solve this issue without programming skills, it is necessary to modularize the screen configuration of the application and also to enable screen layout design in Scenario tools according to the use case.
Next, we asked Q3 ("Was the event using ParmoSense fun?") with a 5-point Likert scale (5: very fun -1: not fun at all) in target scenarios except for S13. Additionally, we also asked Q4 to solicit free responses from participants on how motivating functions such as Reward and Visualization affected their sensing behavior. The total number of answers was 103. The breakdown of answers is shown in Table 6. The average score from the responses was 4.2 and about 80% of participants answered "fun." In Q4, we obtained the following comments: • "It is pleasant to see the behavior of other people and other groups by checking the pins on the map and timeline in real time." (S4, S12) • "Ranking function and leveling function makes it fun and encouraging." (S4, S5, S9) • "This application increased the fun of the stamp rally." (S15, S16)

Survey of non-participants
We also provided questionnaires to non-participants in the free participation scenario (S13) in order to explore the reasons why they did not participate in the sensing tasks. (Open-ended question) - In scenario S13, 35 of 118 candidates did not respond to the request to attend our experiment. We asked these 35 non-participants the open-ended question Q5 ("What factors will encourage you to participate in experiments?"). The factors that promoted participation were related to ease of participation (17 people), battery consumption concerns (10 people), rewards (13 people), and benefit or convenience (12 people) respectively. Furthermore, four people were concerned about privacy, such as "requiring personal information (e.g., e-mail address)" and "cannot be anonymous." Almost half of the non-participants pointed out that the number of steps they needed to take in order to participate was inconvenient. To minimize the steps to participate in multiple scenarios, we first required participants to install ParmoSense Client. After installing the application, participants read a QR code of each scenario in order to participate in that particular scenario. However, people who participated in only one scenario felt that this is complicated and that having a single, pre-configured application for a specific scenario might be easier.
Also, to make the registration and login process easier, we adopted Google Authentication (using Gmail address) and supported auto-login. This is a standard method used in many applications. However, some participants felt that our application could collect private data such as the e-mail address. In the future, we aim to address this by introducing an anonymous participation mode that does not require participant registration.

Survey of organizers
To evaluate the effect of ParmoSense from the view of organizers and the usability of processing functions, we conducted a questionnaire with organizers. The questions for organizers are listed in Table 7. Q6-Q11 were answered with 4-point Likert scales (4: strongly agree -1: strongly disagree), Q12 was a free-response question.
To evaluate the organizer's burden indicated in Fig 1 -Distributing-phase, we asked about Q6 (Ease of creating and distributing scenarios) to organizers. Two people answered "4" and the other three answered "3," giving an average score of 3.4. Although the IT skills of the organizers were quite different, all of them felt that ParmoSense was easy to use for PMS.
Next, we asked Q7 (Whether they were able to collect desired data) and Q8 (Performance of motivating function), to get feedback from organizers about Fig 1 -Sensing-phase. Four organizers answered "4" and one answered "3" in Q7, while two answered "4" and the other three answered "3" in Q8. Consequently, we can say that ParmoSense allows organizers to collect desired data and to motivate participants to contribute the sensing continuously, from a organizer's objective perspective.
To evaluate Fig 1 -Processing-phase, we asked the organizers about the Q9 (Usability of Data tools) and Q10 (Availability of output data). Two organizers answered "4" and two answered "3" and another organizer answered "2" for Q9, while one organizer answered "4" and three answered "3" and other one answered "2" for Q10. The average scores were 3.2 and 3.0 for Q9 and Q10, respectively. According to these results, we can confirm that the Data tools of ParmoSense work substantially fine.
Additionally, we interviewed the organizers who answered "2" in Q9 and Q10, respectively. The organizer whose answer for Q9 is "2" said "Unexpected data was downloaded when exporting data for each tag after tagging data." Therefore, as an additional question, we asked "Is it easy to use if data could be successfully downloaded?," and we got the answer "I think so." This shows that although it is necessary to improve the flexibility in the export function, it seems that the usability of Web editor meets specific service standards. Next, the organizer whose answer for Q10 is "2" said "When doing Web visualization, the operation became heavy and the PC froze many times." About 700 photos were collected in this organizer's event. This data amount was too large to display all photos at once. In the future, it is necessary to set an upper limit on the number of photos that can be displayed, or reduce the image size according to the number of images before uploading.
Finally, three organizers answered "4", two answered "3" in Q11 (Whether to use ParmoSense again). When we asked the reason in Q12, the following answers were obtained: • "It can find places where the participants are interested in." • "It can easily collect data with location information and can edit detailed information later. In addition, by visualizing the data, participants can understand how the data is used intuitively." From these results, we found that ParmoSense is useful for human-behavior analysis and feedback of results to participants and it is easy to edit data.

Conclusion
Demands for the assessment of the urban environments through PMS systems are spread not only in the information science field, but also in a wide range of areas, such as urban planning and public services. In such areas, organizers are not always familiar with information technology. Therefore, ParmoSense was designed to serve as a useful tool to collect the urban environmental information easily and flexible regardless of organizers' IT skills. ParmoSense allows organizers to construct, distribute, and introduce various types of PMS system for various purposes by modularizing functions and describing combinations and settings as "scenario." In PMS systems that involve ordinary citizens for collecting data, motivating participants is also important for continuous system operation. Thus, ParmoSense supported several motivating functions as an incorporated feature, which was not supported on conventional platforms.
Through performance comparison in the perspective of varieties of functions and preparation workload with existing PMS platforms, we confirmed ParmoSense shows the best cost-performance. In addition, through 19 case studies over four years, we confirmed that ParmoSense could be compatible flexibly with various sensing tasks, motivate methods, and data processing. Moreover, ParmoSense can suppress the burden of organizers and participants in system operation, and we found that it has higher cost performance than any conventional platforms. On the other hand, we also found ParmoSense has not been a "magic bullet" solution that can be used in every situation. ParmoSense should improve privacy and security by allowing organizers to control them based on the sensing purpose. In addition, to operate scenarios that involve a more significant number of participants, robustness and scalability should be guaranteed based on distributed processing and prediction of score update and so on.