Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Monitoring sleep posture is important for avoiding bedsores after surgery, reducing apnea events, tracking the progression of Parkinson's disease, and even alerting epilepsy patients to potentially fatal sleep postures. Today, there is no easy way to track sleep postures. Past work has proposed installing cameras in the bedroom, mounting accelerometers on the subject's chest, or embedding pressure sensors in their bedsheets. Unfortunately, such solutions jeopardize either the privacy of the user or their sleep comfort. In this paper, we introduce BodyCompass, the first RF-based system that provides accurate sleep posture monitoring overnight in the user's own home. BodyCompass works by studying the RF reflections in the environment. It disentangles RF signals that bounced off the subject's body from other multipath signals. It then analyzes those signals via a custom machine learning algorithm to infer the subject's sleep posture. BodyCompass is easily transferable and can apply to new homes and users with minimal effort. We empirically evaluate BodyCompass using over 200 nights of sleep data from 26 subjects in their own homes. Our results show that, given one week, one night, or 16 minutes of labeled data from the subject, BodyCompass's corresponding accuracy is 94%, 87%, and 84%, respectively.

Shichao Yue

Yuzhe Yang

Hao Wang

Hariharan Rahul

Dina Katabi

Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies

BodyCompass

Tooth brushing monitors have the potential to enhance oral hygiene and encourage the development of healthy brushing habits. However, previous studies fall short of recognizing each tooth due to limitations in external sensors and variations among users. To address these challenges, we present ToothFairy, a real-time tooth-by-tooth brushing monitor that uses earphone reverse signals captured within the oral cavity to identify each tooth during brushing. The key component of ToothFairy is a novel bone-conducted acoustic attenuation model, which quantifies sound propagation within the oral cavity. This model eliminates the need for machine learning and can be calibrated with just one second of brushing data for each tooth by a new user. ToothFairy also addresses practical issues such as brushing detection and tooth region determination. Results from extensive experiments, involving 10 volunteers and 25 combinations of five commercial off-the-shelf toothbrush and earphone models each, show that ToothFairy achieves tooth recognition with an average accuracy of 90.5%.

Yang Wang

Feng Hong

Yufei Jiang

Chenyu Bao

Chao Liu

Zhongwen Guo

ToothFairy

牙仙

The development of a machine-learning-based human activity recognition (HAR) system using body-worn sensors is mainly composed of three phases: data collection, model training, and evaluation. During data collection, the HAR developer collects labeled data from participants wearing inertial sensors. In the model training phase, the developer trains the HAR model on the collected training data. In the evaluation phase, the developer evaluates the trained HAR model on the collected test data. When the HAR model cannot achieve the target recognition accuracy, the developer iterates the above procedures by taking certain measures, including collecting additional training data, until the re-trained model achieves the target accuracy. However, collecting labeled data for HAR requires additional time and incurs high monetary costs. In addition, it is difficult to determine the amount and type of data to collect for achieving the target accuracy while reducing costs. To address this issue, this paper proposes a new method that predicts the performance improvement of the current HAR model, i.e., it determines the level of performance improvement achievable by re-training the HAR model with additional data, before collecting the additional data. Thus, the method enables the HAR developer to establish a strategy for additional data collection by providing advice such as "If labeled data for the Walking and Running activities from two additional participants is collected, the HAR accuracy of the current HAR model for Walking will improve by 20%." To achieve this, a neural network called AIP-Net is proposed to estimate the improvement in performance by analyzing the feature space of the current HAR model using the proposed entropy-based attention mechanism. The performance of AIP-Net was evaluated on eight HAR datasets using leave-one-dataset-out cross-validation.

Kei Tanigaki

Tze Chuin Teoh

Naoya Yoshimura

Takuya Maekawa

Takahiro Hara

Predicting Performance Improvement of Human Activity Recognition Model by Additional Data Collection

通过额外数据收集预测人体活动识别模型性能提升

This study presents a new neural network model for recognizing manual works using body-worn accelerometers in industrial settings, named Lightweight Ordered-work Segmentation Network (LOS-Net). In industrial domains, a human worker typically repetitively performs a set of predefined processes, with each process consisting of a sequence of activities in a predefined order. State-of-the-art activity recognition models, such as encoder-decoder models, have numerous trainable parameters, making their training difficult in industrial domains because of the consequent substantial cost for preparing a large amount of labeled data. In contrast, the LOS-Net is designed to be trained on a limited amount of training data. Specifically, the decoder in the LOS-Net has few trainable parameters and is designed to capture only the necessary information for precise recognition of ordered works. These are (i) the boundary information between consecutive activities, because a transition in the performed activities is generally associated with the trend change of the sensor data collected during the manual works and (ii) long-term context regarding the ordered works, e.g., information about the previous and next activity, which is useful for recognizing the current activity. This information is obtained by introducing a module that can collect it at distant time steps using few trainable parameters. Moreover, the LOS-Net can refine the activity estimation by the decoder by incorporating prior knowledge regarding the order of activities. We demonstrate the effectiveness of the LOS-Net using sensor data collected from workers in actual factories and a logistics center, and show that it can achieve state-of-the-art performance.

Atsushi Wada

Yasuo Namioka

Acceleration-based Activity Recognition of Repetitive Works with Lightweight Ordered-work Segmentation Network

基于加速度的重复性工作活动识别与轻量级有序工作分割网络

A major bottleneck in training robust Human-Activity Recognition models (HAR) is the need for large-scale labeled sensor datasets. Because labeling large amounts of sensor data is an expensive task, unsupervised and semi-supervised learning techniques have emerged that can learn good features from the data without requiring any labels. In this paper, we extend this line of research and present a novel technique called Collaborative Self-Supervised Learning (ColloSSL) which leverages unlabeled data collected from multiple devices worn by a user to learn high-quality features of the data. A key insight that underpins the design of ColloSSL is that unlabeled sensor datasets simultaneously captured by multiple devices can be viewed as natural transformations of each other, and leveraged to generate a supervisory signal for representation learning. We present three technical innovations to extend conventional self-supervised learning algorithms to a multi-device setting: a Device Selection approach which selects positive and negative devices to enable contrastive learning, a Contrastive Sampling algorithm which samples positive and negative examples in a multi-device setting, and a loss function called Multi-view Contrastive Loss which extends standard contrastive loss to a multi-device setting. Our experimental results on three multi-device datasets show that ColloSSL outperforms both fully-supervised and semi-supervised learning techniques in majority of the experiment settings, resulting in an absolute increase of upto 7.9% in F1 score compared to the best performing baselines. We also show that ColloSSL outperforms the fully-supervised methods in a low-data regime, by just using one-tenth of the available labeled data in the best case.

Yash Jain

Chi Ian Tang

Chulhong Min

Fahim Kawsar

Akhil Mathur

ColloSSL

Federated Learning (FL), an emerging distributed machine learning framework that enables each client to collaboratively train a global model by sharing local knowledge without disclosing local private data, is vulnerable to backdoor model poisoning attacks. By compromising some users, the attacker manipulates their local training process, and uploads malicious gradient updates to poison the global model, resulting in the poisoned global model behaving abnormally on the sub-tasks specified by the malicious user. Prior research has proposed various strategies to mitigate backdoor attacks. However, existing FL backdoor defense methods affect the fairness of the FL system, while fair FL performance may not be robust. Motivated by these concerns, in this paper, we propose Self-Awareness Revision (SARS), a personalized FL framework designed to resist backdoor attacks and ensure the fairness of the FL system. SARS consists of two key modules: adaptation feature extraction and knowledge mapping. In the adaptation feature extraction module, benign users can adaptively extract clean global knowledge with self-awareness and self-revision of the backdoor knowledge transferred from the global model. Based on the previous module, users can effectively ensure the correct mapping of clean sample features and labels. Through extensive experimental results, SARS can defend against backdoor attacks and improve the fairness of the FL system by comparing several state-of-the-art FL backdoor defenses or fair FL methods, including FedAvg, Ditto, WeakDP, FoolsGold, and FLAME.

Weibin Zhang

Youpeng Li

Lingling An

Bo Wan

Xuyu Wang

SARS: A Personalized Federated Learning Framework Towards Fairness and Robustness against Backdoor Attacks

SARS：一种面向公平性和抗后门攻击的个性化联邦学习框架

We propose TRAMBA, a hybrid transformer and Mamba architecture for acoustic and bone conduction speech enhancement, suitable for mobile and wearable platforms. Bone conduction speech enhancement has been impractical to adopt in mobile and wearable platforms for several reasons: (i) data collection is labor-intensive, resulting in scarcity; (ii) there exists a performance gap between state-of-art models with memory footprints of hundreds of MBs and methods better suited for resource-constrained systems. To adapt TRAMBA to vibration-based sensing modalities, we pre-train TRAMBA with audio speech datasets that are widely available. Then, users fine-tune with a small amount of bone conduction data. TRAMBA outperforms state-of-art GANs by up to 7.3% in Perceptual Evaluation of Speech Quality (PESQ) and 1.8% in Short-Time Objective Intelligibility (STOI), with an order of magnitude smaller memory footprint and an inference speed up of up to 465 times. We integrate TRAMBA into real systems and show that TRAMBA (i) improves battery life of wearables by up to 160% by requiring less data sampling and transmission; (ii) generates higher quality voice in noisy environments than over-the-air speech; (iii) requires a memory footprint of less than 20.0 MB.

Yueyuan Sui

Minghui Zhao

Junxi Xia

Xiaofan Jiang

Stephen Xia

TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platforms

TRAMBA：一种用于移动和可穿戴平台上实用音频和骨传导语音超分辨率与增强的混合Transformer和Mamba架构

Passive haptic learning (PHL) is a method for training motor skills via intensive repetition of haptic stimuli while a user is focused on other tasks. For the practical application of PHL to music education, we propose passive haptic rehearsal (PHR) where PHL is combined with deliberate active practice. We designed a piano teaching system that includes haptic gloves compatible with daily wear, a Casio keyboard with light-up keys, and an online learning portal that enables users to track performance, choose lessons, and connect with the gloves and keyboard. We conducted a longitudinal two-week study in the wild, where 36 participants with musical experience learned to play two piano songs with and without PHR. For 20 participants with complete and valid data, we found that PHR boosted the learning rate for the matching accuracy by 49.7% but did not have a significant effect on learning the notes' rhythm. Participants across all skill levels in the study require approximately two days less to reach mastery on the songs practiced when using PHR. We also confirmed that PHR boosts recall between active practice sessions. We hope that our results and system will enable the deployment of PHL beyond the laboratory.

Tan Gemicioglu

Elijah Hopper

Brahmi Dwivedi

Richa Kulkarni

Asha Bhandarkar

Priyanka Rajan

Nathan Eng

Adithya Ramanujam

Charles Ramey

Scott M. Gilliland

Celeste Mason

Caitlyn Seim

Thad Starner

Passive Haptic Rehearsal for Augmented Piano Learning in the Wild

增强钢琴学习的被动触觉排练

The scarcity of publicly available cellular association traces hinders user location-based research and various data-driven services, highlighting the importance of data synthesis in this field. In this paper, we investigate the cellular association trace synthesis (CATS) problem, aiming to generate diverse and realistic cellular association traces based on road segment-based trajectories and corresponding departure times. To substantiate our research, we first gather substantial data, including road segment-based trajectories, base station (BS) distribution, and ground truths of cellular association traces. We then perform systematic data analysis to reveal technical challenges such as disparity in geographic spaces, complex and dynamic BS handover, and poor performance of single-dimension approaches. To address these challenges, we propose SynthCAT, a novel scheme that fuses model-based and data-driven approaches. Specifically, SynthCAT includes: i) A model-based coarse-grained cellular association trace generation component, encompassing GPS reference generation, weighted historical average time generation, Bayesian decision, and time mapping modules. This component establishes a unified GPS space to map road and BS spaces, generates initial time information, synthesizes coarse-grained spatial cellular association traces by following explicit BS handover rules, and maps the corresponding arrival time for each trace point; ii) A fine-grained cellular association trace generation component, which combines model-based and data-driven approaches. This employs a two-stage Autoencoder Generative Adversarial Network (AEGAN) to refine cellular association traces based on the coarse-grained ones. Extensive field experiments validate the efficacy of SynthCAT in terms of trace similarity to ground truths and its efficiency in supporting practical downstream applications.

Feng Lyu

Jie Zhang

Huali Lu

Huaqing Wu

Fan Wu

Yongmin Zhang

Yaoxue Zhang

SynthCAT: Synthesizing Cellular Association Traces with Fusion of Model-Based and Data-Driven Approaches

SynthCAT：基于模型和数据驱动方法融合的细胞关联轨迹合成

Smart glasses have become more prevalent as they provide an increasing number of applications for users. They store various types of private information or can access it via connections established with other devices. Therefore, there is a growing need for user identification on smart glasses. In this paper, we introduce a low-power and minimally-obtrusive system called SonicID, designed to authenticate users on glasses. SonicID extracts unique biometric information from users by scanning their faces with ultrasonic waves and utilizes this information to distinguish between different users, powered by a customized binary classifier with the ResNet-18 architecture. SonicID can authenticate users by scanning their face for 0.06 seconds. A user study involving 40 participants confirms that SonicID achieves a true positive rate of 97.4%, a false positive rate of 4.3%, and a balanced accuracy of 96.6% using just 1 minute of training data collected for each new user. This performance is relatively consistent across different remounting sessions and days. Given this promising performance, we further discuss the potential applications of SonicID and methods to improve its performance in the future.

Abbreviation	Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.
Journal Impact	3.45
Quartiles(Global)	TELECOMMUNICATIONS(Q2)

ISSN	2474-9567
h-index	36

Publication Information	Publisher: Association for Computing Machinery (ACM)，Publishing cycle: ，Journal Type: journal，Open Access Journals: No
Basic data	Year of publication: 2017，Proportion of original research papers: 0.00%，Self Citation Rate:N.A.， Gold OA Rate: 0.00%

Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies

Journal Citation Format

A journal article with 1 author

A journal article with 2 authors

A journal article with 3 authors

A journal article with 5 or more authors

Books Citation Format

Thesis Citation Format

Web sites Citation Format

Patent Citation Format

Staying up late manually editing references? ivySCI automatically matches journals and helps you generate references with a single click.

Share Submission Experience