Acceleration-Guided Diffusion Model for Multivariate Time Series Imputation.pdf

Apache IoTDB

16页

0次

2025-02-25

100墨值下载

Acceleration-Guided Diﬀusion Model for

Multivariate Time Series Imputation

Xinyu Yang

, Yu Sun

, Shaoxu Song

, Xiaojie Yuan

, and Xinyang Chen

sxsong@tsinghua.edu.cn

School of Computer Science and Technology, Harbin Institute of Technology,

Shenzhen, China

chenxinyang@hit.edu.cn

Abstract. Multivariate time series data are pervasive in various do-

mains, often plagued by missing values due to diverse reasons. Diﬀusion

models have demonstrated their prowess for imputing missing values

in time series by leveraging stochastic processes. Nonetheless, a persis-

tent challenge surfaces when diﬀusion models encounter the task of ac-

curately modeling time series data with quick changes. In response to

this challenge, we present the Acceleration-guided Diﬀusion model for

Multivariate time series Imputation (ADMI). Time-series representa-

tion learning is ﬁrst eﬀectively conducted through an acceleration-guided

masked modeling framework. Subsequently, representations with a spe-

cial care of quick changes are incorporated as guiding elements in the

diﬀusion model, utilizing the cross-attention mechanism. Thus our model

can self-adaptively adjust the weights associated with the representation

during the denoising process. Our experiments, conducted on real-world

datasets featuring genuine missing values, conclusively demonstrate the

superior performance of our ADMI model. It excels in both imputation

accuracy and the overall enhancement of downstream applications.

Keywords: Multivariate time series · Data imputation · Self-supervised

learning · Diﬀusion model.

1 Introduction

Incomplete time series data are common in various ﬁelds such as meteorology

[13], traﬃc [6], and medical treatment [24], impairing the performance of down-

stream analysis [11]. Researchers have developed various methods for imputing

missing values in multivariate time series, based on statistics [23], machine learn-

ing [6, 15], deep learning [22, 14], etc. Because of the powerful data generation

ability, generative models [17, 28] are widely used in imputing missing values.

As one of the most advanced generative models, the diﬀusion model has

demonstrated impressive results in imputing time series data [28, 1, 14], tak-

ing advantages of the multi-step noise-adding and denoising processes to model

M. Onizuka et al. (Eds.): DASFAA 2024, LNCS 14851, pp. 130, 2025.

https://doi.org/10.1007/978-981-97-5779-4_8

115–

College of Computer Science, DISSec, Nankai University, Tianjin, China

Tsinghua University, Beijing, China

yangxinyu@dbis.nankai.edu.cn, {sunyu,yuanxj}@nankai.edu.cn

Fig. 1. The imputation performance results over Beijing18 [13] with 10% missing values

of exiting diﬀusion-based method [28] and our ADMI over Beijing18 [13]. (a) For the

example segment on 2017/09/21-2017/09/22; (b) Temporal attention weights of CSDI

[28]; (c)(d) For all missing values and those with high acceleration (>1).

data distributions. However, as illustrated in Figure 1(a), diﬀusion-based meth-

ods, e.g., CSDI [28], SSSD [1] and PriSTI [14], face the challenge of imputing

missing values with quick changes, which are prevalent in time series data and

important to a broad range of real-world problems [2]. Moreover, Figure 1(b)

reﬂects the strength of each timestamp in focusing on other timestamps in the

self-attention mechanism when modeling the time series, with lighter colors rep-

resenting stronger attention. We can ﬁnd that modeling each timestamp will

aﬀect that for neighboring timestamps as well. Therefore, inaccurate modeling

data with quick changes can impact the global modeling of data changes, leading

to decreased overall imputation performance. The primary reason for aforesaid

two challenges stems from the innate nature of deep neural networks, which tend

to inherently smooth their outputs without speciﬁc guidance [21, 27].

Researchers usually utilize the acceleration [25] to describe how fast the time

series data change, by measuring the diﬀerence on value changes between neigh-

boring timestamps.

In Figures 1(c)(d), we observe that it is more diﬃcult for

existing diﬀusion-based models [28, 1, 14] to impute missing values with high

acceleration, compared with the whole missing values. Such results show that

constructing a more accurate modeling of high acceleration data is crucial for

improving the imputation accuracy of diﬀusion models.

Enlightened by this, we present the Acceleration-guided Diﬀusion model for

MultivariatetimeseriesImputation (ADMI). Our model exploits guidance rep-

resentations that are obtained through an acceleration-guided mask modeling

framework, to guide the diﬀusion model for imputation. Speciﬁcally, we ﬁrst

introduce a masking mechanism that prioritizes observations with higher accel-

erations to be masked, allowing the guidance representations to extract more

Please see the formal deﬁnition in Equation 1 in Section 3.1.

116 X. Yan

et al.

beneﬁcial information for eﬀective time series modeling. We pre-train the guid-

ance extractor for extracting guidance representations by utilizing the mask

prediction self-supervised task with the masking mechanism. For incorporating

guidance representations into the denoising of diﬀusion models, we further utilize

cross-attention to fuse guidance representations into temporal and channel de-

pendencies modeling. The fusion mechanisms allow the model to self-adaptively

adjust the weights of the guidance representation during the denoising process.

Our contributions are summarized as follows:

(1) We propose an acceleration-guided mask modeling framework and learning

the guidance representations by the self-supervised mask prediction task.

(2) We introduce the guidance fusion mechanisms to self-adaptively incorporate

guidance representations into the denoising process of the diﬀusion model.

(3) We conduct evaluations over real datasets with real-world missing values.

Experimental results demonstrate the superior performance in both imputation

accuracy and downstream applications of ADMI, boosted by the mask modeling

framework and the guidance fusion mechanisms.

2 Related Work

Our ADMI draws upon and contributes to two critical domains: time series data

imputation and self-supervised time series representation learning.

2.1 Time Series Data Imputation

Within the realm of traditional imputation methods for multivariate time series,

common tactics involve the use of statistical measurements such as mean, me-

dian [23], or the most recently observed value [3]. Since the matrix factorization

is good at recovering the sparse matrix, BTMF [6] and TIDER [15] use the low-

rank matrix factorization to impute missing values in multivariate time series.

Considering the powerful ability of deep learning techniques, they are introduced

into the time series data imputation. Various models are considered for multi-

variate time series data imputation, e.g., RNN in BRITS [4], self-attention in

TST [36] and SAITS [9], CNN in TimesNet [32]. Moreover, STCPA [33] utilizes

self-attention to ﬁll the spatiotemporal traﬃc speed data. With the relationship

between diﬀerent channels, GRIN [7] and DAMR [22] take advantage of graph

neural networks to impute missing data.

Generative models [5], like generative adversarial networks (GAN, as used in

GAIN [34], GAN-2-stage [16], and E

GAN [17]), learn the distribution of miss-

ing data to generate ﬁlling values. However, adversarial training of the generator

and discriminator can lead to mode collapse. To address this issue, SSGAN [19]

uses labels to guide the GAN towards convergence with the real data distribu-

tion in those datasets with classiﬁcation labels. On the other hand, with the

sequential generation approach, diﬀusion models tend to exhibit more stable

training, mitigating the mode collapse issue to a certain extent. CSDI [28] thus

Acceleration-Guided Diffusion Model 117

of 16

100墨值下载

相关文档

评论