Increase your forecast accuracy with time collection clustering

Time collection are sequences of knowledge factors that happen in successive order over some time period. We regularly analyze these information factors to make higher enterprise selections or achieve aggressive benefits. An instance is Shimamura Music, who used Amazon Forecast to enhance scarcity charges and enhance enterprise effectivity. One other nice instance is Arneg, who used Forecast to predict upkeep wants.

AWS gives varied providers catered to time collection information which might be low code/no code, which each machine studying (ML) and non-ML practitioners can use for constructing ML options. These consists of libraries and providers like AutoGluon, Amazon SageMaker Canvas, Amazon SageMaker Information Wrangler, Amazon SageMaker Autopilot, and Amazon Forecast.

On this put up, we search to separate a time collection dataset into particular person clusters that exhibit the next diploma of similarity between its information factors and cut back noise. The aim is to enhance accuracy by both coaching a worldwide mannequin that incorporates the cluster configuration or have native fashions particular to every cluster.

We discover the right way to extract traits, additionally known as options, from time collection information utilizing the TSFresh library—a Python package deal for computing a lot of time collection traits—and carry out clustering utilizing the Okay-Means algorithm applied within the scikit-learn library.

We use the Time Sequence Clustering utilizing TSFresh + KMeans pocket book, which is accessible on our GitHub repo. We advocate operating this pocket book on Amazon SageMaker Studio, a web-based, built-in improvement setting (IDE) for ML.

Answer overview

Clustering is an unsupervised ML approach that teams objects collectively primarily based on a distance metric. The Euclidean distance is mostly used for non-sequential datasets. Nevertheless, as a result of a time collection inherently has a sequence (timestamp), the Euclidean distance doesn’t work properly when used immediately on time collection as a result of it’s invariant to time shifts, ignoring the time dimension of knowledge. For a extra detailed clarification, check with Time Sequence Classification and Clustering with Python. A greater distance metric that works immediately on time collection is Dynamic Time Warping (DTW). For an instance of clustering primarily based on this metric, check with Cluster time collection information to be used with Amazon Forecast.

On this put up, we generate options from the time collection dataset utilizing the TSFresh Python library for information extraction. TSFresh is a library that calculates a lot of time collection traits, which embody the usual deviation, quantile, and Fourier entropy, amongst others. This permits us to take away the time dimensionality of the dataset and apply widespread methods that work for information with flattened codecs. Along with TSFresh, we additionally use StandardScaler, which standardizes options by eradicating the imply and scaling to unit variance, and Principal element evaluation (PCA) to carry out dimensionality discount. Scaling reduces the space between information factors, which in flip promotes stability within the mannequin coaching course of, and dimensionality discount permits the mannequin to study from fewer options whereas retaining the main tendencies and patterns, thereby enabling extra environment friendly coaching.

Information loading

For this instance, we use the UCI On-line Retail II Information Set and carry out primary information cleaning and preparation steps as detailed within the Information Cleansing and Preparation pocket book.

Characteristic extraction with TSFresh

Let’s begin through the use of TSFresh to extract options from our time collection dataset:

from tsfresh import extract_features
extracted_features = extract_features(
    df_final, 
    column_id="StockCode", 
    column_sort="timestamp")

Word that our information has been transformed from a time collection to a desk evaluating StockCode values vs. Characteristic values.

Subsequent, we drop all options with n/a values by using the dropna methodology:

extracted_features_cleaned=extracted_features
extracted_features_cleaned=extracted_features_cleaned.dropna(axis=1)

Then we scale the options utilizing StandardScaler. The values within the extracted options encompass each unfavorable and optimistic values. Subsequently, we use StandardScaler as a substitute of MinMaxScaler:

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
extracted_features_cleaned_std = scaler.fit_transform(extracted_features_cleaned)

We use PCA to do dimensionality discount:

from sklearn.decomposition import PCA
pca = PCA()
pca.match(extracted_features_cleaned_std)

And we decide the optimum variety of parts for PCA:

plt.determine(figsize=(20,10))
plt.grid()
plt.plot(np.cumsum(pca.explained_variance_ratio_))
plt.xlabel('variety of parts')
plt.ylabel('cumulative defined variance')

The defined variance ratio is the share of variance attributed to every of the chosen parts. Usually, you identify the variety of parts to incorporate in your mannequin by cumulatively including the defined variance ratio of every element till you attain 0.8–0.9 to keep away from overfitting. The optimum worth often happens on the elbow.

As proven within the following chart, the elbow worth is roughly 100. Subsequently, we use 100 because the variety of parts for PCA.

Clustering with Okay-Means

Now let’s use Okay-Means with the Euclidean distance metric for clustering. Within the following code snippet, we decide the optimum variety of clusters. Including extra clusters decreases the inertia worth, however it additionally decreases the knowledge contained in every cluster. Moreover, extra clusters means extra native fashions to take care of. Subsequently, we wish to have a small cluster measurement with a comparatively low inertia worth. The elbow heuristic works properly for locating the optimum variety of clusters.

from sklearn.cluster import KMeans
wcss = []
for i in vary(1,10):
    km = KMeans(n_clusters=i) 
    km.match(scores_pca)
    wcss.append(km.inertia_)
plt.determine(figsize=(20,10))
plt.grid()
plt.plot(vary(1,10),wcss,marker="o",linestyle="--")
plt.xlabel('variety of clusters')
plt.ylabel('WCSSS')

The next chart visualizes our findings.

Primarily based on this chart, we now have determined to make use of two clusters for Okay-Means. We made this choice as a result of the within-cluster sum of squares (WCSS) decreases on the highest fee between one and two clusters. It’s essential to steadiness ease of upkeep with mannequin efficiency and complexity, as a result of though WCSS continues to lower with extra clusters, extra clusters enhance the danger of overfitting. Moreover, slight variations within the dataset can unexpectedly cut back accuracy.

It’s essential to notice that each clustering strategies, Okay-Means with Euclidian distance (mentioned on this put up) and Okay-means algorithm with DTW, have their strengths and weaknesses. The very best strategy relies on the character of your information and the forecasting strategies you’re utilizing. Subsequently, we extremely advocate experimenting with each approaches and evaluating their efficiency to realize a extra holistic understanding of your information.

Conclusion

On this put up, we mentioned the highly effective methods of characteristic extraction and clustering for time collection information. Particularly, we confirmed the right way to use TSFresh, a well-liked Python library for characteristic extraction, to preprocess your time collection information and procure significant options.

When the clustering step is full, you may prepare a number of Forecast fashions for every cluster, or use the cluster configuration as a characteristic. Seek advice from the Amazon Forecast Developer Information for details about information ingestion, predictor coaching, and producing forecasts. If in case you have merchandise metadata and associated time collection information, you may also embody these as enter datasets for coaching in Forecast. For extra data, check with Begin your profitable journey with time collection forecasting with Amazon Forecast.

References

Concerning the Authors

Aleksandr Patrushev is AI/ML Specialist Options Architect at AWS, primarily based in Luxembourg. He’s passionate concerning the cloud and machine studying, and the way in which they might change the world. Outdoors work, he enjoys mountain climbing, sports activities, and spending time together with his household.

Chong En Lim is a Options Architect at AWS. He’s all the time exploring methods to assist prospects innovate and enhance their workflows. In his free time, he loves watching anime and listening to music.

Egor Miasnikov is a Options Architect at AWS primarily based in Germany. He’s passionate concerning the digital transformation of our lives, companies, and the world itself, in addition to the position of synthetic intelligence on this transformation. Outdoors of labor, he enjoys studying journey books, mountain climbing, and spending time together with his household.