This can be a joint submit by NXP SEMICONDUCTORS N.V. & AWS Machine Studying Options Lab (MLSL)
Machine studying (ML) is getting used throughout a variety of industries to extract actionable insights from knowledge to streamline processes and enhance income technology. On this submit, we show how NXP, an business chief within the semiconductor sector, collaborated with the AWS Machine Studying Options Lab (MLSL) to make use of ML strategies to optimize the allocation of the NXP analysis and growth (R&D) price range to maximise their long-term return on funding (ROI).
NXP directs its R&D efforts largely to the event of latest semiconductor options the place they see vital alternatives for progress. To outpace market progress, NXP invests in analysis and growth to increase or create main market positions, with an emphasis on fast-growing, sizable market segments. For this engagement, they sought to generate month-to-month gross sales forecasts for brand spanking new and present merchandise throughout totally different materials teams and enterprise strains. On this submit, we show how the MLSL and NXP employed Amazon Forecast and different customized fashions for long-term gross sales predictions for varied NXP merchandise.
“We engaged with the group of scientists and specialists at [the] Amazon Machine Studying Options Lab to construct an answer for predicting new product gross sales and perceive if and which extra options may assist inform [the] decision-making course of for optimizing R&D spending. Inside only a few weeks, the group delivered a number of options and analyses throughout a few of our enterprise strains, materials teams, and on [an] particular person product degree. MLSL delivered a gross sales forecast mannequin, which enhances our present manner of guide forecasting, and helped us mannequin the product lifecycle with novel machine studying approaches utilizing Amazon Forecast and Amazon SageMaker. Whereas conserving a relentless collaborative workstream with our group, MLSL helped us with upskilling our professionals in the case of scientific excellence and finest practices on ML growth utilizing AWS infrastructure.”
– Bart Zeeman, Strategist and Analyst at CTO workplace in NXP Semiconductors.
Objectives and use case
The aim of the engagement between NXP and the MLSL group is to foretell the general gross sales of NXP in varied finish markets. Generally, the NXP group is all in favour of macro-level gross sales that embrace the gross sales of assorted enterprise strains (BLs), which include a number of materials teams (MAGs). Moreover, the NXP group can be all in favour of predicting the product lifecycle of newly launched merchandise. The lifecycle of a product is split into 4 totally different phases (Introduction, Development, Maturity, and Decline). The product lifecycle prediction permits the NXP group to establish the income generated by every product to additional allocate R&D funding to the merchandise producing the very best quantities of gross sales or merchandise with the very best potential to maximise the ROI for R&D exercise. Moreover, they’ll predict the long-term gross sales on a micro degree, which provides them a bottom-up look on how their income modifications over time.
Within the following sections, we current the important thing challenges related to growing sturdy and environment friendly fashions for long-term gross sales forecasts. We additional describe the instinct behind varied modeling strategies employed to attain the specified accuracy. We then current the analysis of our last fashions, the place we examine the efficiency of the proposed fashions by way of gross sales prediction with the market specialists at NXP. We additionally show the efficiency of our state-of-the-art level cloud-based product lifecycle prediction algorithm.
Challenges
One of many challenges we confronted whereas utilizing fine-grained or micro-level modeling like product-level fashions on the market prediction was lacking gross sales knowledge. The lacking knowledge is the results of lack of gross sales throughout each month. Equally, for macro-level gross sales prediction, the size of the historic gross sales knowledge was restricted. Each the lacking gross sales knowledge and the restricted size of historic gross sales knowledge pose vital challenges by way of mannequin accuracy for long-term gross sales prediction into 2026. We noticed through the exploratory knowledge evaluation (EDA) that as we transfer from micro-level gross sales (product degree) to macro-level gross sales (BL degree), lacking values turn out to be much less vital. Nonetheless, the utmost size of historic gross sales knowledge (most size of 140 months) nonetheless posed vital challenges by way of mannequin accuracy.
Modeling strategies
After EDA, we centered on forecasting on the BL and MAG ranges and on the product degree for one of many largest finish markets (the car finish market) for NXP. Nonetheless, the options we developed may be prolonged to different finish markets. Modeling on the BL, MAG, or product degree has its personal execs and cons by way of mannequin efficiency and knowledge availability. The next desk summarizes such execs and cons for every degree. For macro-level gross sales prediction, we employed the Amazon Forecast AutoPredictor for our last answer. Equally, for micro-level gross sales prediction, we developed a novel level cloud-based method.
Macro gross sales prediction (top-down)
To foretell the lengthy phrases gross sales values (2026) on the macro degree, we examined varied strategies, together with Amazon Forecast, GluonTS, and N-BEATS (carried out in GluonTS and PyTorch). Total, Forecast outperformed all different strategies primarily based on a backtesting method (described within the Analysis Metrics part later on this submit) for macro-level gross sales prediction. We additionally in contrast the accuracy of AutoPredictor towards human predictions.
We additionally proposed utilizing N-BEATS attributable to its interpretative properties. N-BEATS is predicated on a quite simple however highly effective structure that makes use of an ensemble of feedforward networks that make use of the residual connections with stacked residual blocks for forecasting. This structure additional encodes the inductive bias in its structure to make the time sequence mannequin able to extracting development and seasonality (see the next determine). These interpretations had been generated utilizing PyTorch Forecasting.
Micro gross sales prediction (bottom-up)
On this part, we focus on a novel methodology developed to foretell the product lifecycle proven within the following determine whereas bearing in mind the chilly begin product. We carried out this methodology utilizing PyTorch on Amazon SageMaker Studio. First, we launched some extent cloud-based methodology. This methodology first converts gross sales knowledge into some extent cloud, the place every level represents gross sales knowledge at a sure age of the product. The purpose cloud-based neural community mannequin is additional educated utilizing this knowledge to be taught the parameters of the product lifecycle curve (see the next determine). On this method, we additionally integrated extra options, together with product description as a bag of phrases to sort out the chilly begin drawback for predicting the product lifecycle curve.
Time sequence as level cloud-based product lifecycle prediction
We developed a novel level cloud-based method to foretell the product lifecycle and micro-level gross sales predictions. We additionally integrated extra options to additional enhance the mannequin accuracy for the chilly begin product lifecycle predictions. These options embrace product fabrication strategies and different associated categorical data associated to the merchandise. Such extra knowledge may also help the mannequin predict gross sales of a brand new product even earlier than the product is launched in the marketplace (chilly begin). The next determine demonstrates the purpose cloud-based method. The mannequin takes the normalized gross sales and age of the product (variety of months for the reason that product is launched) as enter. Primarily based on these inputs, the mannequin learns parameters through the coaching utilizing gradient descent. Through the forecast part, the parameters together with the options of a chilly begin product are used for predicting the lifecycle. The big variety of lacking values within the knowledge on the product degree negatively impacts practically all the present time sequence fashions. This novel answer is predicated on the concepts of lifecycle modeling and treating time sequence knowledge as level clouds to mitigate the lacking values.
The next determine demonstrates how our level cloud-based lifecycle methodology addresses the lacking knowledge values and is able to predicting the product lifecycle with only a few coaching samples. The X-axis represents the age in time, and the Y-axis represents the gross sales of a product. Orange dots characterize the coaching samples, inexperienced dots characterize the testing samples, and the blue line demonstrates the anticipated lifecycle of a product by the mannequin.
Methodology
To foretell macro-level gross sales, we employed Amazon Forecast amongst different strategies. Equally, for micro gross sales, we developed a state-of-the-art level cloud-based customized mannequin. Forecast outperformed all different strategies by way of mannequin efficiency. We used Amazon SageMaker pocket book cases to create an information processing pipeline that extracted coaching examples from Amazon Easy Storage Service (Amazon S3). The coaching knowledge was additional used as enter for Forecast to coach a mannequin and predict long-term gross sales.
Coaching a time sequence mannequin utilizing Amazon Forecast consists of three principal steps. In step one, we imported the historic knowledge into Amazon S3. Second, a predictor was educated utilizing the historic knowledge. Lastly, we deployed the educated predictor to generate the forecast. On this part, we offer an in depth rationalization together with code snippets of every step.
We began by extracting the newest gross sales knowledge. This step included importing the dataset to Amazon S3 within the appropriate format. Amazon Forecast takes three columns as inputs: timestamp, item_id, and target_value (gross sales knowledge). The timestamp column incorporates the time of gross sales, which could possibly be formatted as hourly, every day, and so forth. The item_id column incorporates the title of the bought objects, and the target_value column incorporates gross sales values. Subsequent, we used the trail of coaching knowledge situated in Amazon S3, outlined the time sequence dataset frequency (H, D, W, M, Y), outlined a dataset title, and recognized the attributes of the dataset (mapped the respective columns within the dataset and their knowledge sorts). Subsequent, we known as the create_dataset operate from the Boto3 API to create a dataset with attributes similar to Area, DatasetType, DatasetName, DatasetFrequency, and Schema. This operate returned a JSON object that contained the Amazon Useful resource Title (ARN). This ARN was subsequently used within the following steps. See the next code:
dataset_path = "PATH_OF_DATASET_IN_S3"
DATASET_FREQUENCY = "M" # Frequency of dataset (H, D, W, M, Y)
TS_DATASET_NAME = "NAME_OF_THE_DATASET"
TS_SCHEMA = {
"Attributes":[
{
"AttributeName":"item_id",
"AttributeType":"string"
},
{
"AttributeName":"timestamp",
"AttributeType":"timestamp"
},
{
"AttributeName":"target_value",
"AttributeType":"float"
}
]
}
create_dataset_response = forecast.create_dataset(Area="CUSTOM",
DatasetType="TARGET_TIME_SERIES",
DatasetName=TS_DATASET_NAME,
DataFrequency=DATASET_FREQUENCY,
Schema=TS_SCHEMA)
ts_dataset_arn = create_dataset_response['DatasetArn']After the dataset was created, it was imported into Amazon Forecast utilizing the Boto3 create_dataset_import_job operate. The create_dataset_import_job operate takes the job title (a string worth), the ARN of the dataset from the earlier step, the placement of the coaching knowledge in Amazon S3 from the earlier step, and the time stamp format as arguments. It returns a JSON object containing the import job ARN. See the next code:
TIMESTAMP_FORMAT = "yyyy-MM-dd"
TS_IMPORT_JOB_NAME = "SALES_DATA_IMPORT_JOB_NAME"
ts_dataset_import_job_response =
forecast.create_dataset_import_job(DatasetImportJobName=TS_IMPORT_JOB_NAME,
DatasetArn=ts_dataset_arn,
DataSource= {
"S3Config" : {
"Path": ts_s3_path,
"RoleArn": role_arn
}
},
TimestampFormat=TIMESTAMP_FORMAT,
TimeZone = TIMEZONE)
ts_dataset_import_job_arn = ts_dataset_import_job_response['DatasetImportJobArn']The imported dataset was then used to create a dataset group utilizing the create_dataset_group operate. This operate takes the area (string values defining the area of the forecast), dataset group title, and the dataset ARN as inputs:
DATASET_GROUP_NAME = "SALES_DATA_GROUP_NAME"
DATASET_ARNS = [ts_dataset_arn]
create_dataset_group_response =
forecast.create_dataset_group(Area="CUSTOM",
DatasetGroupName=DATASET_GROUP_NAME,
DatasetArns=DATASET_ARNS)
dataset_group_arn = create_dataset_group_response['DatasetGroupArn']
Subsequent, we used the dataset group to coach forecasting fashions. Amazon Forecast provides varied state-of-the-art fashions; any of those fashions can be utilized for coaching. We used AutoPredictor as our default mannequin. The principle benefit of utilizing AutoPredictor is that it mechanically generates the item-level forecast, utilizing the optimum mannequin from an ensemble of six state-of-the-art fashions primarily based on the enter dataset. The Boto3 API offers the create_auto_predictor operate for coaching an auto prediction mannequin. The enter parameters of this operate are PredictorName, ForecastHorizon, and ForecastFrequency. Customers are additionally answerable for deciding on the forecast horizon and frequency. The forecast horizon represents the window measurement of the long run prediction, which may be formatted hours, days, weeks, months, and so forth. Equally, forecast frequency represents the granularity of the forecast values, similar to hourly, every day, weekly, month-to-month, or yearly. We primarily centered on predicting month-to-month gross sales of NXP on varied BLs. See the next code:
PREDICTOR_NAME = "SALES_PREDICTOR"
FORECAST_HORIZON = 24
FORECAST_FREQUENCY = "M"
create_auto_predictor_response =
forecast.create_auto_predictor(PredictorName = PREDICTOR_NAME,
ForecastHorizon = FORECAST_HORIZON,
ForecastFrequency = FORECAST_FREQUENCY,
DataConfig = {
'DatasetGroupArn': dataset_group_arn
})
predictor_arn = create_auto_predictor_response['PredictorArn']The educated predictor was then used to generate forecast values. Forecasts had been generated utilizing the create_forecast operate from the beforehand educated predictor. This operate takes the title of the forecast and the ARN of the predictor as inputs and generates the forecast values for the horizon and frequency outlined within the predictor:
FORECAST_NAME = "SALES_FORECAST"
create_forecast_response =
forecast.create_forecast(ForecastName=FORECAST_NAME,
PredictorArn=predictor_arn)Amazon Forecast is a totally managed service that mechanically generates coaching and check datasets and offers varied accuracy metrics to judge the reliability of the model-generated forecast. Nonetheless, to construct consensus on the anticipated knowledge and examine the anticipated values with human predictions, we divided our historic knowledge into coaching knowledge and validation knowledge manually. We educated the mannequin utilizing the coaching knowledge with out exposing the mannequin to validation knowledge and generated the prediction for the size of validation knowledge. The validation knowledge was in contrast with the anticipated values to judge the mannequin efficiency. Validation metrics could embrace imply absolute p.c error (MAPE) and weighted absolute p.c error (WAPE), amongst others. We used WAPE as our accuracy metric, as mentioned within the subsequent part.
Analysis metrics
We first verified the mannequin efficiency utilizing backtesting to validate the prediction of our forecast mannequin for long run gross sales forecast (2026 gross sales). We evaluated the mannequin efficiency utilizing the WAPE. The decrease the WAPE worth, the higher the mannequin. The important thing benefit of utilizing WAPE over different error metrics like MAPE is that WAPE weighs the person affect of every merchandise’s sale. Due to this fact, it accounts for every product’s contribution to the overall sale whereas calculating the general error. For instance, if you happen to make an error of two% on a product that generates $30 million and an error of 10% in a product that generates $50,000, your MAPE won’t inform the whole story. The two% error is definitely costlier than the ten% error, one thing you may’t inform by utilizing MAPE. Comparatively, WAPE will account for these variations. We additionally predicted varied percentile values for the gross sales to show the higher and decrease bounds of the mannequin forecast.
Macro-level gross sales prediction mannequin validation
Subsequent, we validated the mannequin efficiency by way of WAPE values. We calculated the WAPE worth of a mannequin by splitting the information into check and validation units. For instance, within the 2019 WAPE worth, we educated our mannequin utilizing gross sales knowledge between 2011–2018 and predicted gross sales values for the following 12 months (2019 sale). Subsequent, we calculated the WAPE worth utilizing the next formulation:
We repeated the identical process to calculate the WAPE worth for 2020 and 2021. We evaluated the WAPE for all BLs within the auto finish marketplace for 2019, 2020, and 2021. Total, we noticed that Amazon Forecast can obtain a 0.33 WAPE worth even for the 12 months of 2020 (through the COVID-19 pandemic). In 2019 and 2020, our mannequin achieved lower than 0.1 WAPE values, demonstrating excessive accuracy.
Macro-level gross sales prediction baseline comparability
We in contrast the efficiency of the macro gross sales prediction fashions developed utilizing Amazon Forecast to 3 baseline fashions by way of WAPE worth for 2019, 2020 and 2021 (see the next determine). Amazon Forecast both considerably outperformed the opposite baseline fashions or carried out on par for all 3 years. These outcomes additional validate the effectiveness of our last mannequin predictions.
Macro-level gross sales prediction mannequin vs. human predictions
To additional validate the boldness of our macro-level mannequin, we subsequent in contrast the efficiency of our mannequin with the human-predicted gross sales values. Firstly of the fourth quarter yearly, market specialists at NXP predict the gross sales worth of every BL, bearing in mind international market traits in addition to different international indicators that might doubtlessly affect the gross sales of NXP merchandise. We examine the p.c error of the mannequin prediction vs. human prediction to the precise gross sales values in 2019, 2020, and 2021. We educated three fashions utilizing knowledge from 2011–2018 and predicted the gross sales values till 2021. We subsequent calculated the MAPE for the precise gross sales values. We then used the human-predicted values by the tip of 2018 (check the mannequin forecast 1Y forward to 3Y forward forecast). We repeated this course of to foretell the values in 2019 (1Y forward forecast to 2Y forward forecast) and 2020 (for 1Y forward forecast). Total, the mannequin carried out on par with the human predictors or higher in some circumstances. These outcomes show the effectiveness and reliability of our mannequin.
Micro-level gross sales prediction and product lifecycle
The next determine depicts how the mannequin behaves utilizing product knowledge whereas accessing only a few observations for every product (particularly one or two observations on the enter for product lifecycle prediction). The orange dots characterize the coaching knowledge, the inexperienced dots characterize the testing knowledge, and the blue line represents the mannequin predicted product lifecycle.
The mannequin may be fed extra observations for context with out the necessity for re-training as new gross sales knowledge turn out to be accessible. The next determine demonstrates how the mannequin behaves whether it is given extra context. In the end, extra context results in decrease WAPE values.
As well as, we managed to include extra options for every product, together with fabrication strategies and different categorical data. On this regard, exterior options helped scale back the WAPE worth within the low-context regime (see the next determine). There are two explanations for this conduct. First, we have to let the information converse for itself within the high-context regimes. The extra options can intervene with this course of. Second, we’d like higher options. We used 1,000 dimensional one-hot-encoded options (bag of phrases). The conjecture is that higher characteristic engineering strategies may also help scale back WAPE even additional.
Such extra knowledge may also help the mannequin predict gross sales of latest merchandise even earlier than the product is launched in the marketplace. For instance, within the following determine, we plot how a lot mileage we will get solely out of exterior options.
Conclusion
On this submit, we demonstrated how the MLSL and NXP groups labored collectively to foretell macro- and micro-level long-term gross sales for NXP. The NXP group will now learn to use these gross sales predictions of their processes—for instance, to make use of it as enter for R&D funding choices and improve ROI. We used Amazon Forecast to foretell the gross sales for enterprise strains (macro gross sales), which we known as the top-down method. We additionally proposed a novel method utilizing time sequence as some extent cloud to sort out the challenges of lacking values and chilly begin on the product degree (micro degree). We referred to this method as bottom-up, the place we predicted the month-to-month gross sales of every product. We additional integrated exterior options of every product to boost the efficiency of the mannequin for chilly begin.
Total, the fashions developed throughout this engagement carried out on par in comparison with human prediction. In some circumstances, the fashions carried out higher than human predictions in the long run. These outcomes show the effectiveness and reliability of our fashions.
This answer may be employed for any forecasting drawback. For additional help by way of designing and growing ML options, please free to get in contact with the MLSL group.
In regards to the authors
Souad Boutane is an information scientist at NXP-CTO, the place she is remodeling varied knowledge into significant insights to help enterprise determination utilizing superior instruments and strategies.
Ben Fridolin is an information scientist at NXP-CTO, the place he coordinates on accelerating AI and cloud adoption. He focuses on machine studying, deep studying and end-to-end ML options.
Cornee Geenen is a venture lead within the Knowledge Portfolio of NXP supporting the group in it’s digital transformation in the direction of turning into knowledge centric.
Bart Zeeman is a strategist with a ardour for knowledge & analytics at NXP-CTO the place he’s driving for higher knowledge pushed choices for extra progress and innovation.
Ahsan Ali is an Utilized Scientist on the Amazon Machine Studying Options Lab, the place he works with prospects from totally different domains to unravel their pressing and costly issues utilizing state-of-the-art AI/ML strategies.
Yifu Hu is an Utilized Scientist within the Amazon Machine Studying Options lab, the place he helps design inventive ML options to handle prospects’ enterprise issues in varied industries.
Mehdi Noori is an Utilized Science Supervisor at Amazon ML Options Lab, the place he helps develop ML options for giant organizations throughout varied industries and leads the Power vertical. He’s keen about utilizing AI/ML to assist prospects obtain their Sustainability objectives.
Huzefa Rangwala is a Senior Utilized Science Supervisor at AIRE, AWS. He leads a group of scientists and engineers to allow machine studying primarily based discovery of information belongings. His analysis pursuits are in accountable AI, federated studying and functions of ML in well being care and life sciences.
