EKZ: Synthetic Load Profile Generation

Reliable electricity load monitoring for non-metered nodes

Started

September 2, 2024

Status

Completed

Share this project

Reliable electricity monitoring in the Canton of Zurich

EKZ (Elektrizitätswerke des Kantons Zürich), is one of Switzerland's largest energy suppliers, based in Zurich and serves about one million people with electricity. In this role, EKZ manages an extensive electricity distribution network, structured hierarchically across three levels - transformers, buildings, and facilities. While smart-meter rollout is ongoing, many nodes still lack 15-minute resolution data. To maintain full situational awareness, SDSC and EKZ developed a pipeline to synthesize realistic load profiles for non-metered customers based on available consumption data and network metadata.

People

Collaborators

SDSC Team:

Arshjot Khehra

Senior Data Scientist

Before joining SDSC, Arshjot Khehra received his MSc in Artificial Intelligence from USI Lugano, where he completed his thesis on hierarchical graph reinforcement learning. Previously, he worked for 4+ years across India and Singapore gaining data science experience in insurance, logistics, and manufacturing sectors. He also holds a BSc in Industrial Engineering from PEC Chandigarh. Over the course of his career, Arshjot worked on a wide array of projects, such as, handwritten text recognition and generation, voice matching across phone call recordings, policy lapse rate prediction for customer retention, and automated insurance claim processing.

Arshjot Khehra

Christian Schneebeli

Senior Data Scientist

Christian joined the SDSC in July 2021 as a data scientist in the industry cell. Before that he worked at the Media Technology Center at ETH Zurich to develop new ML technology for their industry partners. He completed his Master's degree in Mathematics at ETH Zurich (2017) with focus on algorithmics and machine learning. After his studies he worked as a online marketing data analyst in the news publishing business. His expertise lies in statistics, NLP and software engineering.

Christian Schneebeli

Saurabh Bhargava

Principal Data Scientist

Saurabh Bhargava, joined the SDSC as a Principal Data Scientist in the Industry Cell at the Zürich office in 2022. Saurabh previously worked in the retail sector and the advertising industry in Germany. He lead and built various data products for customers using state of the art machine learning methods and industrializing them thereby adding value for the customers. He completed his PhD from ETH Zürich in June 2017 specializing in machine learning applications on Audio data. He obtained his Master’s and Bachelor’s degrees from EPFL and Indian Institute of Technology (IIT), Roorkee, India in 2011 and 2009 respectively. His interests and expertise are in combining state of the art data science and data engineering tools for building scalable data products.

Saurabh Bhargava

PI | Partners:

Elektrizitätswerke des Kantons Zürich (EKZ)

More information

description

Objectives

The main goal of this collaboration was to create a scalable machine learning (ML) pipeline to generate high quality synthetic load profiles for non-metered customers, enabling EKZ to monitor its network comprehensively.

Specific objectives included:
• Adapt the existing pipeline to use weekly and daily consumption data instead of yearly data.
• Reduce overall computation time by refactoring and parallelizing the synthesis process.
• Implement validation metrics and confidence intervals to ensure reliability and interpretability.
• Integrate the solution within EKZ’s cloud environment for automated weekly runs and monitoring.

‍

Figure 1: EKZ network structure illustrating metered and non-metered nodes.

Approach | Solution

SDSC redesigned the proof-of-concept into a fast and reliable system for synthetic load profile generation. The solution integrates data preprocessing, clustering, probabilistic modeling, and validation components into a unified pipeline, deployed on Azure ML.

The primary advancements introduced by this approach were:

• Modular preprocessing with dynamic outlier removal and weekly normalization.
• Clustering using DTW-UMAP [1] and HDBSCAN [2] for homogeneous grouping.
• Probabilistic modeling using Gaussian, Log-Normal, and Conditional Gaussian Mixture Models.
• Evaluation metrics comparing real and synthetic profiles.
• Confidence intervals at both profile and transformer levels through Monte Carlo aggregation.

A graph with numbers and linesAI-generated content may be incorrect.

Figure 2: Comparison of closest profiles by absolute distance (top) and DTW (bottom).

Impact

The optimized ML pipeline delivered significant improvements across multiple dimensions:
• 10× faster synthesis and 5× overall pipeline speed-up.
• Synthetic profiles are statistically indistinguishable from real ones.
• Automated weekly execution with robust logging and monitoring.
• Confidence intervals indicating load peaks according to different uncertainty levels.
• Scalable integration with Azure ML for continuous distribution network monitoring.

Figure 3: Example of confidence interval visualization at profile level.

‍

Footnotes:

[1] DTW-UMAP: Dynamic Time Warping method combined with Uniform Manifold Approximation and Projection - suitable to visualize and cluster time series or sequential behavior in a meaningful way.

[2] Hierarchical Density-Based Spatial Clustering of Applications with Noise - a clustering algorithm that automatically groups similar data points and identifies outliers without needing predefined cluster numbers - suitable for exploratory analysis.