--- /dev/null
+\documentclass[12pt, a4paper]{article}
+
+\usepackage[margin=2.5cm]{geometry}
+\usepackage[utf8]{inputenc} % in encoding
+\usepackage[T1]{fontenc} % out-encoding f
+\usepackage{eurosym}
+\usepackage{lmodern, microtype} % goes OK with T1 fontenc
+%\usepackage[authoryear, round]{natbib}
+\usepackage{natbib}
+\usepackage{color, tikz, graphicx, subfig}
+\usepackage{amssymb, amsmath, amsthm}
+\usepackage{setspace, lineno, url, xcolor}
+\usepackage{savetrees}
+
+\newcommand{\todo}[1]{\textcolor{blue}{TODO: #1}} % macro for todo entries
+
+% Style options
+\renewcommand\familydefault{\sfdefault} % Use with sans serif font
+\setlength{\bibsep}{0.0pt} % Compact bibliography (natbib)
+
+\title{Disaggregated Electricity Forecasting using Clustering of Individual Consumers \\
+ {\normalsize \color{gray} IRSDI - RESEARCH INITIATIVE IN INDUSTRIAL DATA SCIENCE}}
+
+\author{Benjamin Auder \and
+ Jairo Cugliari \and
+ Yannig Goude \and
+ Jean-Michel Poggi
+}
+\date{\normalsize\today
+\vspace{-1.2\baselineskip}}
+
+
+
+\begin{document}
+\maketitle
+
+%\begin{abstract}
+
+%\end{abstract}
+
+% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% S E C T I O N
+%
+\section{Context}
+
+\subsection{Industrial}
+
+Electricity load forecasting is crucial for utilities for production
+planning as well as marketing offers. Recently, the increasing deployment of
+smart grids infrastructure requires the development of more flexible data
+driven forecasting methods adapting quite automatically to new data sets.
+Electricity load forecasting is crucial for utilities for production planning as
+well as marketing offers. New metering infrastructures as smart meters
+provide new and potentially massive informations about individual (household,
+small and medium enterprise) consumption. As an example, in France,
+ERDF (Electricite Reseau Distribution de France the French manager of
+the public electricity distribution network) deployed 250000 smart meters,
+covering a rural and an urban territory and providing half-hourly household
+energy used each day. ERDF plans to install 35 millions of them over the
+French territory by the end of 2020 and exploiting such an amount of data
+is an exciting but challenging task (see \url{http://www.erdf.fr/Linky}).
+We propose to build clustering tools useful for forecasting the load
+consumption. The idea is to disaggregate the global signal in such a way that
+the sum of disaggregated forecasts significantly improves the prediction of the
+whole global signal. The strategy is in three steps: first we cluster curves
+defining super-consumers, then we build a hierarchy of partitions within which
+the best one is finally selected with respect to a disaggregated forecast
+criterion. The proposed strategy is applied to a dataset of individual
+consumers from the French electricity provider EDF. A substantial gain
+of $16$ \% in forecast accuracy comparing to the 1-cluster approach is provided
+by disaggregation while preserving meaningful classes of consumers.
+
+\subsection{Academic}
+
+In the context of economic seasonal univariate continuous time series, it is often
+natural to segment it in time, into consecutive curves, for example days, which
+are then treated as a discrete time series of functions. In particular, in the
+electrical context, the shape of the curves exhibits rich information about the
+calendar day type, the meteorological conditions or the existence of special
+electricity tariffs. Using the information contained in the shape of the load
+curves leads to very elegant formulation of functional forecasting.
+
+
+%Electricity load experts naturally look at daily demand data as time functions
+%called load curves. In a recent paper, \cite{shang2013} uses a functional time
+%series approach for forecasting short-term electricity demand. This paper is
+%illustrated by the half-hourly electricity demand from Monday to Sunday in South
+%Australia. The strategy is also to consider a seasonal univariate time series as
+%a time series of curves, then to reduce the dimensionality of curves by applying
+%a functional principal component analysis and finally, following
+%\cite{shang2011}, the principal component scores are forecasted using a
+%univariate ARIMA models. In addition, since data points in the daily electricity
+%demand are sequentially observed, a forecast updating method based on
+%nonparametric bootstrap approach is proposed to improve the accuracy of point
+%forecasts. With respect to this strategy, the scheme we propose handles the
+%forecasting problem in a functional way avoiding the hour by hour processing and
+%considers a more flexible way to construct the distribution leading to the
+%prediction interval.
+
+The shape of the curves exhibits rich information about the calendar day type,
+the meteorological conditions or the existence of special electricity tariffs.
+Using the information contained in the shape of the load curves, \cite{antoniadis2012prevision} proposed a flexible nonparametric function-valued
+forecast model called KWF (\textit{Kernel + Wavelet + Functional}) well suited
+to handle nonstationary series. The predictor can be seen as a weighted average
+of futures of past situations, where the weights increase with the similarity
+between the past situations and the actual one. In addition, this strategy
+provides with a simultaneous multiple horizon prediction for a global forecast.
+
+However, there is a need for local electricity load forecasting at different levels of the grid.
+Bottom-up approaches, based on a two stage process combining clustering and forecasting
+methods, are a promising perspective. First, it
+consists in building classes in a population such that each class could be
+sufficiently well forecast but corresponds to different load shapes or reacts
+differently to exogenous variables like temperature or prices (see e.g.
+\cite{labeeuw} in the context of demand response). The second stage consists in
+aggregating forecasts to forecast the total or any subtotal of the population
+consumption. For example, identify and forecast the consumption of a
+sub-population reactive to an incentive is an important need to optimize a
+demand response program.
+
+\section{Past work}
+
+Few papers consider the problem of clustering individual consumption for
+forecasting (e.g. \cite{iwafune2014short, Alzate, carevic2010applications, MisitiElec}). Recently, \cite{energycon} proposed to build clustering tools useful for the two tasks simultaneously: clustering individual customers and forecasting the load consumption. The idea is to disaggregate the global signal in such a way that the sum of disaggregated forecasts significantly improves the prediction of the whole global signal. The general strategy is in three steps: first we cluster individual curves defining super-consumers, then we built a hierarchy of partitions within which a best one is finally selected with respect to a disaggregated forecast criterion. The predictions are made with the KWF model which allows one to use it as a off-the-shelve tool.
+
+While this work has ended with an the specification of an algorithm, a current need is a real upscaling proof. A first step on this direction was done in
+\cite{auder2014}.
+
+
+% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% S E C T I O N
+%
+\section{Aims}
+
+The method proposed in \cite{energycon} has been successfully tested on a small data set of EDF clients. With the current development of smart meters in France the available volume of individual data is increasing day after day. Then, there is a genuine need of measuring the upscale skills of the existent methods.
+
+This projet's aim is twofold. First, we will evaluate the upscaling capacity of the strategy developed in \cite{energycon} to cope with the upgrowing volume of data. Second, we will study how to adapt the KWF prediction method to take into account an exogeneous variable. In our particular problem the exogeneous variables can be any meteorological measurement that affects the load demand and is available at the moment of the prediction.
+
+
+% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% S E C T I O N
+%
+
+\section{Means considered}
+
+\subsection{Methods}
+\paragraph{Clustering analysis.} In general, clustering methods look for groups of individuals on data in such a way that those belonging to the same group are more similar than those from other groups. Many methods exists to cluster data:
+hierarchical, center-based, probabilistic, etc. Almost all of them depends heavily
+on the choice of a similarity measure between individuals. For this challenge we plan
+to compare individuals in terms of their wavelet spectrum signature. Thanks to this strategy, non
+stationary signals may be fairly compared. Moreover, the signals need not to be
+measured on the same temporal grid. However, in order to detect relevant results
+the wavelet signatures should be corrected by exogenous information (e.g. the one
+provided as client characteristics).
+
+\paragraph{Wavelet analysis.} Since the objects to analyze (load curves) can be viewed
+as functions of time, functional data analysis techniques are one possible choice to
+represent these objects. From a stochastic point of view the functions are realizations
+of a non stationary random process. Wavelet transform can be used to extract
+relevant information about the functions both on time and frequency. With an
+appropriate representation of the objects, it is then possible to construct
+a meaningful distance between load curves.
+
+\paragraph{Forecasting with KWF}
+The basic idea of nonparametric forecasting is that similar cases in the past
+have similar future consequences. For example the electricity consumption is
+divided into blocks of one day size. Then, using a dissimilarity measure, the
+blocks similar to the last observed block are searched in the past and a vector
+of weights is built. Finally, the forecast of the next day is obtained by a
+weighted average of the most similar future days using previous vector of
+weights. From the statistical point of view, the model is an estimate of the
+regression function using the kernel method, of the last block against all the
+blocks in the past. In \cite{antoniadis2006functional} this basic model is
+extended to the case of stationary functional random variables. But in the
+context of electrical power demand, the hypothesis of stationarity generally
+fails: an evolving mean level and the existence of groups that may be seen as
+classes of stationarity are to be considered. Corrections to take into
+account these two main nonstationary features are considered in
+\cite{antoniadis2012prevision} defining a flexible nonparametric function-valued
+forecast model called KWF (\textit{Kernel + Wavelet + Functional}) well suited
+to handle nonstationary series. The predictor can be seen as a weighted average
+of futures of past situations, where the weights increase with the similarity
+between the past situations and the actual one. Again the similarity is defined
+thanks to the wavelet decompositions of the two segments.
+
+
+\subsection{Technology} % to be employed (hardware y software)}
+
+
+The volume of data to deal for this projet can be handled with standard
+but recent tools for data analysis.
+The specific software tools will be statistical programming language like \texttt{R} with some popular
+libraries (\texttt{data.table}, \texttt{dplyr}) and specific packages to cope with wavelet analysis. All these elements are open source.
+
+When the computational burden will grow, we have direct access to larger computation capacities.
+
+All the tools developed on the project will be made available as open source software licences.
+
+\subsection{Research team}
+
+The proposed team for developing this projet is composed by theree
+academic members :
+\begin{itemize}
+\item Benjamin Auder, LMO, Univ Paris Saclay
+\item Jairo Cugliari, ERIC, Univ Lyon
+\item Jean-Michel Poggi, LMO, Univ Paris Saclay, Univ Paris Descartes
+\end{itemize}
+
+% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% S E C T I O N
+%
+\section{Data description}
+\begin{itemize}
+\item a first dataset already used in \cite{energycon} could be used, at least in a first step, to calibrate the method.
+\item simulated data could be obtained at EDF following \cite{bondu15} or any simulation method preserving confidentiality
+of individual consumers. Obviously, any amount of such data could be produced to benchmark the scalability of our approach.
+\item Irish data provided by the Irish commission for energy regulation consisting in 2000 individual consumption (small and
+medium enterprise and residential) at an half-hourly resolution as well as pre and post experiment survey (see \cite{Cer_a, Cer_b}).
+\end{itemize}
+
+
+
+% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% S E C T I O N
+%
+\section{Budget}
+The expected global budget for the projet is of 15000 \euro, which comprises a 1 day workshop.
+
+\paragraph{Internal budget} The members of the research team are based on the Paris area and Lyon.
+The way we work includes video and audio conferences in a regular basis as well as several in-person meetings.
+
+We plan to present the work on international conferences both on data science and energy oriented meetings.
+
+Last, a stress test for the upscale skill of the proposed method will need to hire computing time on a specialized platform. We have access to
+the Centre de Calcul de l'Institut National de Physique Nucléaire et de Physique des Particules (\url{http://cc.in2p3.fr/}) through the laboratory ERIC, Lyon 2.
+
+\paragraph{Worshop organization on Individual Electricity Consumers}
+A 1-day workshop dedicated to Individual Electricity Consumers including
+sessions on data, packages and methods, could be organized in September
+2017, and could be proposed to The French Statistical Society (SFdS) as a
+satellite meeting of the Journées de Statistique 2018 which will be held in
+the campus of EDF Lab in May 2018.
+
+
+\begin{center}
+\begin{tabular}{lr} \hline
+\textbf{Internal budget} & \textbf{10 000 \euro}\\
+\; Travels & 3 000 \euro\\
+\; Conference fees & 3 000 \euro\\
+\; Internal meetings & 2 000 \euro\\
+\; Hiring of high performance computing time & 2 000 \euro\\
+\textbf{Worshop organization} & \textbf{5 000 \euro} \\
+\; Invitations of researchers & 3 000 \euro\\
+\; Organization workshop & 2 000 \euro\\ \hline
+\textbf{Global budget} & \textbf{15 000 \euro} \\ \hline
+\end{tabular}
+\end{center}
+
+
+% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% S E C T I O N
+%
+\section{Vitas}
+
+\paragraph{Benjamin Auder} is CNRS Research Engineer at LMO, University Paris-Sud Orsay in France.
+He obtained his PhD in statistics in 2011 at the university Université Pierre et Marie Curie, Paris.
+His main research areas are Clustering, dimensionality reduction, manifold learning, machine learning
+in addition to software development and implementation issues of algorithmic solutions.
+
+(\url{http://auder.net/page-upsud/})
+
+\paragraph{Jairo Cugliari} is Assistant Professor of Statistics at University of Lyon in France. He obtained his PhD in statistics
+in 2011 at the university Paris-Sud 11 Orsay. His main research areas are functional data analysis methods
+for classification and prediction for applied statistical problems.
+
+(\url{http://eric.univ-lyon2.fr/~jcugliari/})
+
+
+
+\paragraph{Jean-Michel Poggi} is Professor of Statistics at University of Paris Descartes
+and at University Paris-Sud Orsay in France. His main research areas are
+tree-based methods for classification and regression, nonparametric time
+series forecasting, wavelet methods and applied statistical modeling in energy
+and environment fields. His publications combine theoretical and practical
+contributions together with industrial applications and software development.
+
+\noindent
+He is an elected member of the ISI, he was President of the French Statistical
+Society (SFdS) and he is Vice-President of the FENStatS, Vice-President of ENBIS and President of ECAS.
+
+(\url{http://www.math.u-psud.fr/~poggi/})
+
+% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+%
+% S E C T I O N
+%
+\section{Associated industrial company} % And members
+
+
+\paragraph{Yannig Goude} is a research-engineer/project manager at EDF R\&D and associate
+professor at University Paris-Sud Orsay, France. He obtained his PhD in statistics and probability
+in 2008 at the university Paris-Sud 11 Orsay. His research interests are electricity load forecasting,
+more generally time series analysis and forecasting, non-parametric models and expert aggregation.
+
+(\url{https://fr.linkedin.com/in/yannig-goude-768b3980})
+
+\bibliographystyle{plain}
+\bibliography{biblio_irsdi} %,predintervals,rapportfinal}
+
+\end{document}
+
+
+
+\bibitem{Alzate}
+C.~Alzate and M.~Sinn,
+ Improved electricity load forecasting via kernel spectral clustering of
+ smartmeter,
+ \emph{International Conference on Data Mining}, vol. 948, pp. 943 -- 948,
+ 2013
+
+\bibitem{antoniadis2006functional}
+A.~Antoniadis, E.~Paparoditis and T.~Sapatinas,
+ A functional wavelet-kernel approach for time series prediction,
+ \emph{Journal of the Royal Statistical Society, Series B},
+ vol. 68(5), pp. 837 -- 857, 2006
+
+\bibitem{antoniadis2013clustering}
+A.~Antoniadis, X.~Brossat, J.~Cugliari, and J.-M.~Poggi,
+ Clustering functional data using wavelets,
+ \emph{International Journal of Wavelets, Multiresolution and Information
+ Processing},
+ vol. 11(1), 2013
+
+\bibitem{antoniadis2012prevision}
+A. Antoniadis, X. Brossat, J. Cugliari, J.-M. Poggi,
+ Pr\'{e}vision d'un processus \`{a} valeurs fonctionnelles en pr\'{e}sence de
+ non stationnarit\'{e}s. Application \`{a} la consommation
+ d'\'{e}lectricit\'{e}
+ Journal de la Soci\'{e}t\'{e} Fran\c{c}aise de Statistique,
+ Vol. 153, No. 2, 52--78, 2012
+
+\bibitem{brabec2015statistical}
+Brabec, M. and Kon{\'a}r, O. and Mal{\`y}, M. and Kasanick{\`y}, I and Pelik{\'a}n, E.,
+ Statistical models for disaggregation and reaggregation of natural gas
+ consumption data,
+ \emph{Journal of Applied Statistics}, vol. 42(5), pp. 921--937, 2015
+
+\bibitem{carevic2010applications}
+Carevi{\'c}, S. and Capuder, T. and Delimar, M.
+ Applications of clustering algorithms in long-term load forecasting
+ \emph{Proceedings Energy Conference and Exhibition (EnergyCon),
+ 2010 IEEE International} 688--693, 2010
+
+\bibitem{Chicco}
+G. Chicco
+ Overview and performance assessment of the clustering methods for electrical
+ load pattern grouping, Energy , 42, 68 -- 80, 2012.
+
+\bibitem{Figueiredo}
+Figueiredo, V., Rodrigues, F., Vale, Z., Gouveia, J. B.
+ An electric energy consumer characterization framework based on data mining
+ techniques.
+ Power Systems, IEEE Transactions on, 20(2), 596--602, 2005
+
+\bibitem{iwafune2014short}
+Iwafune, Y., Yagita, Y., Ikegami, T., Ogimoto K.
+ Short-term forecasting of residential building load for distributed energy
+ management
+ \emph{Proceedings Energy Conference (ENERGYCON), 2014 IEEE International}
+ 1197--1204, 2014
+
+\bibitem{kaufmanpj}
+Kaufman, L. and Rousseeuw, P
+ Finding groups in data: An introduction to cluster analysis,
+ Hoboken NJ John Wiley \& Sons Inc, 1990
+
+\bibitem{Kwac}
+J. Kwac, Flora, J., Rajagopal, R.
+ Household Energy Consumption Segmentation Using Hourly Data
+ Smart Grid, IEEE Transactions on, 5, 420--430, 2014
+
+\bibitem{labeeuw}
+Labeeuw, W., Stragier, J., and Deconinck, G.
+ Potential of active demand reduction with residential wet appliances:
+ A case study for Belgium.
+ Smart Grid, IEEE Transactions on, 6(1), 315--323, 2015
+
+\bibitem{Liao}
+Warren Liao, T.
+ Clustering of time series data--a survey
+ Pattern recognition, 38(11), 1857--1874, 2005
+
+\bibitem{MisitiElec}
+M.~Misiti, Y.~Misiti, G.~Oppenheim, and J.-M.~Poggi,
+ Optimized Clusters for Disaggregated Electricity Load Forecasting,
+ \emph{REVSTAT -- Statistical Journal}, vol. 8(2), pp. 105 -- 124, 2010
+
+\bibitem{Mutanen}
+ Mutanen, A., Ruska, M., Repo, S., Jarventausta, P.
+ Customer classification and load profiling method for distribution systems.
+ Power Delivery, IEEE Transactions on, 26(3), 1755--1763, 2011
+
+%\bibitem{Piao}
+%Piao, M., Lee, H. G., Park, J. H., Ryu, K. H.
+% Application of Classification Methods for Forecasting Mid-Term
+% Power Load Patterns.
+% In Advanced Intelligent Computing Theories and Applications. Springer, 2008
+
+\bibitem{Rasanen}
+T., R\"{a}s\"{a}nen, D., Voukantsis, H., Niska, K., Karatzas, M., Kolehmainen
+ Data-based method for creating electricity use load profiles using large
+ amount of customer-specific hourly measured electricity use data
+ Applied Energy, 87(11), 3538--3545, 2010
+
+\bibitem{Rhodes}
+J.D. Rhodes, W.J. Cole, C.R. Upshaw, T.F. Edgar, M.E. Webber
+ Clustering analysis of residential electricity demand profiles
+ Preprint submitted to Applied Energy, March 18, 2014
+
+\bibitem{steinley2008new}
+D. Steinley and M. Brusco,
+A new variable weighting and selection procedure for k-means cluster analysis.
+\emph{Multivariate Behavioral Research}, 43:32, 2008.
+
+\bibitem{wijaya2015forecasting}
+Wijaya, T. K., Sinn, M., and Chen, B.,
+ Forecasting Uncertainty in Electricity Demand,
+ \emph{AAAI-15 Workshop on Computational Sustainability, EPFL-CONF-203769},
+ 2015
+
+\bibitem{Zhou}
+K. Zhou, S. Yang, C. Shen
+ A review of electric load classification in smart grid environment,
+ Renewable and Sustainable Energy Reviews, 24, 103 -- 110, 2013.
+
projects / epclust.git / blobdiff