Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction

Aditya Sundararajan; Arif I. Sarwat

doi:10.1007/978-3-030-32520-6_43

Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction

Aditya Sundararajan
, Arif I. Sarwat

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

16 Scopus citations

Abstract

To effectively predict generation of distributed photovoltaic (PV) systems, three parameters are critical: irradiance, ambient temperature, and module temperature. However, their completeness cannot be guaranteed because of issues in data acquisition. Many methods in literature address missingness, but their applicability varies with missingness mechanism. Exploration of methods to impute missing data in PV systems is lacking. This paper conducts statistical analyses to understand missingness mechanism in data of a real grid-tied 1.4MW PV system at Miami, and compares the imputation performance of different methods: random imputation, multiple imputation using expectation-maximization, kNN, and random forests, using error metrics and size effect measures. Imputed values are used in a multilayer perceptron to predict and compare PV generation with observed values. Results show that values imputed using kNN and random forests have the least differences in proportions and help utilities make more accurate prediction of generation for distribution planning.

Original language	English
Title of host publication	Proceedings of the Future Technologies Conference, FTC 2019 Volume 1
Editors	Kohei Arai, Rahul Bhatia, Supriya Kapoor
Publisher	Springer
Pages	590-609
Number of pages	20
ISBN (Print)	9783030325190
DOIs	https://doi.org/10.1007/978-3-030-32520-6_43
State	Published - 2020
Externally published	Yes
Event	4th Future Technologies Conference, FTC 2019 - San Francisco, United States Duration: Oct 24 2019 → Oct 25 2019

Publication series

Name	Advances in Intelligent Systems and Computing
Volume	1069
ISSN (Print)	2194-5357
ISSN (Electronic)	2194-5365

Conference

Conference	4th Future Technologies Conference, FTC 2019
Country/Territory	United States
City	San Francisco
Period	10/24/19 → 10/25/19

Funding

Acknowledgments. The work published is a result of the research sponsored by the National Science Foundation (NSF) CNS division under the award 1553494.

Keywords

Data processing
Distributed PV
Imputation methods
Missing data
PV Generation Prediction

Access to Document

10.1007/978-3-030-32520-6_43

Cite this

Sundararajan, A., & Sarwat, A. I. (2020). Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction. In K. Arai, R. Bhatia, & S. Kapoor (Eds.), Proceedings of the Future Technologies Conference, FTC 2019 Volume 1 (pp. 590-609). (Advances in Intelligent Systems and Computing; Vol. 1069). Springer. https://doi.org/10.1007/978-3-030-32520-6_43

@inproceedings{9ca355bd0e114a40b5b328d31e84c384,

title = "Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction",

abstract = "To effectively predict generation of distributed photovoltaic (PV) systems, three parameters are critical: irradiance, ambient temperature, and module temperature. However, their completeness cannot be guaranteed because of issues in data acquisition. Many methods in literature address missingness, but their applicability varies with missingness mechanism. Exploration of methods to impute missing data in PV systems is lacking. This paper conducts statistical analyses to understand missingness mechanism in data of a real grid-tied 1.4MW PV system at Miami, and compares the imputation performance of different methods: random imputation, multiple imputation using expectation-maximization, kNN, and random forests, using error metrics and size effect measures. Imputed values are used in a multilayer perceptron to predict and compare PV generation with observed values. Results show that values imputed using kNN and random forests have the least differences in proportions and help utilities make more accurate prediction of generation for distribution planning.",

keywords = "Data processing, Distributed PV, Imputation methods, Missing data, PV Generation Prediction",

author = "Aditya Sundararajan and Sarwat, \{Arif I.\}",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 4th Future Technologies Conference, FTC 2019 ; Conference date: 24-10-2019 Through 25-10-2019",

year = "2020",

doi = "10.1007/978-3-030-32520-6\_43",

language = "English",

isbn = "9783030325190",

series = "Advances in Intelligent Systems and Computing",

publisher = "Springer",

pages = "590--609",

editor = "Kohei Arai and Rahul Bhatia and Supriya Kapoor",

booktitle = "Proceedings of the Future Technologies Conference, FTC 2019 Volume 1",

}

Sundararajan, A & Sarwat, AI 2020, Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction. in K Arai, R Bhatia & S Kapoor (eds), Proceedings of the Future Technologies Conference, FTC 2019 Volume 1. Advances in Intelligent Systems and Computing, vol. 1069, Springer, pp. 590-609, 4th Future Technologies Conference, FTC 2019, San Francisco, United States, 10/24/19. https://doi.org/10.1007/978-3-030-32520-6_43

Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction. / Sundararajan, Aditya; Sarwat, Arif I.
Proceedings of the Future Technologies Conference, FTC 2019 Volume 1. ed. / Kohei Arai; Rahul Bhatia; Supriya Kapoor. Springer, 2020. p. 590-609 (Advances in Intelligent Systems and Computing; Vol. 1069).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction

AU - Sundararajan, Aditya

AU - Sarwat, Arif I.

PY - 2020

Y1 - 2020

N2 - To effectively predict generation of distributed photovoltaic (PV) systems, three parameters are critical: irradiance, ambient temperature, and module temperature. However, their completeness cannot be guaranteed because of issues in data acquisition. Many methods in literature address missingness, but their applicability varies with missingness mechanism. Exploration of methods to impute missing data in PV systems is lacking. This paper conducts statistical analyses to understand missingness mechanism in data of a real grid-tied 1.4MW PV system at Miami, and compares the imputation performance of different methods: random imputation, multiple imputation using expectation-maximization, kNN, and random forests, using error metrics and size effect measures. Imputed values are used in a multilayer perceptron to predict and compare PV generation with observed values. Results show that values imputed using kNN and random forests have the least differences in proportions and help utilities make more accurate prediction of generation for distribution planning.

AB - To effectively predict generation of distributed photovoltaic (PV) systems, three parameters are critical: irradiance, ambient temperature, and module temperature. However, their completeness cannot be guaranteed because of issues in data acquisition. Many methods in literature address missingness, but their applicability varies with missingness mechanism. Exploration of methods to impute missing data in PV systems is lacking. This paper conducts statistical analyses to understand missingness mechanism in data of a real grid-tied 1.4MW PV system at Miami, and compares the imputation performance of different methods: random imputation, multiple imputation using expectation-maximization, kNN, and random forests, using error metrics and size effect measures. Imputed values are used in a multilayer perceptron to predict and compare PV generation with observed values. Results show that values imputed using kNN and random forests have the least differences in proportions and help utilities make more accurate prediction of generation for distribution planning.

KW - Data processing

KW - Distributed PV

KW - Imputation methods

KW - Missing data

KW - PV Generation Prediction

UR - https://www.scopus.com/pages/publications/85075656932

U2 - 10.1007/978-3-030-32520-6_43

DO - 10.1007/978-3-030-32520-6_43

M3 - Conference contribution

AN - SCOPUS:85075656932

SN - 9783030325190

T3 - Advances in Intelligent Systems and Computing

SP - 590

EP - 609

BT - Proceedings of the Future Technologies Conference, FTC 2019 Volume 1

A2 - Arai, Kohei

A2 - Bhatia, Rahul

A2 - Kapoor, Supriya

PB - Springer

T2 - 4th Future Technologies Conference, FTC 2019

Y2 - 24 October 2019 through 25 October 2019

ER -

Evaluation of Missing Data Imputation Methods for an Enhanced Distributed PV Generation Prediction

Abstract

Publication series

Conference

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this