General overview
As soon as the dataset is successfully "time-reference-checked", the next step is to check the quality of the data. The Analyst specialized module (Automatic quality check of irradiation) brings a set of quality tests used today in Solargis. These include detection of invalid values, nighttime/daytime, artificial static values, breaking physical limits, consistency of irradiance components (only for GHI, DNI and DIF). Each record that failed in these tests is flagged by a pre-defined flag value. The user has some control over this process (i.e., disable/enable tests, changing test properties). After quality tests are performed the user is provided with a preview of a summary quality report (plots and text information). At the end of this process, generated flags are saved with the data and the quality status.
There are four main tabs in this window:
Heatmap flag plot. The heatmap flag plot depicts the occurrence of quality flags from automatic QC in time.
Time series flag plot. The Time series flag plot allows the basic data visualization of the quality control flags in time.
Consistency plots. Consistency plots depict the consistency between measured and calculated values separately for GHI, DNI and DIF.
Quality control summary. The Quality control summary resumes the overview of the statistics after the quality control.
How test groups work:
Multi component tests compare related parameters against each other within a test group.
Single component test requires a parameter within a test group.
We recommend to group related parameters e.g. SPN1 GHI and DNI measurements.
Use flag icons to indicate which quality control results to save for particular parameter.
Flags of a parameter can be saved only within one test group.
Analyst pre-fills test groups by default after import. To change parameter assignment or add / remove test groups use test group editor.
Component | Description |
Irradiance columns | Select columns of irradiance data type you want to quality check. Flags will be calculated into respective flag columns. Assign columns into groups - this is required for performing consistency test - testing calculated irradiance components versus measured. |
Advanced settings | Opens separated dialog with automatic quality check advanced settings. |
run between | Quality check can be performed on a subset of the dataset. If checked, values from both calendar widgets are taken into account. |
date from, date to | Defines a subset of the dataset by selecting two dates. |
Graph | The graphs including the QC flags are displayed. |
Irradiance components | The user can select the irradiances component to be displayed. Plot settings menu is active only for these Graph tabs: Heatmap flag plot, Time series flag plot. |
split heatmap by year | If checked, the heatmap is split by years. This checkbox is active only for the Heatmap flag plot. |
view in time zone | The time zone can be selected. |
Edit test groups | The user can edit, create or delete a test group. |
Run | Will start the process of automatic quality check. The result of the procedure is a flagged instance of every data record (if there are no flag columns, they will be created). Preview dialog (Quality report) is provided to let the user review the results. |
Save flags | This button is only enabled after running the quality check. It permanently saves generated flags into the dataset. QC status “AUTOMATIC_QUALITY_CHECK_IRRADIATION” will be assign to the dataset. |
Edit test groups
Edit test groups to change parameter assignment or add / remove test groups.
Test Groups are prefilled by default based on column order. In case default test groups do not correctly assign parameters, it is recommended to reorder columns via metadata editor and set default test groups. It is also possible to manually modify test groups.
Advanced settings
This dialog allows the user to control the test methodology and to select which automatic QC tests should be performed. Some of the tests have additional settings affecting their outcome.
Component | Description |
Test methodology | Solargis or BSRN methodology can be selected. |
List of tests | The user can select the automatic tests to be performed. |
Acceptable minima | For particular irradiance component the user can override Analyst default values of minimum physical value. |
Consecutive static values limits | User can override the default value of how many consecutive static values of the irradiance component will be considered as artificial (and flagged out from valid values). The limit can be set up separately for DIF values. There is a special treatment for DNI component as consecutive zero values can be valid (i.e., the test is performed for DNI values other than zero). |
Periods to be excluded | If checked, periods from the list will be excluded from GHI-DNI-DIF consistency test. |
list of excluded periods | Sometimes you want to exclude one or more time periods from checking the consistency of irradiance components because you know of the malfunction of one of the sensors (i.e., corrupted rotating shadow band). In that case, you do not want to exclude other irradiance components as they may be correct. |
Add period | Opens input dialog to add a time period to the list of excluded periods. |
Remove period | Remove selected time period from the list. |
Remove all periods | Clear the list of periods. |
Select flag values | The user can select the flag values that should be updated |