Tools, data resource and methods

Adam H. Sparks

Curtin Biometry and Agricultural Data Analytics, Centre for Crop and Disease Management, Curtin University

Why Even Try?

“…all models are wrong…”

George E. P. Box

Photo by DavidMCEddy at en.wikipedia, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=14941622

Why Even Try?

“…all models are wrong,
but some are useful.”

George E. P. Box

We know that climate change will affect the host as well as the pathogen and the systems in which they are interacting.

Shifts in precipitation patterns, temperature, and other environmental factors such as relative humidity will occur due to the effects of climate change.

So why even try to model what the possibilities are? Many, most of us, in this room intrinsically understand these concepts and would agree with these premises, I think. I mean, G. E. Box was right, all models are wrong, but some are useful.

So, here, I think that they are useful because they allow us to explore the possibilities. They allow us to -see- where and when these things may (or may not) occur. Most people, 65%, (Zopf et al. 2004) are visual learners. Using models allows us to visualise these concepts and possibilities using maps, figures, and other interactive methods of exploring the model outputs.

[Photo by DavidMCEddy at en.wikipedia, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=14941622]

Disclaimer

Before I get to far, I’d like to provide the framework in which this talk was written.

• I’ve approached this as a literature review of scientific software.

• I have focused on tools and methods that are open source and lend themselves to open science and reproducibility.

  • This means no GUI programs, e.g. STELLA, ArcGIS, etc.

  • This also means, that there are many more models that exist that I will not mention as the model code is not published or openly available and/or the model cannot be or has not been reproduced from the original publication. Therefore, this means that I have chosen only to discuss models for which I could get the source code and install and use, again openness and reproducibility.

• This is as exhaustive as I could make it with the time that I had. If I have missed something, I welcome the addition or correction of any models or tools that I have missed.

• This presentation is available as a webpage with links to resources and references, you may make pull requests to add additional resources.

About Open Science and Reproducibility

From Sparks et al. (2023b)

Recently, Emerson and I along with a few others surveyed 450 plant pathology focused articles from 21 journals published over five years.

The scale on the x-axis is the count of articles. The scale on the y-axis is

• 0 Not available and not mentioned in publication.

• 1 Available upon request to author.

• 2 Online, but inconvenient/non-permanent, e.g., login necessary, pay wall, FTP server, personal lab website.

• 3 Freely available online to anonymous users for foreseeable future, e.g. archived using Zenodo, dataverse or university library or some other proper archiving system.

• NA No computational methods were used that can be shared, i.e., there is no statistical analysis or figures that were prepared. For example, a molecular study used proprietary software for the experiment imaging and only gel images are included in the article. Then there are no computational methods to be shared.

About Open Science and Reproducibility

From Sparks et al. (2023b)

Most of the articles were not focused on modelling either as evidenced by the most commonly used software products that were properly cited.

About Open Science and Reproducibility

From Sparks et al. (2023b)

Most of the articles were not focused on modelling either as evidenced by the most commonly used software products that were properly cited. R does appear in this list, but it was mainly for statistical analysis. SAS still is the most frequent for analysis, but I will note for the purposes of this presentation that I am aware that at least at one time a version of LATEBLIGHT did exist written in SAS. However, that is outside the scope of this talk.

I want to highlight here that most articles did not freely or openly share their code, making reproducibility difficult or impossible.

It is through this lens that I’ve approached this talk, hoping to improve awareness of tools that can be used to make models more open and reproducible, which will make them more widely used rather than just a one-off publication that never gets revisited or reused.

Common Objectives

• Disease epidemics

• Yield losses

• Sometimes both

Most modelling exercises I’ve been involved in or come across in the literature fall into two broad categories.

In 2015 (Juroszek and von Tiedemann 2015) reviewed 70 papers that modelled the effects of climate change in about 35 plant diseases in 15 crops with wheat followed by grape and then canola diseases were most modelled. The geographic areas that were most focused on were Brazil, Western Europe with few being global. Interestingly, when I researched for this presentation, I found that most published models with freely available code focused on rice diseases or potato late blight.

Challenges

Issues of Scale

• Epidemiological processes

• Crop models

Both the long-term This doesn’t match either of the processes or crop model time-scales

Time slices are an average set of values representing a period of years so you don’t need to run multiple years, but this lacks the uncertainty inherent in weather as it’s climate. You end up with an estimate of an “average” year.

GCM spatial outputs often don’t match epidemiological processes or the scientists desired geographic scale either.

(See Savary et al. 2015 for more on these challenges)

Issues with Getting Data

General Circulation Model outputs are freely available for download and use, but they are in cumbersome file formats, e.g. NetCDF or difficult to find or understand what you want to download for your own uses.

Or the data you need isn’t available, e.g., many models use RH, but this parameter often isn’t available or the CMIP version you want to use isn’t available, it goes on.

Or the downloads are huge, you can’t just download a little bit, you must download gigabytes of data that takes time and storage space.

Computational Tools

The Main Scientific Programming Languages

As I previously noted, I’ve focused on freely-available resources, that means that the languages themselves should be ideally Free-Open-Source (FOSS) offerings so that anyone can use, reproduce, modify the work.

The three most commonly used languages are arguably, Julia, Python and R. There are other languages, e.g. Matlab, SAS, Rust, FORTRAN and others, however, most of these are not FOSS and are not as widely used.

Julia logo by By The Julia Project - https://julialang.org, Public Domain, Link
Python logo by www.python.org - www.python.org, GPL, Link
R logo by The R Foundation - https://www.r-project.org/logo/, CC BY-SA 4.0, Link

Primary Package Registries

I intentionally (mostly) limited myself to climate tools that are only available for global data and installable from the languages authoritative central repository, i.e., JuliaHub, CRAN or PyPi only to keep these tools to a manageable list and hopefully help ensure some level of quality control. I did deviate a bit since several Python packages are available via GitHub but not PyPi and one was reasonably easy to use and responsive to a pull request that I made.

[Julia logo by By The Julia Project - https://julialang.org, Public Domain, Link
PyPi logo by Python Packaging Authority / Python Software Foundation - https://pypi.org/static/images/logo-large.svg, GPL, Link
R logo by The R Foundation - https://www.r-project.org/logo/, CC BY-SA 4.0, Link]

Acquiring Climate Data

Using a programmatic/scriptable method

Here I’ve selected interfaces for which a client in any one of the three languages is available and that I found I could follow the README and install and get data. Even only using those packages in official repositories, I found a few that just didn’t work along the way and so they aren’t mentioned here.

Acquiring Climate Data with Julia

• CDSAPI.jl Julia API to the Climate Data Store (a.k.a. CDS)¹ [monthly and daily] (Yun Chan 2023)

• RasterDataSources.jl Easily download and use raster data sets in Julia (e.g., WorldClim, CHELSA) (Schouten and Poisot 2023)

  • WorldClim [monthly] (Fick and Hijmans 2017)
  • CHELSA (Climatologies at high resolution for the earth’s land surface areas) [monthly and daily] (Karger et al. 2017, 2020)

I was able to get data with CDSAPI.jl easily.

I struggled with the RasterDataSources.jl, I was able to fetch WorldClim data with it, but not any of the CHELSA future data for CMIP5 or CMIP6 that are supposed to be supported. I found the documentation to be unclear and lacking examples of how to use it to request these data.

1. The ECMWF CDS service is currently undergoing changes which might impact performance including longer download queues, download times and or dropped requests.

Acquiring Climate Data with Python

• pynasapower Download meteorological data from NASA POWER using a simple Python API client¹ [annual, monthly and daily] (Falagas 2023)

• cdsapi Python API to access the Copernicus Climate Data Store (CDS)² [monthly and daily] (ECMWF 2023)

• latlon_utils Retrieve WorldClim climate and other information for lat-lon grid cells [monthly] (Sommer 2022)

1. Not in PyPi, only GitHub.

2. The ECMWF CDS service is currently undergoing changes which might impact performance including longer download queues, download times and or dropped requests.

Acquiring Climate Data with R

• {ecmwfr} Interface to the public ECMWF API Web Services (Hufkens et al. 2019), includes:

  • Copernicus Climate Data Store (CDS) [monthly and daily]¹

1. The ECMWF CDS service is currently undergoing changes which might impact performance including longer download queues, download times and or dropped requests.

Acquiring Climate Data with R

• {geodata} Download geographic data (Hijmans et al. 2023), including:

  • CMIP6 Data [monthly]

  • WorldClim

    • Historical [monthly]

    • Future [monthly]

Acquiring Climate Data with R

• {nasapower} API Client for NASA POWER Global Meteorology, Surface Solar Energy and Climatology in R [annual, monthly and daily] (Sparks 2018)

• {climenv} Download, extract and visualise climatic and elevation data (e.g., CHELSA) [monthly and daily] (Tsakalos et al. 2023)

On Plant Disease Models and Modelling

Model Frameworks, Scripts and Packages or Libraries

There are several models that have been published. Unfortunately, few have the source code freely and openly available as well. Searching returns many models for detection and classification/identification using imagery, not many for yield loss predictions or disease severity. Most seem to be potato or rice related where they exist.

Plant Disease Models and Frameworks

I’ve not limited to traditional repositories as they are fewer and farther between, but I have tried to limit them to models or frameworks that were published in peer-reviewed manuscripts.

Plant Disease Models and Frameworks in Julia

• Epicrop.jl (Sparks 2022) based on (Savary et al. 2012)

Provides models for unmanaged epidemics of:

• bacterial blight,
• brown spot,
• leaf blast,

• sheath blight, and
• rice tungro disease.

I’m only aware of one plant disease-related Julia package in the Julia package registry. This is based on Serge’s EPIRICE model (Savary et al. 2012) that I translated to Julia for efficiency when doing global modelling. This model framework is officially registered, but has not been used in any peer-reviewed publications to my knowledge.

I’ve started implementing framework for EPIWHEAT but am lacking some of the coefficients that are necessary for the model to run and I’ve not had time to think about how to derive them. This goes back to my point about being open and reproducible and publishing the code for the models.

Plant Disease Models and Frameworks in Python

• RICEPEST¹, ² (Duku et al. 2015)

Provides models for rice yield losses due to:

• leaf blast, and
• bacterial blight

There is one Python script that Dr. Confidence Duku, AfricaRice at the time, wrote with my assistance based on Laetitia Willocquet’s RICEPEST. I’m hoping to convert this script from being reliant on ArcGIS to a more FOSS foundation and eventually strip out the reliance on GIS and make it something more generic and vectorised like Epicrop.jl and {epicrop}, which do not need to be in a GIS to be run, but can be run in a GIS.

I did find lots of image analysis modules for identifying plant disease, mainly foliar diseases and many of these were rice foliar diseases, but nothing for predicting or modelling epidemic dynamics, just this one that does crop losses.

It’s also unique because I didn’t find any other models in any of these three languages that were used for crop losses.

It could be extended to WHEATPEST and this would be ideal.

1. Technically this relies on ArcGIS currently so it shouldn’t be here. But, efforts to translate it to be truly FOSS are under way.

2. There are no other good options written in Python that I can find, so I felt bad for the language and included this in y presentation.

Plant Disease Models and Frameworks in R

Scripts for Potato Late Blight

• SimCast (blight units only; hourly time step) (Sparks et al. 2011) 4.0

• SimCastMeta (SimCast on a daily and monthly time-step) (Sparks et al. 2014) 4.0

• BLIGHTSIM (Script is a PDF file hosted with MDPI) (Narouei-Khandan et al. 2020) 4.0

SimCast model based on work by (Fry et al. 1983; Grünwald et al. 2002)

Plant Disease Models and Frameworks in R

Scripts for Rice Diseases

• Modified EPIRICE (Kim 2015), see Kim et al. (2015) Tables 1 and 2 for modifications.

Provides models for:

• leaf blast, and
• sheath blight

in Korean rice paddies with localised cultivars.

Dr Kim used the original EPIRICE with modifications which are well documented in the paper and available in a script that is archived on FigShare as well. If I recall correctly, this was how I first met Dr Kim, when he contacted me with questions about this model and the codebase.

Plant Disease Models and Frameworks in R

Packages for Potato Late Blight

• {Rlateblight} (Hijmans 2017) provides:

  • Hyre (Hyre 1954),

  • Wallin (Wallin 1962),

  • Blightcast (MacKenzie 1981)

• {agricolae} (Mendiburu 2023) provides:

  • lateblight() (Bruhn et al. 1980; Doster et al. 1990; Andrade-Piedra et al. 2005b, 2005c, 2005a)

Plant Disease Models and Frameworks in R

Packages for Rice Diseases

• {cropsim} (Hijmans et al. 2009)

• {epicrop} (Sparks et al. 2023a) (a fork of {cropsim})

Both provide models for unmanaged epidemics of:

• bacterial blight,
• brown spot,
• leaf blast,

• sheath blight, and
• rice tungro disease.

{cropsim} was the original version that was used at IRRI that implemented the EPIRICE model from (Savary et al. 2012). I used and modified this version for (Duku et al. 2015).

I’ve forked it to {epicrop} to simplify the use of the model and make it more intuitive and less reliant on being run inside a GIS. The model can run now with just weather data in a single data.frame or can be run over a spatially enabled data.frame to be used in mapping disease severities.

I’ve started implementing framework for EPIWHEAT into {epicrop} but am lacking some of the coefficients that are necessary for the model to run and I’ve not had time to think about how to derive them. This goes back to my point about being open and reproducible and publishing the code for the models.

Summary

• Overall lack of support for this sort of work

  • Few off-the shelf data sets on a daily-time step that can be (programmatically) downloaded

• All three programming languages offer some level of support

• All three have advantages and disadvantages, but you can mix-n-match languages

• The ways of working in this area still are not optimal

I finished my PhD almost 15 years ago now working on the data/model mismatch issue. It’s still a dog’s breakfast now.

There are more resources to get daily data but they’re hard to find and understand what you are downloading if you’re new to this, or coming back to it after much time away.

Weather generators are real and a thing but require some know-how to work with them, I hesitate to do that myself.

To pick the one with the best support for your needs, get weather data with Python, model in Julia and visualise in R.

DOI: 10.5281/zenodo.10949552

0000-0002-0061-8359
@adamhsparks
adamhsparks
Codeberg adamhsparks
adam.sparks@curtin.edu.au

Resources

Working Environments with Examples of the Tools in this Presentation

Julia

Python

R

Resources

Support for Climate Projects, Tools and Data

• AU Government and CSIRO Climate Change Information, projections, tools and data learning and support

• What are RCPs

Useful Climate Change Data Sets

• https://climatedataguide.ucar.edu/climate-data

• https://atlas.climate.copernicus.eu/atlas

R for Plant Disease Epidemiology

• https://r4pde.net/, a good primer, this covers temporal analysis, spatial analysis, epidemics and yield and disease prediction.

Weather Data Generators

CGIAR MarkSim Web for IPCC AR5 data (CMIP5)¹

LARS-WG A Stochastic Weather Generator for Use in Climate Impact Studies²

SDSM Statistical Downscaling Model³

pyClim-SDM⁴

1. Online interface

2. Windows .exe only

3. Windows .exe only

4. Claims to be Linux only

Examples

Modelling Country Level Yield Losses

Flowchart illustrating the loose coupling between EPIRICE, used to simulate unmanaged disease epidemics as affected by weather conditions, and RICEPEST, used to estimate yield losses due to leaf blast and bacterial leaf blight severity. Three time slices, 2000, 2030 and 2050 were examined for four climate scenarios, current or base, IPCC A1B, IPCC A2 and IPCC B1 (Duku et al. 2015)

“Spatial Modelling of Rice Yield Losses in Tanzania Due to Bacterial Leaf Blight and Leaf Blast in a Changing Climate” (Duku et al. 2015)

Modelling Country Level Yield Losses

Bacterial leaf blight disease severity curves for the growing season from December to March as predicted by the EPIRICE model averaged for the whole country of Tanzania using observed and future, modelled weather data for three climate change scenarios, A1B; A2 and B1 and three time slices, 2000; 2030 and 2050. The EPIRICE model predicts the area under the disease progress curve (AUDPC) of unmanaged disease epidemics of a susceptible rice variety using temperature, relative humidity and precipitation data (Duku et al. 2015)

“Spatial Modelling of Rice Yield Losses in Tanzania Due to Bacterial Leaf Blight and Leaf Blast in a Changing Climate” (Duku et al. 2015)

Modelling Country Level Yield Losses

Changes in yield losses due to bacterial leaf blight disease expressed as tons per hectare for two future time slices and three IPCC emission scenarios as predicted by the RICEPEST model. The change in yield loss is calculated as the difference of the future yield loss and a base time-slice, 2000 representing 1991 to 2010. Negative values indicate lower yield losses (yield gains) while positive values indicate higher yield losses. RICEPEST is a simple empirical growth model that simulates the growth, yield and yield losses due to pests and diseases of tropical rice based using temperature and radiation inputs and predetermined input levels that include irrigation and fertilization practices and disease severity as predicted by the EPIRICE model. The EPIRICE model predicts disease severity of unmanaged disease of susceptible rice varieties given temperature, relative humidity and precipitation (Duku et al. 2015)

“Spatial Modelling of Rice Yield Losses in Tanzania Due to Bacterial Leaf Blight and Leaf Blast in a Changing Climate” (Duku et al. 2015)

Global Disease Severity

The change in global potato late blight relative risk as predicted by SimCastMeta model using historical climate normal, 1961–1990 (1975 time-slice) and 2040–2059 (2050 time-slice) A2 climate scenario for a susceptible potato genotype. IPCC A2 time-slices are ensemble model averages of three global climate models. Blight units are a predictor of biological risk based on weather and potato genotypic resistance. (Sparks et al. 2014)

“Climate Change May Have Limited Effect on Global Risk of Potato Late Blight” (Sparks et al. 2014)

References

Andrade-Piedra, J. L., Forbes, G. A., Shtienberg, D., Grünwald, N. J., Chacón, M. G., Taipe, M. V., et al. 2005a. Qualification of a plant disease simulation model: Performance of the LATEBLIGHT model across a broad range of environments. Phytopathology. 95:1412–1422.

Andrade-Piedra, J. L., Hijmans, R. J., Forbes, G. A., Fry, W. E., and Nelson, R. J. 2005b. Simulation of potato late blight in the Andes. I: Modification and parameterization of the LATEBLIGHT model. Phytopathology. 95:1191–1199.

Andrade-Piedra, J. L., Hijmans, R. J., Juárez, H. S., Forbes, G. A., Shtienberg, D., and Fry, W. E. 2005c. Simulation of potato late blight in the Andes. II: Validation of the LATEBLIGHT model. Phytopathology. 95:1200–1208.

Bruhn, J. A., Bruck R., I., Fry W., E., Arneson P., A., and Keokosky, E. V. 1980. User’s manual for LATEBLIGHT: A plant disease management game. Ithaca, NY, USA: Cornell University, Department of Plant Pathology.

Doster, M. A., Milgroom, M. G., Fry, W. E., et al. 1990. Quantification of factors influencing potato late blight suppression and selection for metalaxyl resistance in Phytophthora infestans: A simulation approach. Phytopathology. 80:1190–1198.

Duku, C., Sparks, A. H., and Zwart, S. J. 2015. Spatial modelling of rice yield losses in Tanzania due to bacterial leaf blight and leaf blast in a changing climate. Climatic Change. 135:569–583.

ECMWF. 2023. Cdsapi. ECMWF. Available at: https://github.com/ecmwf/cdsapi/tree/master.

Falagas, A. 2023. Pynasapower. Available at: https://github.com/alekfal/pynasapower.

Fick, S. E., and Hijmans, R. J. 2017. WorldClim 2: New 1‐km spatial resolution climate surfaces for global land areas. International Journal of Climatology. 37:4302–4315.

Fry, W. E., Apple, A. E., and Bruhn, J. A. 1983. Evaluation of potato late blight forecasts modified to incorporate host resistance and fungicide weathering. Phytopathology. 73:1054–1059.

Grünwald, N. J., Montes, G. R., Saldaña, H. L., Covarrubias, O. A. R., and Fry, W. E. 2002. Potato late blight management in the Toluca Valley: Field validation of SimCast modified for cultivars with high field resistance. Plant Disease. 86:1163–1168.

Hijmans, R. J. 2017. Rlateblight: Potato late blight models.

Hijmans, R. J., Barbosa, M., Ghosh, A., and Mandel, A. 2023. geodata: Download geographic data. Available at: https://CRAN.R-project.org/package=geodata.

Hijmans, R. J., Savary, S., Pangga, R., and Aunario, J. 2009. cropsim: Simulation modeling of crops and their diseases.

Hufkens, K., Stauffer, R., and Campitelli, E. 2019. The ecwmfr package: An interface to ECMWF API endpoints. Available at: https://bluegreen-labs.github.io/ecmwfr/.

Hyre, R. A. 1954. Progress in forecasting late blight of potato and tomato.

Juroszek, P., and von Tiedemann, A. 2015. Linking plant disease models to climate change scenarios to project future risks of crop diseases: A review. Journal of Plant Diseases and Protection. 122:3–15.

Karger, D. N., Conrad, O., Böhner, J., Kawohl, T., Kreft, H., Soria-Auza, R. W., et al. 2017. Climatologies at high resolution for the earth’s land surface areas. Scientific Data. 4.

Karger, D. N., Schmatz, D. R., Dettling, G., and Zimmermann, N. E. 2020. High-resolution monthly precipitation and temperature time series from 2006 to 2100. Scientific Data. 7.

Kim, K.-H. 2015. Modified EPIRICE ref.

Kim, K.-H., Cho, J., Lee, Y. H., and Lee, W.-S. 2015. Predicting potential epidemics of rice leaf blast and sheath blight in South Korea under the RCP 4.5 and RCP 8.5 climate change scenarios using a rice disease epidemiology model, EPIRICE. Agricultural and Forest Meteorology. 203:191–207.

MacKenzie, D. R. 1981. Scheduling fungicide applications for potato late blight with Blitecast.

Mendiburu, F. de. 2023. agricolae: Statistical procedures for agricultural research. Available at: https://CRAN.R-project.org/package=agricolae.

Narouei-Khandan, H. A., Shakya, S. K., Garrett, K. A., Goss, E. M., Dufault, N. S., Andrade-Piedra, J. L., et al. 2020. BLIGHTSIM: A new potato late blight model simulating the response of Phytophthora infestans to diurnal temperature and humidity fluctuations in relation to climate change. Pathogens. 9:659.

Savary, S., Esker, P., McRoberts, N., Willocquet, L., Caffi, T., Rossi, V., et al. 2015. Models for crop diseases: Overview of approaches & scales.

Savary, S., Nelson, A., Willocquet, L., Pangga, I., and Aunario, J. 2012. Modeling and mapping potential epidemics of rice diseases globally. Crop Protection. 34:6–17.

Schouten, R., and Poisot, T. 2023. RasterDataSources.jl. Available at: https://github.com/EcoJulia/RasterDataSources.jl.

Sommer, P. S. 2022. Latlon_utils. Available at: https://github.com/Chilipp/latlon-utils.

Sparks, A. H. 2018. nasapower: A NASA POWER global meteorology, surface solar energy and climatology data client for R. The Journal of Open Source Software. 3:1035.

Sparks, A. H. 2022. Simulation modelling of crop diseases using a Healthy-Latent-Infectious-Postinfectious (HLIP) model in Julia. Available at: https://github.com/adamhsparks/Epicrop.jl.

Sparks, A. H., Forbes, G. A., Hijmans, R. J., and Garrett, K. A. 2011. A metamodeling framework for extending the application domain of process-based ecological models. Ecosphere. 2:art90.

Sparks, A. H., Forbes, G. A., Hijmans, R. J., and Garrett, K. A. 2014. Climate change may have limited effect on global risk of potato late blight. Global Change Biology. 20:3621–3631.

Sparks, A. H., Hijmans, R., Savary, S., Pangga, I., and Aunario, J. 2023a. epicrop: Simulation modelling of crop diseases using a susceptible-exposed-infectious-removed (SEIR) model. Available at: https://codeberg.org/adamhsparks/epicrop.

Sparks, A. H., Ponte, E. M. D., Alves, K. S., Foster, Z. S. L., and Grünwald, N. J. 2023b. Openness and computational reproducibility in plant pathology: Where we stand and a way forward. Phytopathology®. 113:1159–1170.

Tsakalos, J. L., Smith, M. R., Luebert, F., and Mucina, L. 2023. climenv: Download, extract and visualise climatic and elevation data. Journal of Vegetation Science.

Wallin, J. R. 1962. Summary of recent progress in predicting late blight epidemics in United States and Canada. American Potato Journal. 39:306–312.

Yun Chan, M. 2023. CDSAPI.jl. Available at: https://github.com/JuliaClimate/CDSAPI.jl.

Zopf, R., Giabbiconi, C. M., Gruber, T., and Müller, M. M. 2004. Attentional modulation of the human somatosensory evoked potential in a trial-by-trial spatial cueing and sustained spatial attention task measured with high density 128 channels EEG. Cognitive Brain Research. 20:491–509 Available at: https://www.sciencedirect.com/science/article/pii/S0926641004001296 [Accessed March 6, 2024].