REcoTox - a workflow to process US EPA ECOTOX Knowledgebase ASCII files
+ +2023-11-04
+ +1 Background
+The search and extraction of experimental ecotoxicological information +is often a tedious work. A good and comprehensive data source is the US +EPA ECOTOX +Knowledgebase. +It contains more than 1 million data points for almost 13,000 chemicals +and 14,000 single species. However, for a high-throughput hazard +assessment, it is not possible to extract all relevant data of the +online database. The purpose of REcoTox is to extract the relevant +information and to aggregate the data based on the user criteria out of +the entire database ASCII +files.
+2 Introduction
+REcoTox is a semi-automated,
+interactive workflow to process US EPA ECOTOX
+Knowledgebase
+entire database ASCII
+files
+to extract and process ecotoxicological data relevant (but not
+restricted) to the ecotoxicity groups algae, crustaceans, and fish in
+the aquatic domain. The focus is aquatic ecotoxicity and the unit of the
+retrieved data is mg/L.
3 Input files and folders
+REcoTox requires an unzipped US EPA Knowlegdebase database in
+ASCII format (Zitat). The database is preferable expanded in an own
+database folder to be defined during the processing, The database
+consists of relatively referenced text files. The separator of the data
+is the pipeline | symbol.
In the first session of REcoTox, a file chemical_properties.csv is
+created in the database folder. This files contains chemical identifiers
+and chemical properties required for the processing of the chemical data
+in the knowlegdebase and to tag the results.
The chemical property file is dynamically updated and requires also some +manual curation. It will grow as soon new chemicals are added to the +knowledgebase.
+The project_folder contains the R script for processing as well as
+the intermediate and final processing files. The naming of the folder is
+arbitrary, but do not use spaces, but underscores (_) or hyphens (-)
+for separating parts.
To run the queries, a predefined processing script is available on
+GitHub
+(Query_EcoTox_DB.R)
+or in the local REcoTox package folder.
4 Using REcoTox
+The following tutorial explains the different steps of REcoTox in a
+comprehensive demonstration. REcoTox includes different interactive
+steps, which require the evaluation of comma separated text files
+(*.csv) in an external spreadsheet application (preferable LibreOffice
+[1]).
4.1 Load the REcoTox package
+# Load the REcoTox package
+library(REcoTox)4.2 Documentation for MZquant
+A detailed description of all functions of REcoTox functions is
+available in the R Documentation.
# Documentation of REcoTox
+help(package = "REcoTox")4.3 Preparation of the working environment (for beginners)
+The processing in REcoTox is interactivally controlled by a processing
+script Query_EcoTox_DB.R.
If you run REcoTox for the first time, a tutorial project is available
+to demonstrate all important steps of REcoTox processing. The
+following script is preparing an example folder in your home directory
+and copies all necessary files in the folder.
# Path of the project folder
+project_folder <- "REcoTox_demo"
+
+database_folder <- system.file("extdata/database_folder", package="REcoTox")
+# The project folder is created in the home directory
+project_path <- normalizePath(ifelse(.Platform$OS.type == "unix",
+ paste0("~/", project_folder),
+ paste0(
+ Sys.getenv("HOMEPATH"),
+ "\\",
+ project_folder
+ )
+))
+
+# An existing folder is deleted
+if (dir.exists(project_folder)) {
+ unlink(project_folder, recursive = TRUE)
+}This command initializes the project folder and the database folder. It +copies also the processing script to the project folder.
+project <- REcoTox::create_project(database_path = database_folder,
+ project_path,
+ initalise_database_project = TRUE, # create the basic project from current ASCII files in DB folder
+ initalise_project = TRUE, # initializes the project folder
+ load_default = FALSE) # loads the default project in the project folder in the memoryfault_example = TRUE
+
+file.copy(
+ from = system.file(
+ "extdata",
+ "Query_EcoTox_DB.R",
+ package = "REcoTox"
+ ),
+ to = normalizePath(
+ path = file.path(
+ project_folder,
+ "Query_EcoTox_DB.R"
+ ),
+ winslash = "\\",
+ mustWork = FALSE
+ ),
+ overwrite = TRUE
+ )## [1] FALSEThe project_folder contains the following files:
# List files and directories in project_folder
+list.files(project_folder, recursive = TRUE, include.dirs = TRUE)## character(0)The database_folder contains the following files and folders:
+chemical_properties.csv is the file containing the curated chemical
+properties, results.txt contains the testing results collected in the
+knowledgebase, and test.txt contains the the metadate of the tests.
The folder validation contains the files chemicals.txt with chemical
+information, the file references.txt contains the references and
+species.txt the species.
# List files and directories in project_folder
+list.files(database_folder, recursive = TRUE, include.dirs = TRUE)## [1] "chemical_properties.csv" "project.RData"
+## [3] "results.txt" "tests.txt"
+## [5] "validation" "validation/chemicals.txt"
+## [7] "validation/references.txt" "validation/species.txt"It contains only the Query_EcoTox_DB.R file.
4.4 Review of the input data
+To review the input data, let us look in the data:
+# Review of the chemical properties
+chemical_properties <- readr::read_csv(file = normalizePath(path = file.path(
+ database_folder,
+ "chemical_properties.csv"
+), ), show_col_types = FALSE)
+
+kable(
+ chemical_properties %>%
+ head(5),
+ format = "html",
+ digits = 2
+) %>%
+kable_styling("striped", full_width = TRUE) %>%
+scroll_box(width = "700px", height = "300px")| +cas_number + | ++cas + | ++chemical_name + | ++dtxsid_ecotox + | ++PubChem_CID + | ++FOUND_BY + | ++DTXSID_DTX + | ++PREFERRED_NAME + | ++CASRN + | ++INCHIKEY + | ++IUPAC_NAME + | ++SMILES + | ++INCHI_STRING + | ++QSAR_READY_SMILES + | ++MOLECULAR_FORMULA + | ++AVERAGE_MASS + | ++MONOISOTOPIC_MASS + | ++QC_LEVEL + | ++LOG_S + | ++LOG_S_AD + | ++S_AD_index + | ++LOG_S_COMMENT + | ++EXCLUDE + | ++REMARKS + | +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| +1912249 + | ++1912-24-9 + | ++6-Chloro-N-ethyl-N’-(1-methylethyl)-1,3,5-triazine-2,4-diamine + | ++DTXSID9020112 + | ++2256 + | ++DSSTox_Substance_Id + | ++DTXSID9020112 + | ++Atrazine + | ++1912-24-9 + | ++MXWJVTOOROXGIU-UHFFFAOYSA-N + | ++6-Chloro-N2-ethyl-N4-(propan-2-yl)-1,3,5-triazine-2,4-diamine + | ++CCNC1=NC(NC(C)C)=NC(Cl)=N1 + | ++InChI=1S/C8H14ClN5/c1-4-10-7-12-6(9)13-8(14-7)11-5(2)3/h5H,4H2,1-3H3,(H2,10,11,12,13,14) + | ++CCNC1N=C(Cl)N=C(NC(C)C)N=1 + | ++C8H14ClN5 + | ++215.69 + | ++215.09 + | ++1 + | ++-3.7 + | ++1 + | ++1 + | ++OPERA_2.9 + | ++NA + | ++NA + | +
# Review of the result table
+results <-
+ readr::read_delim(
+ file = normalizePath(
+ path = file.path(
+ database_folder,
+ "results.txt"
+ ),
+ ),
+ show_col_types = FALSE,
+ delim = "|"
+
+ )
+
+kable(
+ results %>%
+ head(5),
+ format = "html", digits = 2
+) %>%
+kable_styling("striped", full_width = TRUE) %>%
+scroll_box(width = "700px", height = "300px")| +result_id + | ++test_id + | ++sample_size_mean_op + | ++sample_size_mean + | ++sample_size_min_op + | ++sample_size_min + | ++sample_size_max_op + | ++sample_size_max + | ++sample_size_unit + | ++sample_size_comments + | ++obs_duration_mean_op + | ++obs_duration_mean + | ++obs_duration_min_op + | ++obs_duration_min + | ++obs_duration_max_op + | ++obs_duration_max + | ++obs_duration_unit + | ++obs_duration_comments + | ++endpoint + | ++endpoint_comments + | ++trend + | ++effect + | ++effect_comments + | ++measurement + | ++measurement_comments + | ++response_site + | ++response_site_comments + | ++effect_pct_mean_op + | ++effect_pct_mean + | ++effect_pct_min_op + | ++effect_pct_min + | ++effect_pct_max_op + | ++effect_pct_max + | ++effect_pct_comments + | ++conc1_type + | ++ion1 + | ++conc1_mean_op + | ++conc1_mean + | ++conc1_min_op + | ++conc1_min + | ++conc1_max_op + | ++conc1_max + | ++conc1_unit + | ++conc1_comments + | ++conc2_type + | ++ion2 + | ++conc2_mean_op + | ++conc2_mean + | ++conc2_min_op + | ++conc2_min + | ++conc2_max_op + | ++conc2_max + | ++conc2_unit + | ++conc2_comments + | ++conc3_type + | ++ion3 + | ++conc3_mean_op + | ++conc3_mean + | ++conc3_min_op + | ++conc3_min + | ++conc3_max_op + | ++conc3_max + | ++conc3_unit + | ++conc3_comments + | ++bcf1_mean_op + | ++bcf1_mean + | ++bcf1_min_op + | ++bcf1_min + | ++bcf1_max_op + | ++bcf1_max + | ++bcf1_unit + | ++bcf1_comments + | ++bcf2_mean_op + | ++bcf2_mean + | ++bcf2_min_op + | ++bcf2_min + | ++bcf2_max_op + | ++bcf2_max + | ++bcf2_unit + | ++bcf2_comments + | ++bcf3_mean_op + | ++bcf3_mean + | ++bcf3_min_op + | ++bcf3_min + | ++bcf3_max_op + | ++bcf3_max + | ++bcf3_unit + | ++bcf3_comments + | ++significance_code + | ++significance_type + | ++significance_level_mean_op + | ++significance_level_mean + | ++significance_level_min_op + | ++significance_level_min + | ++significance_level_max_op + | ++significance_level_max + | ++significance_comments + | ++chem_analysis_method + | ++chem_analysis_method_comments + | ++endpoint_assigned + | ++organism_final_wt_mean_op + | ++organism_final_wt_mean + | ++organism_final_wt_min_op + | ++organism_final_wt_min + | ++organism_final_wt_max_op + | ++organism_final_wt_max + | ++organism_final_wt_unit + | ++organism_final_wt_comments + | ++intake_rate_mean_op + | ++intake_rate_mean + | ++intake_rate_min_op + | ++intake_rate_min + | ++intake_rate_max_op + | ++intake_rate_max + | ++intake_rate_unit + | ++intake_rate_comments + | ++lipid_pct_mean_op + | ++lipid_pct_mean + | ++lipid_pct_min_op + | ++lipid_pct_min + | ++lipid_pct_max_op + | ++lipid_pct_max + | ++lipid_pct_comments + | ++dry_wet + | ++dry_wet_pct_mean_op + | ++dry_wet_pct_mean + | ++dry_wet_pct_min_op + | ++dry_wet_pct_min + | ++dry_wet_pct_max_op + | ++dry_wet_pct_max + | ++dry_wet_pct_comments + | ++steady_state + | ++additional_comments + | ++companion_tag + | ++created_date + | ++modified_date + | ++old_terretox_result_number + | +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| +198582 + | ++1212738 + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++72 + | ++NA + | ++NA + | ++NA + | ++NA + | ++h + | ++NA + | ++EC50 + | ++NA + | ++DEC + | ++POP + | ++NA + | ++PGRT + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++F + | ++NA + | ++NA + | ++3367.9 + | ++NA + | ++NA + | ++NA + | ++NA + | ++ug/L + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++U + | ++NA + | ++P + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++LIFESTG/STRAIN// + | ++NA + | ++01/19/2001 + | ++NA + | ++NA + | +
| +2176441 + | ++2095522 + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++72 + | ++NA + | ++NA + | ++NA + | ++NA + | ++h + | ++NA + | ++LOEC + | ++NA + | ++DEC + | ++BCM + | ++NA + | ++FLRS/ + | ++NON-PHOTOCHEMICAL QUENCHING + | ++NA + | ++NA + | ++NA + | ++NA + | ++> + | ++60 + | ++< + | ++80 + | ++OF CONTROL, FROM GRAPH + | ++F + | ++NA + | ++NA + | ++250 + | ++NA + | ++NA + | ++NA + | ++NA + | ++nM + | ++DATA NOT PRESENTED FOR 25, 65, 125, 500, 1000, 2000 AND 10000 NM CONCS + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++SIG + | ++P + | ++< + | ++0.05 + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++U + | ++NA + | ++R + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++EFCT%/OF CONTROL, FROM GRAPH//CONC1/DATA NOT PRESENTED FOR 25, 65, 125, 500, 1000, 2000 AND 10000 NM CONCS// + | ++NA + | ++02/07/2014 + | ++02/07/2014 + | ++NA + | +
| +713354 + | ++1262587 + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++24 + | ++NA + | ++NA + | ++NA + | ++NA + | ++h + | ++NA + | ++EC50 + | ++NA + | ++DEC + | ++POP + | ++NA + | ++ABND/ + | ++AUTHOR REPORTED CULTURE POPULATION CHANGE AS INHIBITION OF ALGAL REPRODUCTION + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++A + | ++NA + | ++NA + | ++0.082 + | ++NA + | ++0.07 + | ++NA + | ++0.09 + | ++umol/L + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++U + | ++NA + | ++P + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++LIFESTG/KRAUSS, STRAIN 211-15, GROWN PHOTOAUTOTROPHICALLY IN INORGANIC STERILISED MEDIUM, ENRICHED WITH 1.9MMOL PER LITER NAHCO3 AS BUFFER//DNUM/13//TESTID/1//OEF, BASELINE EC50 BASED ON QSAR, NOT CODED, FROM REVIEW// + | ++NA + | ++10/24/2007 + | ++NA + | ++NA + | +
| +2200676 + | ++2110007 + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++6 + | ++NA + | ++NA + | ++NA + | ++NA + | ++wk + | ++NA + | ++NOEC + | ++NA + | ++DEC + | ++POP + | ++NA + | ++ABND/ + | ++STATS TO V CONTROL + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++A + | ++NA + | ++NA + | ++NA + | ++NA + | ++4.33 + | ++NA + | ++268.00 + | ++ug/L + | ++ONLY CONC TESTED + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++ANOSIG + | ++A + | ++NA + | ++0.05 + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++M + | ++NA + | ++R + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++CONC1/ONLY CONC TESTED// + | ++NA + | ++06/11/2014 + | ++06/11/2014 + | ++NA + | +
| +2380412 + | ++2190107 + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++96 + | ++NA + | ++NA + | ++NA + | ++NA + | ++h + | ++NA + | ++NOEL + | ++NA + | ++DEC + | ++CEL + | ++NA + | ++NLEI/ + | ++RELATIVE VOLUME FRACTION. MITOCHONDRIA ALSO REPORTED. SEVERAL STATISTICAL METHODS USED. + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++A + | ++NA + | ++NA + | ++89 + | ++NA + | ++NA + | ++NA + | ++NA + | ++ug/L + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++ANOSIG + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++M + | ++NA + | ++R + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++NA + | ++CHAR/USED WITH TREATMENT.//DNUM/4//DOSES/22/42/89 ug/L// + | ++NA + | ++04/11/2017 + | ++04/11/2017 + | ++NA + | +
# Review of the substance_table
+substances <-
+ readr::read_delim(
+ file = normalizePath(
+ path = file.path(
+ database_folder,
+ "validation",
+ "chemicals.txt"
+ ),
+ ),
+ show_col_types = FALSE,
+ delim = "|"
+
+ )
+
+kable(
+ substances %>%
+ head(5),
+ format = "html", digits = 2
+) %>%
+kable_styling("striped", full_width = TRUE) %>%
+scroll_box(width = "700px", height = "300px")| +cas_number + | ++chemical_name + | ++ecotox_group + | ++dtxsid + | +
|---|---|---|---|
| +1912249 + | ++6-Chloro-N-ethyl-N’-(1-methylethyl)-1,3,5-triazine-2,4-diamine + | ++Endocrine Disrupting Chemicals (EDCs) + | ++DTXSID9020112 + | +
# Review of the substance_table
+references <-
+ readr::read_delim(
+ file = normalizePath(
+ path = file.path(
+ database_folder,
+ "validation",
+ "references.txt"
+ ),
+ ),
+ show_col_types = FALSE,
+ delim = "|"
+
+ )
+
+kable(
+ references %>%
+ head(5),
+ format = "html", digits = 2
+) %>%
+kable_styling("striped", full_width = TRUE) %>%
+scroll_box(width = "700px", height = "300px")| +reference_number + | ++reference_db + | ++reference_type + | ++author + | ++title + | ++source + | ++publication_year + | +
|---|---|---|---|---|---|---|
| +95 + | ++A + | ++NA + | ++Abou-Waly,H., M.M. Abou-Setta, H.N. Nigg, and L.L. Mallory + | ++Growth Response of Freshwater Algae, Anabaena flos-aquae and Selenastrum capricornutum to Atrazine and Hexazinone Herbicides + | ++Bull. Environ. Contam. Toxicol.46(2): 223-229 + | ++1991 + | +
| +344 + | ++AT + | ++OPP + | ++U.S. Environmental Protection Agency + | ++Pesticide Ecotoxicity Database (Formerly: Environmental Effects Database (EEDB)) + | ++Environmental Fate and Effects Division, U.S.EPA, Washington, D.C.: + | ++1992 + | +
| +393 + | ++A + | ++NA + | ++Hersh,C.M., and W.G. Crumpton + | ++Atrazine Tolerance of Algae Isolated From Two Agricultural Streams + | ++Environ. Toxicol. Chem.8(4): 327-332 + | ++1989 + | +
| +624 + | ++A + | ++NA + | ++Valentine,J.P., and S.W. Bingham + | ++Influence of Algae on Amitrole and Atrazine Residues in Water + | ++Can. J. Bot.54(18): 2100-2107 + | ++1976 + | +
| +682 + | ++A + | ++NA + | ++Isensee,A.R. + | ++Variability of Aquatic Model Ecosystem-Derived Data + | ++Int. J. Environ. Stud.10:35-41 + | ++1976 + | +
# Review of the substance_table
+species <-
+ readr::read_delim(
+ file = normalizePath(
+ path = file.path(
+ database_folder,
+ "validation",
+ "species.txt"
+ ),
+ ),
+ show_col_types = FALSE,
+ delim = "|"
+
+ )
+
+kable(
+ species %>%
+ head(5),
+ format = "html", digits = 2
+) %>%
+kable_styling("striped", full_width = TRUE) %>%
+scroll_box(width = "700px", height = "300px")| +species_number + | ++common_name + | ++latin_name + | ++kingdom + | ++phylum_division + | ++subphylum_div + | ++superclass + | ++class + | ++tax_order + | ++family + | ++genus + | ++species + | ++subspecies + | ++variety + | ++ecotox_group + | ++ncbi_taxid + | +
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| +31 + | ++Green Algae + | ++Chlorella ovalis + | ++Plantae + | ++Chlorophyta + | ++NA + | ++NA + | ++Chlorophyceae + | ++Chlorococcales + | ++Oocystaceae + | ++Chlorella + | ++ovalis + | ++NA + | ++NA + | ++Algae + | ++3071 + | +
| +47 + | ++Green Algae + | ++Scenedesmus sp. + | ++Plantae + | ++Chlorophyta + | ++NA + | ++NA + | ++Chlorophyceae + | ++Chlorococcales + | ++Scenedesmaceae + | ++Scenedesmus + | ++sp. + | ++NA + | ++NA + | ++Algae + | ++3087 + | +
| +147 + | ++Green Algae + | ++Spirogyra crassa + | ++Plantae + | ++Chlorophyta + | ++NA + | ++NA + | ++Chlorophyceae + | ++Zygnematales + | ++Zygnemataceae + | ++Spirogyra + | ++crassa + | ++NA + | ++NA + | ++Algae + | ++NA + | +
| +298 + | ++Green Algae + | ++Dunaliella tertiolecta + | ++Plantae + | ++Chlorophyta + | ++NA + | ++NA + | ++Chlorophyceae + | ++Volvocales + | ++Dunaliellaceae + | ++Dunaliella + | ++tertiolecta + | ++NA + | ++NA + | ++Standard Test SpeciesAlgae + | ++3047 + | +
| +300 + | ++Green Algae + | ++Chlorococcum sp. + | ++Plantae + | ++Chlorophyta + | ++NA + | ++NA + | ++Chlorophyceae + | ++Chlorococcales + | ++Chlorococcaceae + | ++Chlorococcum + | ++sp. + | ++NA + | ++NA + | ++Algae + | ++44649 + | +
4.5 Preparation of the database environment and initialisation of the project folder
+In the first step, the function create_project creates the database project and
+initializes the database and project folders:
-
+
Load the ASCII files.
+Create a file
chemical_properties.csvbased o thechemicals.txttable. +If this table exists, it is loaded.
+Store the initial database project in
project.Rdatain the database folder.
+Store the initial projcet in
initial_project.Rdatain the project folder.
+
4.5.1 Parameters
+-
+
initialise_database_project(TRUE/FALSE): Creates the basic database project +from the current ASCII files in the database folder and (if not existing) stores +thechemical_properties.csvin the database folder.
+initialise_project(TRUE/FALSE): Stores theREcoToxenvironment in an initial +Rdataobject namedinitial_project.Rdatain the project folder.
+load_default(TRUE/FALSE): Loads an existing basic database project from the +database folder and stores it in the project.
+
chemical_properties.csv: This tables contains the internal chemical ID cas_number (i.e.
+the cas number in integer format) and related user-curated metadata (e.g.,
+chemical identifiers such as InChIKey, or PubChem CIDs) and chemical property
+data (i.e. log S values). It will be re-used and extended in future
+analyses to minimize curation efforts. If this file exists, it will be loaded
+to the project environment. Because the chemicals.txt table only contains CAS numbers
+in integer format, a regular CAS number is added (e.g., 1912-24-9 for 1912249).
project <- REcoTox::create_project(database_path = database_folder,
+ project_path,
+ initalise_database_project = TRUE,
+ initalise_project = TRUE,
+ load_default = FALSE)4.6 Preparation of the initial project
+In the second step, utilizing the function prepare_data, the tables’ test,
+results, species, chemicals, and references are joined IDs test_id,
+cas_number, species_number, and reference_number. The initial environment
+is stored in the file ìnitial_project.Rdata in the project folder.
+This file will be the same for all analyses related to a database revision.
+Thus, it could be copied from another project to avoid rerunning initial steps.
4.6.1 Parameters
+-
+
project: Name of the project environment.
+load_initial_project(TRUE/FALSE): Loads theinitial_project.Rdataof the +project folder.
+new_project_path: Theinitial_project.Rdatacontains the project folder +path where it was initially created. For example, in case of moving the project +folder or if theinitial_project.Rdatawas copied from another folder, it is +required to set a new project path.
+save_project(TRUE/FALSE): Save theinitial_project.Rdata. For example, +in case, the project folder was renewed.
+
project <- REcoTox::prepare_data(project = project,
+ load_initial_project = FALSE,
+ new_project_path = NA,
+ save_project = TRUE
+)4.7 Processing the data
+In the third step, the function process_data reads the following settings to
+query the database accordingly.
A list of relevant endpoints (e.g., EC50) and all relevant species are generated
+and exported to the project folder for review.
+The two files are ecotoxgroup_endpoint_selection.csv and
+ecotoxgroup_species_selection.csv. The review could be performed in any
+spreadsheet program. The data must be stored in the comma delimited format!
The former table contains a field include_endpoint, this controls the inclusion
+of each endpoint by setting the value to 0 or 1 (0 = not included, 1 = included).
+Other values are not accepted, and the import of the file in the next processing
+step will be declined. The value 0 is the default, and thus, the endpoints to
+be included should be marked with 1.
The latter table contains a field include_species, this controls the inclusion of each species by setting the value to 0 or 1 (0 = not included, 1 = included). Depending on the settings of species_selection, the preset is different:
+-
+
include_speciesis set to 1
+include_speciesis set to 0
+include_speciesis set to 1 for standard test species and set to 0 for other species
+
Review and edit the tables in a preferred spreadsheet program. If changed,
+save the changes in the same file. The separator must be comma.
In this step, the database is queried to select the datasets related to the +goals of the analysis.
+The queries can be controlled by the following parameters:
+-
+
dosing_group: Specifies the compartment to which the dosing is referenced (so far only “water_concentration”, i.e. result is mg/L)
+duration_d: Duration of the exposure in days (e.g.d,dph,dpf)
+duration_h: Duration of the exposure in hours (e.g.h,ht,hph,hpf,hbf,hv)
+duration_m: Duration of the exposure in minutes (e.g.mi)
+ecotoxgroup: Species group (e.g.Algae,Crustacean,Fish)
+effects: Effect endpoints (e.g.MOR,GRO,DEV)
+habitat: Habitat of the ecotoxgroup (i.e.Non-Soil,Water,Soil)
+kingdoms: Specification of thealgaekingdoms (e.g.Chromista,Plantae,Monera)
+measurements: Specification of specific measurements
+min_h: Minimum duration of the experiment in hours
+max_h: Maximum duration of the experiment in hours
+min_d: Minimum duration of the experiment in days
+max_d: Maximum duration of the experiment in days
+min_m: Minimum duration of the experiment in minutes
+max_m: Maximum duration of the experiment in minutes
+species_seleciton: Selection of species (i.e.all,manual,standard_test_species)
+
Where all selects all species of an ecotoxgroup, manual expects manual selection in
+the files mentioned above and standard_test_species selects only species marked as
+standardized species.
4.7.1 Filtering the data
+In the processing step 1, the data in the database is filtered based on the settings +to extract relevant data of the database.
+
+# set the parameters
+dosing_group = "water_concentration" # i.e. mg/L (only available group in this version)
+duration_d = c("d", "dph", "dpf")
+duration_h = c("h", "ht", "hph", "hpf", "hbf", "hv")
+duration_m = "mi"
+ecotoxgroup = "Algae" # c("Algae", "Crustacean", "Fish")
+effects = c("MOR", "GRO", "POP", "REP", "MPH", "DEV") # Algae/Fish
+#effects = c("MOR", "GRO", "POP", "REP", "MPH", "DEV", "ITX") # Crustacean
+habitat = "Water" #c("Non-Soil","Water","Soil")
+kingdoms = NA # vector of specific algae kingdoms: c("Chromista","Plantae","Monera")
+measurements = NA # vector of specific measurements
+min_h = 0
+min_d = 0
+max_h = 120
+max_d = 5
+min_m = 0
+max_m = 7200
+species_selection = "all" # c("all", "manual", "standard_test_species")
+
+# run the processing step
+project <- REcoTox::process_data(project,
+ dosing_group = dosing_group,
+ duration_d = duration_h,
+ duration_h = duration_h,
+ duration_m = duration_m,
+ ecotoxgroup = ecotoxgroup,
+ effects = effects,
+ habitat = habitat,
+ kingdoms = kingdoms,
+ measurements = measurements,
+ max_d = max_d,
+ min_d = min_d,
+ max_h = max_h,
+ min_h = min_h,
+ max_m = max_m,
+ min_m = min_m,
+ remove_formulation = FALSE,
+ save_project_steps = FALSE,
+ species_selection = species_selection
+)
+This step stores two files in the project_folder, ecotoxgroup_species_selection.csv
+and ecotoxgroup_endpoint_selection.csv. The first block of the file is related to the
+ecotoxgroup specified. The species selection file contains all species extracted
+for review and the endpoint the respective endpoints (e.g. EC50). To include
+species or endpoints, mark the data with 1, otherwise to exclude, mark with 0.
4.7.2 Filtering species and endpoints
+After review and saving the files, run the following command. This command +reads the files and the data is filtered accordingly.
+The units in the database are quite divergent and thus a unit conversion is performed
+to transform all units and values to mg/L. In case of mol related units,
+the transformation is automated so far the chemical and the molecular weight is
+already in the database. If not, the file ecotoxgroup_mol_weight.csv is exported
+to the project_folder.
project <- REcoTox::process_data(project, save_project_steps = FALSE)
+4.7.3 Unit conversion
+Review and edit the file ecotoxgroup_mol_weight.csv to add the molecular weight to
+the list. The ecotoxicity data is interactivitely enriched with chemical information
+(e.g. the average mass).
In best case with data linked to US EPA CompTox Chemicals Dashboard +for example by using the output of the batch search +according to Figure 1 and Figure 2.
+
+
+After update of the mol weight table, run the following command to finalise the +unit conversion step.
+project <- REcoTox::process_data(project, save_project_steps = FALSE)
+4.7.4 Chemical properties data and final processing
+The former processing step creates a file named ecotoxgroup_chemical_list.csv.
+Edit this list to include newly added compounds (imputation of phys.-chem.
+properties and metadata).
To score the quality of the data, the solubility domain of the result is +calculated. The calculation requires the experimental or predicted solubility +of the chemical.
+project <- REcoTox::process_data(project, save_project_steps = FALSE, update_chemicals = FALSE)
+The file ecotoxgroup_final_results.csv is stored in the project_folder.
+It contains the results of the processing in the long pivot format.
4.7.5 Preparation of the wide pivot table with the aggregated ecotoxicity information
+For final processing and to aggregate the data in the wide pivot format, +run the following final step.
+project <- REcoTox::aggregate_results(project = project, quantile = 0.05)
+``{r view references, echo = TRUE, eval = TRUE, message = TRUE} +# Review of the substance_table +references <- +readr::read_delim( +file = normalizePath( +path = file.path( +database_folder, +“validation”, +“references.txt” +), +), +show_col_types = FALSE, +delim = “|”
+)kable( +references %>% +head(5), +format = “html”, digits = 2 +) %>% +kable_styling(“striped”, full_width = TRUE) %>% +scroll_box(width = “700px”, height = “300px”)
+
+An example of the final data:
+
+<div style="border: 1px solid #ddd; padding: 0px; overflow-y: scroll; height:300px; overflow-x: scroll; width:700px; "><table class="table table-striped" style="margin-left: auto; margin-right: auto;">
+<tbody>
+ <tr>
+
+ </tr>
+</tbody>
+</table></div>
+
+\newpage
+
+# SessionInfo
+
+
+```r
+sessionInfo()## R version 4.3.1 (2023-06-16)
+## Platform: x86_64-pc-linux-gnu (64-bit)
+## Running under: Ubuntu 23.10
+##
+## Matrix products: default
+## BLAS: /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3
+## LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.23.so; LAPACK version 3.11.0
+##
+## locale:
+## [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
+## [3] LC_TIME=de_DE.UTF-8 LC_COLLATE=C
+## [5] LC_MONETARY=de_DE.UTF-8 LC_MESSAGES=en_US.UTF-8
+## [7] LC_PAPER=de_DE.UTF-8 LC_NAME=C
+## [9] LC_ADDRESS=C LC_TELEPHONE=C
+## [11] LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C
+##
+## time zone: Europe/Berlin
+## tzcode source: system (glibc)
+##
+## attached base packages:
+## [1] stats graphics grDevices utils datasets methods base
+##
+## other attached packages:
+## [1] lubridate_1.9.3 forcats_1.0.0 stringr_1.5.0 dplyr_1.1.3
+## [5] purrr_1.0.2 readr_2.1.4 tidyr_1.3.0 tibble_3.2.1
+## [9] ggplot2_3.4.4 tidyverse_2.0.0 kableExtra_1.3.4 desc_1.4.2
+## [13] BiocStyle_2.28.1 REcoTox_0.4.1
+##
+## loaded via a namespace (and not attached):
+## [1] tidyselect_1.2.0 viridisLite_0.4.2 fastmap_1.1.1
+## [4] promises_1.2.1 digest_0.6.33 timechange_0.2.0
+## [7] mime_0.12 lifecycle_1.0.3 ellipsis_0.3.2
+## [10] processx_3.8.2 magrittr_2.0.3 compiler_4.3.1
+## [13] rlang_1.1.1 sass_0.4.7 progress_1.2.2
+## [16] tools_4.3.1 utf8_1.2.4 yaml_2.3.7
+## [19] data.table_1.14.8 knitr_1.44 prettyunits_1.2.0
+## [22] htmlwidgets_1.6.2 bit_4.0.5 pkgbuild_1.4.2
+## [25] xml2_1.3.5 pkgload_1.3.3 miniUI_0.1.1.1
+## [28] withr_2.5.1 grid_4.3.1 fansi_1.0.5
+## [31] roxygen2_7.2.3 urlchecker_1.0.1 profvis_0.3.8
+## [34] xtable_1.8-4 colorspace_2.1-0 data.tree_1.0.0
+## [37] scales_1.2.1 cli_3.6.1 rmarkdown_2.25
+## [40] crayon_1.5.2 generics_0.1.3 remotes_2.4.2.1
+## [43] webchem_1.3.0 rstudioapi_0.15.0 httr_1.4.7
+## [46] tzdb_0.4.0 sessioninfo_1.2.2 cachem_1.0.8
+## [49] parallel_4.3.1 rvest_1.0.3 BiocManager_1.30.22
+## [52] vctrs_0.6.4 devtools_2.4.5 webshot_0.5.5
+## [55] jsonlite_1.8.7 bookdown_0.36 callr_3.7.3
+## [58] hms_1.1.3 bit64_4.0.5 systemfonts_1.0.5
+## [61] jquerylib_0.1.4 glue_1.6.2 ps_1.7.5
+## [64] stringi_1.7.12 gtable_0.3.4 later_1.3.1
+## [67] EnvStats_2.8.1 munsell_0.5.0 pillar_1.9.0
+## [70] htmltools_0.5.6.1 R6_2.5.1 Rdpack_2.5
+## [73] rprojroot_2.0.3 vroom_1.6.4 evaluate_0.22
+## [76] shiny_1.7.5.1 highr_0.10 rbibutils_2.2.15
+## [79] memoise_2.0.1 httpuv_1.6.12 bslib_0.5.1
+## [82] Rcpp_1.0.11 svglite_2.1.2 xfun_0.40
+## [85] fs_1.6.3 usethis_2.2.2 pkgconfig_2.0.3