2022-02-02 10:18:08,274 UTC [7079] INFO esmvalcore._main:473 ______________________________________________________________________ _____ ____ __ ____ __ _ _____ _ | ____/ ___|| \/ \ \ / /_ _| |_ _|__ ___ | | | _| \___ \| |\/| |\ \ / / _` | | | |/ _ \ / _ \| | | |___ ___) | | | | \ V / (_| | | | | (_) | (_) | | |_____|____/|_| |_| \_/ \__,_|_| |_|\___/ \___/|_| ______________________________________________________________________ ESMValTool - Earth System Model Evaluation Tool. http://www.esmvaltool.org CORE DEVELOPMENT TEAM AND CONTACTS: Birgit Hassler (Co-PI; DLR, Germany - birgit.hassler@dlr.de) Alistair Sellar (Co-PI; Met Office, UK - alistair.sellar@metoffice.gov.uk) Bouwe Andela (Netherlands eScience Center, The Netherlands - b.andela@esciencecenter.nl) Lee de Mora (PML, UK - ledm@pml.ac.uk) Niels Drost (Netherlands eScience Center, The Netherlands - n.drost@esciencecenter.nl) Veronika Eyring (DLR, Germany - veronika.eyring@dlr.de) Bettina Gier (UBremen, Germany - gier@uni-bremen.de) Remi Kazeroni (DLR, Germany - remi.kazeroni@dlr.de) Nikolay Koldunov (AWI, Germany - nikolay.koldunov@awi.de) Axel Lauer (DLR, Germany - axel.lauer@dlr.de) Saskia Loosveldt-Tomas (BSC, Spain - saskia.loosveldt@bsc.es) Ruth Lorenz (ETH Zurich, Switzerland - ruth.lorenz@env.ethz.ch) Benjamin Mueller (LMU, Germany - b.mueller@iggf.geo.uni-muenchen.de) Valeriu Predoi (URead, UK - valeriu.predoi@ncas.ac.uk) Mattia Righi (DLR, Germany - mattia.righi@dlr.de) Manuel Schlund (DLR, Germany - manuel.schlund@dlr.de) Breixo Solino Fernandez (DLR, Germany - breixo.solinofernandez@dlr.de) Javier Vegas-Regidor (BSC, Spain - javier.vegas@bsc.es) Klaus Zimmermann (SMHI, Sweden - klaus.zimmermann@smhi.se) For further help, please read the documentation at http://docs.esmvaltool.org. Have fun! 2022-02-02 10:18:08,276 UTC [7079] INFO esmvalcore._main:474 Package versions 2022-02-02 10:18:08,276 UTC [7079] INFO esmvalcore._main:475 ---------------- 2022-02-02 10:18:08,276 UTC [7079] INFO esmvalcore._main:476 ESMValCore: 2.4.0 2022-02-02 10:18:08,277 UTC [7079] INFO esmvalcore._main:478 ESMValTool: 2.4.0 2022-02-02 10:18:08,277 UTC [7079] INFO esmvalcore._main:479 ---------------- 2022-02-02 10:18:08,277 UTC [7079] INFO esmvalcore._main:480 Using config file /home/b/b309141/config/fail.yml 2022-02-02 10:18:08,277 UTC [7079] INFO esmvalcore._main:481 Writing program log files to: /scratch/b/b309141/work_v2/recipe_test_reference_levels_20220202_101806/run/main_log.txt /scratch/b/b309141/work_v2/recipe_test_reference_levels_20220202_101806/run/main_log_debug.txt 2022-02-02 10:18:08,277 UTC [7079] INFO esmvalcore._config._esgf_pyclient:118 Using default ESGF configuration, configuration file /home/b/b309141/.esmvaltool/esgf-pyclient.yml not present. 2022-02-02 10:18:08,278 UTC [7079] WARNING esmvalcore._config._esgf_pyclient:165 ESGF credentials missing, only data that is accessible without logging in will be available. See https://esgf.github.io/esgf-user-support/user_guide.html for instructions on how to create an account if you do not have one yet. Next, configure your system so esmvaltool can use your credentials. This can be done using the keyring package, or you can just enter them in /home/b/b309141/.esmvaltool/esgf-pyclient.yml. keyring ======= First install the keyring package (requires a supported backend, see https://pypi.org/project/keyring/): $ pip install keyring Next, set your username and password by running the commands: $ keyring set ESGF hostname $ keyring set ESGF username $ keyring set ESGF password To check that you entered your credentials correctly, run: $ keyring get ESGF hostname $ keyring get ESGF username $ keyring get ESGF password configuration file ================== You can store the hostname, username, and password or your OpenID account in a plain text in the file /home/b/b309141/.esmvaltool/esgf-pyclient.yml like this: logon: hostname: "your-hostname" username: "your-username" password: "your-password" or your can configure an interactive log in: logon: interactive: true Note that storing your password in plain text in the configuration file is less secure. On shared systems, make sure the permissions of the file are set so only you can read it, i.e. $ ls -l /home/b/b309141/.esmvaltool/esgf-pyclient.yml shows permissions -rw-------. 2022-02-02 10:18:08,324 UTC [7079] INFO esmvalcore._main:86 Starting the Earth System Model Evaluation Tool at time: 2022-02-02 10:18:08 UTC 2022-02-02 10:18:08,325 UTC [7079] INFO esmvalcore._main:90 ---------------------------------------------------------------------- 2022-02-02 10:18:08,325 UTC [7079] INFO esmvalcore._main:91 RECIPE = /home/b/b309141/recipes/recipe_test_reference_levels.yml 2022-02-02 10:18:08,325 UTC [7079] INFO esmvalcore._main:92 RUNDIR = /scratch/b/b309141/work_v2/recipe_test_reference_levels_20220202_101806/run 2022-02-02 10:18:08,325 UTC [7079] INFO esmvalcore._main:93 WORKDIR = /scratch/b/b309141/work_v2/recipe_test_reference_levels_20220202_101806/work 2022-02-02 10:18:08,325 UTC [7079] INFO esmvalcore._main:94 PREPROCDIR = /scratch/b/b309141/work_v2/recipe_test_reference_levels_20220202_101806/preproc 2022-02-02 10:18:08,326 UTC [7079] INFO esmvalcore._main:95 PLOTDIR = /scratch/b/b309141/work_v2/recipe_test_reference_levels_20220202_101806/plots 2022-02-02 10:18:08,326 UTC [7079] INFO esmvalcore._main:96 ---------------------------------------------------------------------- 2022-02-02 10:18:08,326 UTC [7079] INFO esmvalcore._main:100 Running tasks using at most 8 processes 2022-02-02 10:18:08,327 UTC [7079] INFO esmvalcore._main:102 If your system hangs during execution, it may not have enough memory for keeping this number of tasks in memory. 2022-02-02 10:18:08,327 UTC [7079] INFO esmvalcore._main:105 If you experience memory problems, try reducing 'max_parallel_tasks' in your user configuration file. 2022-02-02 10:18:08,345 UTC [7079] DEBUG esmvalcore._recipe_checks:47 Checking recipe against schema /mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/recipe_schema.yml 2022-02-02 10:18:08,382 UTC [7079] DEBUG esmvalcore._recipe:1194 Retrieving diagnostics from recipe 2022-02-02 10:18:08,382 UTC [7079] DEBUG esmvalcore._recipe:1331 Populating list of variables for diagnostic diagnostic_1 2022-02-02 10:18:08,384 UTC [7079] DEBUG esmvalcore._config._config:48 Loading extra facets from /mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_config/extra_facets/cmip5-institutes.yml 2022-02-02 10:18:08,421 UTC [7079] DEBUG esmvalcore._config._config:48 Loading extra facets from /mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_config/extra_facets/cmip5-product.yml 2022-02-02 10:18:08,430 UTC [7079] INFO esmvalcore._recipe:1697 Creating tasks from recipe 2022-02-02 10:18:08,430 UTC [7079] INFO esmvalcore._recipe:1706 Creating tasks for diagnostic diagnostic_1 2022-02-02 10:18:08,430 UTC [7079] INFO esmvalcore._recipe:1677 Creating preprocessor task diagnostic_1/ta 2022-02-02 10:18:08,430 UTC [7079] INFO esmvalcore._recipe:1066 Creating preprocessor 'preprocessor_1' task for variable 'ta' 2022-02-02 10:18:08,434 UTC [7079] WARNING esmvalcore._data_finder:394 'default' rootpaths '/home/b/b309141/work/empty_path' set in config-user.yml do not exist 2022-02-02 10:18:08,434 UTC [7079] DEBUG esmvalcore._data_finder:20 Looking for files matching ['ta_Amon_bcc-csm1-1_historical_r1i1p1*.nc'] in [] 2022-02-02 10:18:08,434 UTC [7079] DEBUG esmvalcore.esgf._search:79 Searching for datasets on ESGF using facets={'project': 'CMIP5', 'model': 'BCC-CSM1.1', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'cmor_table': 'Amon', 'product': 'output1', 'variable': 'ta'} 2022-02-02 10:18:08,445 UTC [7079] DEBUG urllib3.connectionpool:228 Starting new HTTP connection (1): esgf-node.llnl.gov:80 2022-02-02 10:18:14,441 UTC [7079] DEBUG urllib3.connectionpool:456 http://esgf-node.llnl.gov:80 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=0&distrib=true&type=File&latest=True&project=CMIP5&model=BCC-CSM1.1&ensemble=r1i1p1&experiment=historical&cmor_table=Amon&product=output1&variable=ta&facets=%2A HTTP/1.1" 200 None 2022-02-02 10:18:14,860 UTC [7079] DEBUG urllib3.connectionpool:228 Starting new HTTP connection (1): esgf-node.llnl.gov:80 2022-02-02 10:18:15,242 UTC [7079] DEBUG urllib3.connectionpool:456 http://esgf-node.llnl.gov:80 "GET /esg-search/search?format=application%2Fsolr%2Bjson&limit=500&distrib=true&offset=0&type=File&latest=True&project=CMIP5&model=BCC-CSM1.1&ensemble=r1i1p1&experiment=historical&cmor_table=Amon&product=output1&variable=ta HTTP/1.1" 200 None 2022-02-02 10:18:15,465 UTC [7079] DEBUG esmvalcore.esgf._search:99 Found the following files matching facets {'project': 'CMIP5', 'model': 'BCC-CSM1.1', 'ensemble': 'r1i1p1', 'experiment': 'historical', 'cmor_table': 'Amon', 'product': 'output1', 'variable': 'ta'}: ESGFFile:cmip5/output1/BCC/bcc-csm1-1/historical/mon/atmos/Amon/r1i1p1/v1/ta_Amon_bcc-csm1-1_historical_r1i1p1_185001-201212.nc on hosts ['aims3.llnl.gov', 'esgf-data1.ceda.ac.uk', 'esgf2.dkrz.de'] 2022-02-02 10:18:15,468 UTC [7079] DEBUG esmvalcore.esgf._search:249 Selected files: ESGFFile:cmip5/output1/BCC/bcc-csm1-1/historical/mon/atmos/Amon/r1i1p1/v1/ta_Amon_bcc-csm1-1_historical_r1i1p1_185001-201212.nc on hosts ['aims3.llnl.gov', 'esgf-data1.ceda.ac.uk', 'esgf2.dkrz.de'] 2022-02-02 10:18:15,475 UTC [7079] DEBUG esmvalcore.preprocessor._io:115 Loading: /home/b/b309141/work/empty_path/cmip5/output1/BCC/bcc-csm1-1/historical/mon/atmos/Amon/r1i1p1/v1/ta_Amon_bcc-csm1-1_historical_r1i1p1_185001-201212.nc 2022-02-02 10:18:15,658 UTC [7079] INFO esmvalcore._task:127 Maximum memory used (estimate): 0.2 GB 2022-02-02 10:18:15,658 UTC [7079] INFO esmvalcore._task:129 Sampled every second. It may be inaccurate if short but high spikes in memory consumption occur. 2022-02-02 10:18:15,660 UTC [7079] ERROR esmvalcore._main:511 Program terminated abnormally, see stack trace below for more information: Traceback (most recent call last): File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_main.py", line 499, in run fire.Fire(ESMValTool()) File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/fire/core.py", line 141, in Fire component_trace = _Fire(component, args, parsed_flag_args, context, name) File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/fire/core.py", line 466, in _Fire component, remaining_args = _CallAndUpdateTrace( File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/fire/core.py", line 681, in _CallAndUpdateTrace component = fn(*varargs, **kwargs) File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_main.py", line 443, in run process_recipe(recipe_file=recipe, config_user=cfg) File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_main.py", line 121, in process_recipe recipe = read_recipe_file(recipe_file, config_user) File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 72, in read_recipe_file return Recipe(raw_recipe, File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 1135, in __init__ self.tasks = self.initialize_tasks() if initialize_tasks else None File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 1744, in initialize_tasks tasks = self._create_tasks() File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 1719, in _create_tasks new_tasks, failed = self._create_preprocessor_tasks( File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 1679, in _create_preprocessor_tasks task = _get_preprocessor_task( File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 1094, in _get_preprocessor_task task = _get_single_preprocessor_task( File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 949, in _get_single_preprocessor_task products = _get_preprocessor_products( File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 872, in _get_preprocessor_products _update_target_levels( File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/_recipe.py", line 165, in _update_target_levels settings['extract_levels']['levels'] = get_reference_levels( File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/preprocessor/_regrid.py", line 1011, in get_reference_levels cubes = load(filename, callback=concatenate_callback) File "/mnt/lustre02/work/bd0854/b309141/repos/ESMValCore/esmvalcore/preprocessor/_io.py", line 129, in load raw_cubes = iris.load_raw(file, callback=callback) File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/iris/__init__.py", line 416, in load_raw return _load_collection(uris, constraints, callback).cubes() File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/iris/__init__.py", line 270, in _load_collection result = _CubeFilterCollection.from_cubes(cubes, constraints) File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/iris/cube.py", line 115, in from_cubes for cube in cubes: File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/iris/__init__.py", line 255, in _generate_cubes for cube in iris.io.load_files(part_names, callback, constraints): File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/iris/io/__init__.py", line 189, in load_files all_file_paths = expand_filespecs(filenames) File "/mnt/lustre02/work/bd0854/b309141/mambaforge/envs/esm/lib/python3.9/site-packages/iris/io/__init__.py", line 170, in expand_filespecs raise IOError(msg) OSError: One or more of the files specified did not exist: * "/home/b/b309141/work/empty_path/cmip5/output1/BCC/bcc-csm1-1/historical/mon/atmos/Amon/r1i1p1/v1/ta_Amon_bcc-csm1-1_historical_r1i1p1_185001-201212.nc" didn't match any files 2022-02-02 10:18:15,672 UTC [7079] INFO esmvalcore._main:515 If you have a question or need help, please start a new discussion on https://github.com/ESMValGroup/ESMValTool/discussions If you suspect this is a bug, please open an issue on https://github.com/ESMValGroup/ESMValTool/issues To make it easier to find out what the problem is, please consider attaching the files run/recipe_*.yml and run/main_log_debug.txt from the output directory.