fitting failing due to too many open files

[jgninterline]

Hi,

I have bespokefit 0.1.3 running on ubuntu 20.04, installed via conda. I'm currently running in production mode and submitting via command line. I have not tried using the API yet. It works great on most molecules, however I've found a handful that die with the error:

[✓] fragmentation successful
[x] qc-generation failed

[null, {"type": "OSError", "message": "[Errno 24] Too many open files", "traceback": "Traceback (most recent call last):\n File
"/miniconda3/envs/bespoke/lib/python3.9/site-packages/celery/app/trace.py", line 451, in trace_task\n File
"/miniconda3/envs/bespoke/lib/python3.9/site-packages/celery/app/trace.py", line 734, in protected_call\n File
"/miniconda3/envs/bespoke/lib/python3.9/site-packages/openff/bespokefit/executor/services/qcgenerator/worker.py",
line 133, in compute_torsion_drive\n input_schema, "torsiondrive", raise_error=True, local_options=_task_config()\n File
"/miniconda3/envs/bespoke/lib/python3.9/site-packages/openff/bespokefit/executor/services/qcgenerator/worker.py",
line 43, in _task_config\n (psutil.virtual_memory().total / (1024**3))\n File
"/miniconda3/envs/bespoke/lib/python3.9/site-packages/psutil/init.py", line 1968, in virtual_memory\n File
"/miniconda3/envs/bespoke/lib/python3.9/site-packages/psutil/_pslinux.py", line 415, in virtual_memory\n File
"~/miniconda3/envs/bespoke/lib/python3.9/site-packages/psutil/_common.py", line 728, in open_binary\nOSError: [Errno
24] Too many open files: '/proc/meminfo'\n"}]

I tried changing the allowed number of files to 4096 using ulimit but I still get the error. Googling around, it looks like this is a known issue with xtb writing out too many intermediate files, which they fixed with a verbosity setting. (https://github.com/grimme-lab/xtb-python/issues/73 and implemented in libxtb.py and interface.py in the xtb-python distribution) Is there a way to set the verbosity from the command line or do I need to dive into the API?

Thanks!

[j-wags]

I don't think that xtb verbosity is exposed as a direct option in the CLI. Shooting from the hip... you'll probably need to make a workflow.yaml, where the default_qc_spec has verbosity:"muted" in its keywords. Here's an example of the section in the workflow file to look for:

openff-bespokefit/openff/bespokefit/data/test/miscellaneous/fast-psi4-workflow.yaml

Lines 1 to 15 in c6395ea

	default_qc_specs:
	- basis: STO-3G
	implicit_solvent: null
	keywords: null
	maxiter: 200
	method: hf
	program: psi4
	scf_properties:
	- dipole
	- quadrupole
	- wiberg_lowdin_indices
	- mayer_indices
	spec_description: Standard OpenFF optimization quantum chemistry specification.
	spec_name: default
	store_wavefunction: none

You can try making a workflow file by hand (yaml or json, I think), but it's probably better to use the API to reduce trial and error - Here's the page on that

[jthorton]

I think it would probably be best to have bespokefit automatically add this to the keywords if its not already present Ill try and get a PR for this as well.

Edit: It seems bespokefit is not currently passing on any keywords from the config to qcengine this will need to be changed in future see these lines for the calculation setup. This also means many of the settings in the config such as the scf_properties are currently redundant.

[jgninterline]

Thanks for all your help!

So if I'm following, I should be able to provide a modified workflow file with something like the following in it to change the verbosity setting:

default_qc_specs:

• basis: DZVP
  implicit_solvent: null
  keywords:
  verbosity: 'muted'
  maxiter: 200
  etc.

but, at the moment, those settings are not actually passed to xtb. (any of the settings in that file or just the keywords?) Is that correct? Bespokefit accepts my yaml with that syntax, but I'm waiting on other fits to finish before it can run the ones with errors.

Is there somewhere in the code I could hardcode the verbosity setting until we can figure out a more permanent solution?

[jthorton]

Correct the keyword settings are not currently passed on, only the method, basis and program are actually used. You could try the branch from #239 to confirm this fixes the issue if you'd like to get fits running before we merge it.