pypolychord sometimes hangs when generating live points.

[tillahoffmann]

pypolychord occasionally hangs when generating live points (see below for an example output). The CPU utilization is zero when the problem occurs which makes it seem like pypolychord "gave up" trying to generate live points.

Edit: this seems to work fine if I package up the code in an ubuntu docker container running either on ARM or AMD64.

What I've tried, observations, and configuration

• Run with feedback = 3 to get additional debugging output, but I couldn't discern anything from the console output.
• Tried different versions (the issues seems to happen more frequently with 1.20.1 than 1.18.1).
• I'm running on an M1 Mac (OS version 12.4).
• python version is 3.8.10.
• numpy is at version 1.21.2.
• Made sure evaluating the likelihood on prior samples returned sensible results using the code below.

# Check that at least 10% of prior samples have log_likelihood > logzero and all are finite.
log_likelihoods = []
for _ in range(10_000):
    # Sample from the hypercube.
    x = np.random.uniform(size=model.size)
    # Convert to prior space.
    y = model.sample_params_from_vector(x)
    # Evaluate the log likelihood given the prior sample.
    log_likelihood, _ = model.evaluate_log_likelihood_from_vector(y, data)
    assert np.isfinite(log_likelihood), log_likelihood
    log_likelihoods.append(log_likelihood)

log_likelihoods = np.asarray(log_likelihoods)
assert np.mean(log_likelihoods > settings.logzero) > .1

Example output

PolyChord: Next Generation Nested Sampling
copyright: Will Handley, Mike Hobson & Anthony Lasenby
  version: 1.20.1
  release: 1st June 2021
    email: [email protected]

Run Settings
nlive    :    1025
nDims    :      41
nDerived :       0
Doing Clustering
Synchronous parallelisation
Generating equally weighted posteriors
Generating weighted posteriors
Clustering on posteriors

generating live points

  8% |********                                                                                            |

[williamjameshandley]

If the likelihood works on a linux install, then this is likely a mac issue. Have you tried the CMake install system #76?