Lots of NewConnectionErrors when grounding with SPIRES

[justaddcoffee]

I'm doing a few thousand calls to SPIRES to ground some text to Mondo. I'm seeing a LOT of these errors:

ERROR:root:Encountered error when normalizing OMIM:MTHU003333: HTTPSConnectionPool(host='nodenormalization-sri.renci.org', port=443): Max retries exceeded with url: /1.4/get_normalized_nodes?curie=OMIM%3AMTHU003333&conflate=false (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1782f88d0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
 76%|███████▌  | 4013/5267 [20:47:34<3:39:34, 10.51s/it]ERROR:root:Encountered error when normalizing OMIM:606681: HTTPSConnectionPool(host='nodenormalization-sri.renci.org', port=443): Max retries exceeded with url: /1.4/get_normalized_nodes?curie=OMIM%3A606681&conflate=false (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x28f3d72d0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
 76%|███████▌  | 4014/5267 [20:47:44<3:41:13, 10.59s/it]ERROR:root:Encountered error when normalizing OMIM:602054: HTTPSConnectionPool(host='nodenormalization-sri.renci.org', port=443): Max retries exceeded with url: /1.4/get_normalized_nodes?curie=OMIM%3A602054&conflate=false (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x28f3f3350>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
ERROR:root:Encountered error when normalizing OMIM:188400: HTTPSConnectionPool(host='nodenormalization-sri.renci.org', port=443): Max retries exceeded with url: /1.4/get_normalized_nodes?curie=OMIM%3A188400&conflate=false (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x28f3f14d0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))
 76%|███████▌  | 4015/5267 [20:48:53<9:36:38, 27.63s/it]ERROR:root:Encountered error when normalizing OMIM:601255: HTTPSConnectionPool(host='nodenormalization-sri.renci.org', port=443): Max retries exceeded with url: /1.4/get_normalized_nodes?curie=OMIM%3A601255&conflate=false (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x28f3fc8d0>: Failed to establish a new connection: [Errno 8] nodename nor servname provided, or not known'))

Here's the code I'm running to do the grounding:

from warnings import warn
import logging
from ontogpt.templates.all_disease_grounding import ExtractionResult
from typing import Tuple
from ontogpt.engines.spires_engine import SPIRESEngine
from ontogpt.io.template_loader import get_template_details
import time

# Retry decorator
def retry_on_failure(max_retries=2, delay=1):
    def decorator(func):
        def wrapper(*args, **kwargs):
            retries = 0
            while retries <= max_retries:
                try:
                    return func(*args, **kwargs)
                except (TimeoutError, Exception) as e:  # Handle appropriate exceptions
                    retries += 1
                    if retries > max_retries:
                        logging.error(f"Max retries reached: {e}")
                        raise
                    logging.warning(f"Error occurred: {e}, retrying {retries}/{max_retries}...")
                    time.sleep(delay)
        return wrapper
    return decorator

@retry_on_failure(max_retries=2, delay=2)
def ground_diagnosis_text_to_mondo(
    ke: SPIRESEngine,
    differential_diagnosis: str,
) -> Tuple[ExtractionResult, str, SPIRESEngine]:
    """Run the multilingual analysis with retry on failures."""

    try:
        # Call the extract function here to ground the answer to OMIM (using MONDO, etc)
        extraction = ke.extract_from_text(text=differential_diagnosis)
    except TimeoutError as e:
        logging.error(f"Timeout occurred: {e}")
        raise

    predictions = extraction.named_entities
    pred_ids = []
    pred_names = []
    
    for pred in predictions:
        pred_ids.append(pred.id)
        pred_names.append(pred.label)

    if len(pred_ids) == 0:
        warn(f"No grounded IDs found")
        return None

    # Log the result
    logging.info("input file name" "\tpredicted diagnosis ids\tpredicted diagnosis names\n")
    logging.info(
        f'\t{"|".join(pred_ids)}' f'\t{"|".join(pred_names)}\n'
    )

    return extraction

# make a SPIRES object
template = "all_disease_grounding"
template_details = get_template_details(template=template)

model = "gpt-4-turbo"
temperature = 1.0
    
ke = SPIRESEngine(
    template_details=template_details,
    model=model,
    temperature=temperature
)

[caufieldjh]

Jim Balhoff said on Slack (obo-community/ubergraph) that UNC was having some network issues between this past Friday and Saturday, so that may have been related to the inability to access nodenormalization-sri.renci.org.