Add ability for Cohort Definitions and concept set definitions to update based on vocabulary version

[gowthamrao]

Expected behavior

Atlas updates concept id based on referenced vocabulary

Actual behavior

Atlas keep concept id metadata in cohort json. it does not know that vocabulary has changed a previously standard concept id to non standard

Steps to reproduce behavior

Build a concept set using json below, you will see below

now import the following concept ids in a webApi that has February 2024 version vocabulary. A lot of vocabulary changes occurred in 2024 with many previously standard conceptIds becoming non standard. Also many conceptIds changed from previously condition domain to measurement domain.

192568,200959,434298,442182,4315672

They are now NOT standard (in screenshot below i am intentionally showing the same conceptId twice . Here the non standard information is appended during the import step above to the original concept set expression). Note the current metadata for each conceptId is different from what was previously stored in cohort json.

{
  "items": [
    {
      "concept": {
        "CONCEPT_CLASS_ID": "Clinical Finding",
        "CONCEPT_CODE": "94519005",
        "CONCEPT_ID": 4315672,
        "CONCEPT_NAME": "Secondary malignant neoplasm of retroperitoneal lymph nodes",
        "DOMAIN_ID": "Condition",
        "INVALID_REASON": "V",
        "INVALID_REASON_CAPTION": "Valid",
        "STANDARD_CONCEPT": "S",
        "STANDARD_CONCEPT_CAPTION": "Standard",
        "VOCABULARY_ID": "SNOMED",
        "VALID_START_DATE": "",
        "VALID_END_DATE": ""
      },
      "isExcluded": false,
      "includeDescendants": true,
      "includeMapped": false
    },
    {
      "concept": {
        "CONCEPT_CLASS_ID": "Clinical Finding",
        "CONCEPT_CODE": "94350006",
        "CONCEPT_ID": 200959,
        "CONCEPT_NAME": "Secondary malignant neoplasm of intrapelvic lymph nodes",
        "DOMAIN_ID": "Condition",
        "INVALID_REASON": "V",
        "INVALID_REASON_CAPTION": "Valid",
        "STANDARD_CONCEPT": "S",
        "STANDARD_CONCEPT_CAPTION": "Standard",
        "VOCABULARY_ID": "SNOMED",
        "VALID_START_DATE": "",
        "VALID_END_DATE": ""
      },
      "isExcluded": false,
      "includeDescendants": true,
      "includeMapped": false
    },
    {
      "concept": {
        "CONCEPT_CLASS_ID": "Clinical Finding",
        "CONCEPT_CODE": "94395004",
        "CONCEPT_ID": 442182,
        "CONCEPT_NAME": "Secondary malignant neoplasm of lymph nodes of lower limb",
        "DOMAIN_ID": "Condition",
        "INVALID_REASON": "V",
        "INVALID_REASON_CAPTION": "Valid",
        "STANDARD_CONCEPT": "S",
        "STANDARD_CONCEPT_CAPTION": "Standard",
        "VOCABULARY_ID": "SNOMED",
        "VALID_START_DATE": "",
        "VALID_END_DATE": ""
      },
      "isExcluded": false,
      "includeDescendants": true,
      "includeMapped": false
    },
    {
      "concept": {
        "CONCEPT_CLASS_ID": "Clinical Finding",
        "CONCEPT_CODE": "94347008",
        "CONCEPT_ID": 192568,
        "CONCEPT_NAME": "Secondary malignant neoplasm of intra-abdominal lymph nodes",
        "DOMAIN_ID": "Condition",
        "INVALID_REASON": "V",
        "INVALID_REASON_CAPTION": "Valid",
        "STANDARD_CONCEPT": "S",
        "STANDARD_CONCEPT_CAPTION": "Standard",
        "VOCABULARY_ID": "SNOMED",
        "VALID_START_DATE": "",
        "VALID_END_DATE": ""
      },
      "isExcluded": false,
      "includeDescendants": true,
      "includeMapped": false
    },
    {
      "concept": {
        "CONCEPT_CLASS_ID": "Clinical Finding",
        "CONCEPT_CODE": "94398002",
        "CONCEPT_ID": 434298,
        "CONCEPT_NAME": "Secondary malignant neoplasm of lymph nodes of upper limb",
        "DOMAIN_ID": "Condition",
        "INVALID_REASON": "V",
        "INVALID_REASON_CAPTION": "Valid",
        "STANDARD_CONCEPT": "S",
        "STANDARD_CONCEPT_CAPTION": "Standard",
        "VOCABULARY_ID": "SNOMED",
        "VALID_START_DATE": "",
        "VALID_END_DATE": ""
      },
      "isExcluded": false,
      "includeDescendants": true,
      "includeMapped": false
    }
  ]
}

[chrisknoll]

In Circe 2.0 this information will probably be removed from the concept set (as it doesn't impact anything about generating cohort queries) and instead it will be a secondary lookup to resolve the referenced concepts separately.

[chrisknoll]

Sorry, didn't meant o close. There may be something we can do in the 2.x line of WebAPI to 'reload' referenced concepts in a concept set.

[gowthamrao]

Thank you for reopening. Yes, the ask is to update what the UI displays. With UI displaying (in this case) wrong information, the user has no way of knowing if their cohort definition would be impacted.

In this context - Atlas UI would propagate the illusion that the cohort definition is not impacted by vocabulary change.

The ask is for a button that ensures the content of chosen vocabulary version is shown instead of what was in cohort json.