Add field configuration loading

Because issue field configuration in JIRA is highly dynamic and customizable,
Bugjira needs to be able to retrieve field configuration data from an external
source. This patch adds support for that operation, and also defines a very
minimal set of field configuration data. The classes in bugjira/field.py will
be heavily extended.

Once we can make Bugjira aware of the available fields in the associated JIRA
and Bugzilla backends, we will be able to add support for querying and updating
those field attributes in BugjiraIssue objects.

The module that loads the field configuration data is designed as a plugin. The
plugin included in the source code loads json from a local file when the Bugjira
instance is created. Another approach would be to load field configuration by
querying a JIRA instance directly, possibly dynamically updating the field info
periodically or even each time a BugjiraIssue's fields are accessed. These
alternate field info access approaches could be coded into an external library
that implements the plugin interface, thus allowing us to decouple the public
Bugjira code from any user's particular JIRA instance configuration.

README.md

@@ -8,6 +8,7 @@ Bugjira users can perform common operations (lookup, query, modify, etc) on both
 Configuration is provided with either a dict (see below) or a pathname to a config file containing a json dict (a sample config file is included in `contrib/bugjira.json`):
 
 ```python
+from bugjira.bugjira import Bugjira
 config = {
     "bugzilla": {
         "URL": "https://bugzilla.yourdomain.com",
@@ -48,3 +49,14 @@ or
 jira_api = bugjira_api.jira
 issue = jira_api.get_issue("FOO-123") # using the JIRA api object's get_issue
 ```
+
+## Field Configuration
+Users of the Bugjira library will be able to read and write field contents from `bugjira.Issue` objects uniformly whether the Issue represents a bugzilla bug (`bugjira.BugzillaIssue`) or a JIRA issue (`bugjira.JiraIssue`).
+
+Bugzilla's issue (bug) attributes are largely pre-defined. JIRA's issue attributes, in contrast, consist of a pre-defined set of attributes (e.g. "issuetype", "status", "assignee") and an arbitrarily large set of custom fields (with identifiers like "customfield_12345678").
+
+Consequently, the Bugjira library must rely on user-supplied configuration information to determine what fields are supported by the user's JIRA instance (and, to avoid hard-coding one but not the other, the user's Bugzilla instance). The data required to define fields is specified by the `BugzillaField` and `JiraField` classes in the `bugjira.field` module. Internally, Bugjira uses the `bugjira.field_factory` module as its source of field information.
+
+The `field_factory` module generates `BugzillaField` and `JiraField` objects using json retrieved from a plugin module loaded at runtime. The default plugin is defined by the included `bugjira.json_generator` module, which is specified in the config dict under the "json_generator_module" key. This module defines a class called `JsonGenerator` whose `get_bugzilla_field_json` and `get_jira_field_json` instance methods return json field information. The `bugjira.field_factory` module consumes that json to create lists of `BugzillaField` and `JiraField` objects.
+
+The `bugjira.json_generator.JsonGenerator` class loads its json data from a file whose path is (optionally) specified in the config dict under the "field_data_file_path" key. A sample file is provided in `contrib/sample_fields.json`. The field information in this file is not intended to be comprehensive; if you use the default `bugjira.json_generator` plugin, we encourage you to edit the sample fields file to support your JIRA instance and intended use cases.

contrib/bugjira.json

@@ -6,5 +6,7 @@
     "jira": {
         "URL": "https://issues.redhat.com",
         "token_auth": "your_personal_auth_token_here"
-    }
+    },
+    "json_generator_module": "bugjira.json_generator",
+    "field_data_path": "/path/to/contrib/sample_fields.json"
 }

contrib/sample_fields.json

@@ -0,0 +1,11 @@
+{
+    "bugzilla_field_data": [
+        {"name": "product"},
+        {"name": "component"},
+        {"name": "status"}
+    ],
+    "jira_field_data": [
+        {"name": "Issue Type", "jira_field_id": "issuetype"},
+        {"name": "Assignee", "jira_field_id": "assignee"}
+    ]
+}

src/bugjira/bugjira.py

@@ -1,5 +1,7 @@
+from bugjira import plugin_loader
 from bugjira.broker import BugzillaBroker, JiraBroker
 from bugjira.config import Config
+from bugjira.exceptions import BrokerInitException, JsonGeneratorException
 from bugjira.issue import Issue
 from bugjira.util import is_bugzilla_key, is_jira_key
 
@@ -33,6 +35,19 @@ def __init__(
         elif config_path:
             self.config = Config.from_config(config_path=config_path)
 
+        try:
+            plugin_loader.load_plugin(self.config)
+        except IOError as io_error:
+            raise BrokerInitException(
+                "An IOError was raised when loading the field data "
+                "generator plugin."
+            ) from io_error
+        except JsonGeneratorException as generator_error:
+            raise BrokerInitException(
+                "The field data json generator encountered a problem. Please "
+                "check the field data source."
+            ) from generator_error
+
         self._bugzilla_broker = BugzillaBroker(
             config=self.config, backend=bugzilla
         )

src/bugjira/config.py

@@ -22,6 +22,8 @@ class BugjiraConfigDict(BaseModel):
 
     bugzilla: BugzillaConfig
     jira: JiraConfig
+    json_generator_module: constr(strip_whitespace=True, min_length=1)
+    field_data_path: str = None
 
 
 class Config(BaseModel):

src/bugjira/exceptions.py

@@ -12,3 +12,11 @@ class BrokerAddCommentException(BrokerException):
 
 class BrokerLookupException(BrokerException):
     pass
+
+
+class JsonGeneratorException(Exception):
+    pass
+
+
+class PluginLoaderException(Exception):
+    pass

src/bugjira/field.py

@@ -0,0 +1,20 @@
+from pydantic import BaseModel, ConfigDict, constr
+
+"""The objects in this module are used internally to represent field
+configuration information for the bugzilla and jira backends used by bugjira.
+"""
+
+
+class BugjiraField(BaseModel):
+    """The base field class
+    """
+    model_config = ConfigDict(extra="forbid")
+    name: constr(strip_whitespace=True, min_length=1)
+
+
+class BugzillaField(BugjiraField):
+    pass
+
+
+class JiraField(BugjiraField):
+    jira_field_id: constr(strip_whitespace=True, min_length=1)

src/bugjira/field_factory.py

@@ -0,0 +1,32 @@
+from bugjira.field import BugzillaField, JiraField
+
+_json_generator = None
+
+
+def register_json_generator(generator, config):
+    """Registers a generator class to serve as the source for json that
+    specifies field configuration data.
+
+    :param generator: The generator to register
+    :type generator: class
+    :param config: The config to use when instantiating the generator class
+    :type config: dict
+    """
+    global _json_generator
+    _json_generator = generator(config=config)
+
+
+def get_bugzilla_fields() -> list[BugzillaField]:
+    fields = []
+    if _json_generator is not None:
+        for field_data in _json_generator.get_bugzilla_fields_json():
+            fields.append(BugzillaField(**field_data))
+    return fields
+
+
+def get_jira_fields() -> list[JiraField]:
+    fields = []
+    if _json_generator is not None:
+        for field_data in _json_generator.get_jira_fields_json():
+            fields.append(JiraField(**field_data))
+    return fields

src/bugjira/json_generator.py

@@ -0,0 +1,65 @@
+import json
+from typing import List
+
+from pydantic import BaseModel, ConfigDict, ValidationError
+
+from bugjira import field_factory
+from bugjira.exceptions import JsonGeneratorException
+from bugjira.field import BugzillaField, JiraField
+
+
+def get_generator():
+    """Plugin modules must define the get_generator() method, which should
+    return a generator class.
+    """
+    return JsonGenerator
+
+
+class JsonGenerator:
+    """This is the default plugin class that provides json to field_factory
+    module for generating lists of BugjiraField objects. It loads its data
+    from a json file, specified in the input config dict under the
+    "field_data_path" top level key.
+
+    The static `register` method registers the class with the field_factory
+    module.
+
+    Replacement plugin classes should implement the `get_bugzilla_fields_json`
+    and `get_jira_fields_json` instance methods, as well as the static
+    `register` method.
+    """
+    def __init__(self, config={}):
+        self.config = config
+        self.field_data = {}
+        if config:
+            field_data_path = config.get("field_data_path", "")
+            if field_data_path:
+                with open(field_data_path, "r") as file:
+                    field_data = json.load(file)
+                    try:
+                        # use pydantic class to validate the input data
+                        ValidFieldData(**field_data)
+                    except ValidationError:
+                        raise JsonGeneratorException(
+                            "Invalid field data detected"
+                        )
+                    self.field_data = field_data
+
+    def get_bugzilla_fields_json(self):
+        return self.field_data.get("bugzilla_field_data", [])
+
+    def get_jira_fields_json(self):
+        return self.field_data.get("jira_field_data", [])
+
+    @staticmethod
+    def register(config):
+        field_factory.register_json_generator(JsonGenerator, config)
+
+
+class ValidFieldData(BaseModel):
+    """This class defines the valid format for the json data loaded by the
+    JsonGenerator class
+    """
+    model_config = ConfigDict(extra="forbid")
+    bugzilla_field_data: List[BugzillaField]
+    jira_field_data: List[JiraField]

src/bugjira/plugin_loader.py

@@ -0,0 +1,38 @@
+import importlib
+
+from bugjira.exceptions import PluginLoaderException
+
+
+def import_module(name):
+    """Use importlib.import_module to import a plugin class
+
+    :param name: The fully qualified module name to import
+    :type name: str
+    :return: The module returned from the importlib.import_module method
+    :rtype: module
+    """
+    return importlib.import_module(name)
+
+
+def load_plugin(config={}):
+    """Load a plugin using configuration in the supplied config dict. An empty
+    config will result in loading the default json_generator module.
+
+    :param config: A Bugjira config dict, defaults to {}. Cannot be None.
+    :type config: dict, optional
+    """
+
+    if config is None:
+        raise PluginLoaderException("cannot load plugin with config=None")
+
+    # default to the bugjira-supplied default json_generator module
+    module_name = config.get("json_generator_module",
+                             "bugjira.json_generator")
+    try:
+        imported_module = import_module(module_name)
+        plugin = imported_module.get_generator()
+    except ModuleNotFoundError:
+        raise PluginLoaderException(
+            f"Could not load module named '{module_name}'"
+        )
+    plugin.register(config=config)

tests/conftest.py

@@ -19,6 +19,11 @@ def good_config_dict(good_config_file_path):
     return Config.from_config(config_path=good_config_file_path)
 
 
+@pytest.fixture()
+def good_sample_fields_file_path(config_defaults):
+    return config_defaults + "/data/sample_fields/sample_fields.json"
+
+
 @pytest.fixture
 def good_bz_keys():
     return ["123456", "1"]

tests/data/config/good_config.json

@@ -3,5 +3,7 @@
     "api_key": "your_bugzilla_api_key"},
  "jira": {
     "URL": "https://jira.yourdomain.com",
-    "token_auth": "your_jira_personal_access_token"}
- }
+    "token_auth": "your_jira_personal_access_token"},
+ "json_generator_module": "bugjira.json_generator",
+ "field_data_path": "/path/to/contrib/sample_fields.json"
+}

tests/data/config/missing_config.json

@@ -3,5 +3,6 @@
     "api_key": ""},
  "jira": {
     "URL": "",
-    "token_auth": ""}
- }
+    "token_auth": ""},
+ "json_generator_module": ""
+}

tests/data/sample_fields/sample_fields.json

@@ -0,0 +1,11 @@
+{
+    "bugzilla_field_data": [
+        {"name": "product"},
+        {"name": "component"},
+        {"name": "status"}
+    ],
+    "jira_field_data": [
+        {"name": "Issue Type", "jira_field_id": "issuetype"},
+        {"name": "Assignee", "jira_field_id": "assignee"}
+    ]
+}

tests/unit/test_bugjira.py

@@ -1,5 +1,5 @@
 from copy import deepcopy
-from unittest.mock import Mock, create_autospec
+from unittest.mock import Mock, create_autospec, patch
 from xmlrpc.client import Fault
 
 import pytest
@@ -8,8 +8,9 @@
 from jira.exceptions import JIRAError
 
 import bugjira.broker as broker
+import bugjira.bugjira as bugjira
 from bugjira.exceptions import (
-    BrokerLookupException, BrokerAddCommentException
+    BrokerLookupException, BrokerAddCommentException, JsonGeneratorException
 )
 from bugjira.bugjira import Bugjira
 from bugjira.issue import Issue, BugzillaIssue, JiraIssue
@@ -19,10 +20,12 @@
 def setup(monkeypatch):
     """Patch out the bugzilla.Bugzilla and jira.JIRA constructors
     in the broker module so that we don't attempt to connect to an actual
-    backend
+    backend. Also patch out the plugin_loader from the bugjira.bugjira
+    module so that it does not attempt to load the plugin.
     """
     monkeypatch.setattr(broker, "Bugzilla", create_autospec(Bugzilla))
     monkeypatch.setattr(broker, "JIRA", create_autospec(JIRA))
+    monkeypatch.setattr(bugjira, "plugin_loader", Mock())
 
 
 @pytest.fixture(scope="function")
@@ -82,6 +85,48 @@ def test_init_with_both_path_and_dict(good_config_file_path, good_config_dict):
     assert bugjira.config == edited_dict
 
 
+def test_init_plugin_loader_called(good_config_file_path):
+    """
+    GIVEN the Bugjira class' constructor
+    WHEN we call it with a good config path
+    THEN the plugin_loader.load_plugin method should be called once
+    """
+    with patch("bugjira.bugjira.plugin_loader") as plugin_loader:
+        Bugjira(config_path=good_config_file_path)
+        assert plugin_loader.load_plugin.call_count == 1
+
+
+def test_init_plugin_loader_io_error(good_config_dict):
+    """
+    GIVEN the Bugjira class' constructor
+    WHEN we call it with a valid config dict
+    AND the plugin_loader.load_plugin method raises an IOError
+    THEN a BrokerInitException should be raised
+    AND the error message should reflect that an IOError was handled
+    """
+
+    with patch("bugjira.bugjira.plugin_loader.load_plugin",
+               side_effect=IOError):
+        with pytest.raises(broker.BrokerInitException,
+                           match="An IOError was raised"):
+            Bugjira(config_dict=good_config_dict)
+
+
+def test_init_plugin_loader_json_generator_error(good_config_dict):
+    """
+    GIVEN the Bugjira class' constructor
+    WHEN we call it with a valid config dict
+    AND the plugin_loader.load_plugin method raises a JsonGeneratorException
+    THEN a BrokerInitException should be raised
+    AND the error message should reflect that the generator had a problem
+    """
+    with patch("bugjira.bugjira.plugin_loader.load_plugin",
+               side_effect=JsonGeneratorException):
+        with pytest.raises(broker.BrokerInitException,
+                           match="json generator encountered a problem"):
+            Bugjira(config_dict=good_config_dict)
+
+
 def test_init_with_no_parameters():
     """
     GIVEN the Bugjira class' constructor

tests/unit/test_config.py

@@ -103,6 +103,17 @@ def test_config_missing_value(good_config_dict):
     assert error.get("type") == "missing"
 
 
+def test_config_missing_optional_fields(good_config_dict):
+    """
+    GIVEN a dict containing a Bugjira config with missing optional elements
+    WHEN we call Config.from_config using the dict as the config_dict
+    THEN no ValidationError is raised
+    """
+    modified_config = deepcopy(good_config_dict)
+    modified_config.pop("field_data_path")
+    Config.from_config(config_dict=modified_config)
+
+
 def test_config_empty_value(good_config_dict):
     """
     GIVEN a dict containing a Bugjira config with one empty string value