Plugin framework v2 + new Big Object Archive plugin + new Log Retenti…

…on Rules plugin (#298)

- Replaced `LoggerSObjectHandlerPlugin` abstract class with new class `LoggerPlugin` that contains 2 interfaces + helper methods, switched to using multiple fields on `LoggerPlugin__mdt` to indicate Apex classes & Flows to run for a plugin, removed SObject-specific fields on `LoggerPlugin__mdt`, added new fields `Log__c.LogPurgeAction__c` and `LoggerSettings__c.DefaultLogPurgeAction__c`

- Added support for plugins within LogBatchPurger + added archiving in BigObject plugin (#288)

- Added SObject-specific 'execution order' fields on `LoggerPlugin__mdt`

- Added picklist values for API versions in calendar year 2022

- Reintroduced the ability to disable trigger handler classes by adding new `LoggerParameter__mdt` records to control via configuration

- Renamed CMDT records that control each trigger handler

- Added new abstract methods in LoggerSObjectHandler for retrieving each handler's related records in `LoggerParameter__mdt` & `LoggerPlugin__mdt`

- Added the ability to configure the list of save methods shown in `loggerSettings` LWC via `LoggerParameter__mdt`, using the prefix 'CustomSaveMethod'

- Standardized test-visible method naming conventions to start with `setMock` & updated approaches to use a Map instead of List for CMDT records

- Fixed issues in LoggerSObjectHandler for invalid Apex & Flow plugins, started new test class LoggerSObjectHandler_Tests

- Added `Log__c` list view to show logs that will be purged in the next 10 days, fixed LogEntryArchiveBuilder using the wrong value for LoggedById**c, updated index on LogEntryArchive**b again (still a WIP), retrieved & formatted metadata for LogEntryArchive\*\*b, updated Admin.profile

- Closed #226 by adding new "Log Retentions Rule" plugin, based on my other project, ApexValidationRules

- @jamessimone closed #117, and I closed #228 by adding new "Big Object Archiving" plugin

- Lowercased the plugin directories

- Added plugins folder to the script for generating apex doc files, generated updated docs for Apex

- Bumped package version number for the managed package

- Added 5 minute wait between unlocked package version creation & package installation, added deprecated `LoggerPlugin__mdt` validation rule & field back into the managed-package folder, cleaned up some docs markdown files

- Added tests in LoggerSettingsController_Tests & LogHandler_Tests for new custom setting LoggerSettings**c.DefaultLogPurgeAction**c

- Added 'deprecated' to the label of several deprecated fields on `LoggerPlugin__mdt` & removed the related handler methods

- More improvements for CMDT records used in the classes LoggerParameter & LoggerPlugin

  - Added LoggerParameter.matchOnPrefix() to return CMDT records with a specified prefix in the DeveloperName field
  - Added inner Comparable class in LoggerPlugin to handle custom sorting, since there are some limitations with SOQL queries for CMDT
  - Classes like LogBatchPurger, LoggerSObjectHandler, and LoggerSettingsController no longer need to track their own mock CMDT records, the LoggerPlugin & LoggerParameter classes now fully handle mocks for their corresponding CMDT objects
  - LoggerParameter & LoggerPlugin now require mock CMDT records to have DeveloperName populated - they'll throw errors if the field is null
  - Also added System namespace to some calls for Test.isRunningTest() to handle crazy orgs that have a custom Test class deployed

- Added the ability to disable all triggers during tests via new @testvisible method LoggerSObjectHandler.shouldExecute(Boolean)

- Moved LoggerTestUtils to the configuration folder, added public methods for mocking all CMDT records

- Consolidated plugin-framework folders back into the configuration folders

- Moved demo experience cloud metadata under config, instead of under nebula-logger, updated build.yml to deploy instead of push metadata

- Updated test classes to use new overloads for LoggerTestUtils.setMock()

- Added new class LoggerDataStore to centralize all database, event bus & queueable operations

- Closed #193 by adding classes `LoggerMockDataCreator` for generic mock & data creation (part of configuration layer), `LoggerTestConfigurator` forNebula Logger-specific mocking & setup (part of the log-management layer), and `LoggerMockDataStore` for mocking database & event bus operations throughout the codebase

- Started work on #292 by adding businessStatus, complianceGroup & securityClassification nodes to several custom fields (still a WIP)

- Rewrote tests in `LogEntryEventBuilder_Tests` to use new methods in `LoggerMockDataCreator` for creating mock instances of database result classes

- Scope creep: added fallback fields for setting `RecordName__c` in LogEntryHandler on objects that don't have a display name field, also added some caching

- Moved permissionSets and customPermissions folders from configuration to log-management

  - This should make the configuration folder fully standalone (i.e., it could be deployed by itself)
  - The perm sets were already referencing all of the objects within log-management, so it seems like it's better to keep the perm sets organized with

- Changed labels to just "Entry #" on both `LogEntryEvent__e` and `LogEntryArchive__b`

- Fixed an issue in `LogEntryEventBuilder` where some stack trace lines would be duplicated

- Scope creeped a bit by adding new fields `DatabaseResultCollectionSize__c` and `RecordCollectionSize__c` - I had planned to add these as part of #222, but decided to add them now so I could duplicate the fields to the `LogEntryArchive__b` big object as well. This also helped to identify (and fix) some issues with missing asserts in LogEntryEventHandler_Tests.

- Replaced settings field `IsPlatformEventStorageEnabled__c` with new field `DefaultPlatformEventStorageLocation__**__c`, providing a way for plugins (big object plugin in particular) to provide additional storage location. The big object plugin provides a custom option BIG_OBJECT, in addition to the default value CUSTOM_OBJECTS that will now be used out of the box

- Added stack trace fields to the Slack message created via SlackLoggerPlugin

- Updated most test classes to use `@IsTest(isParallel=true)`. A few tests (especially integration tests) that do DML on User can't use parallel testing, but most test classes are now using it

- Partially implemented #240 by adding HTTP Request & HTTP Response fields to `LogEntryEvent__e`, `LogEntry__c` and `LogEntryArchive__b`. More scope creep yet again, but this will save some headaches long term - since Big Objects can't (easily) have new fields added, I wanted to go ahead and get these new fields added to the data model before the first release of the Big Object plugin. I'll probably add some logging overloads in Logger for these items in a future release, but for now, only LogEntryEventBuilder has new methods for HttpRequest & HttpResponse

- Updated some internal debug messages in various places to use Logger's logging methods, instead of `System.debug()`

- Deprecated the fields `LogEntryDataMaskRule__mdt.ApplyToMessage__c` and `LogEntryDataMaskRule__mdt.ApplyToRecordJson__c`

- Fixed test failures that occur when email deliverability is disabled in an org

Co-authored-by: James Simone <[email protected]>

.forceignore

@@ -4,7 +4,7 @@
 
 nebula-logger/**/main/default/
 nebula-logger/managed-package/sfdx-project.json
-nebula-logger/managed-package/**/testSuites/**
+nebula-logger/managed-package/**/*.testSuite-meta.xml
 
 # Directory/package-specific README files
 **/README.md

.github/codecov.yml

@@ -7,6 +7,6 @@ coverage:
                 if_ci_failed: success
         patch: off
 ignore:
-    - 'nebula-logger-recipes/**/*'
+    - 'nebula-logger/recipes/**/*'
 comment:
     behavior: new

.github/workflows/build.yml

@@ -160,8 +160,8 @@ jobs:
             - name: 'Create Base Scratch Org'
               run: npx sfdx force:org:create --durationdays 1 --definitionfile ./config/scratch-orgs/base-scratch-def.json --wait 20 --setdefaultusername --json
 
-            - name: 'Push Source to Scratch Org'
-              run: npm run source:push
+            - name: 'Deploy Source to Scratch Org'
+              run: npm run source:deploy
 
             - name: 'Assign Logger Admin Permission Set'
               run: npm run permset:assign:admin
@@ -213,8 +213,8 @@ jobs:
             - name: 'Deploy Test Experience Site Metadata'
               run: npm run experience:deploy
 
-            - name: 'Push Source to Scratch Org'
-              run: npm run source:push
+            - name: 'Deploy Source to Scratch Org'
+              run: npm run source:deploy
 
             - name: 'Assign Logger Admin Permission Set'
               run: npm run permset:assign:admin

README.md

@@ -5,10 +5,10 @@
 
 Designed for Salesforce admins, developers & architects. A robust logger for Apex, Lightning Components, Flow, Process Builder & Integrations.
 
-## Unlocked Package - v4.7.0
+## Unlocked Package - v4.7.1
 
-[![Install Unlocked Package in a Sandbox](./images/btn-install-unlocked-package-sandbox.png)](https://test.salesforce.com/packaging/installPackage.apexp?p0=04t5Y0000015lXSQAY)
-[![Install Unlocked Package in Production](./images/btn-install-unlocked-package-production.png)](https://login.salesforce.com/packaging/installPackage.apexp?p0=04t5Y0000015lXSQAY)
+[![Install Unlocked Package in a Sandbox](./images/btn-install-unlocked-package-sandbox.png)](https://test.salesforce.com/packaging/installPackage.apexp?p0=04t5Y0000015lgBQAQ)
+[![Install Unlocked Package in Production](./images/btn-install-unlocked-package-production.png)](https://login.salesforce.com/packaging/installPackage.apexp?p0=04t5Y0000015lgBQAQ)
 [![View Documentation](./images/btn-view-documentation.png)](https://jongpie.github.io/NebulaLogger/)
 
 ## Managed Package - v4.7.0
@@ -702,9 +702,9 @@ The optional [Slack plugin](./nebula-logger-plugins/Slack/) leverages the Nebula
 -   Extend permission sets to include field-level security for your custom fields
 -   Leverage the new `LoggerParameter__mdt` CMDT object to store configuration for your plugin
 
-Check out the [Slack plugin](./nebula-logger-plugins/Slack/) for more details on how to install & customize the plugin
+Check out the [Slack plugin](./nebula-logger-plugins/slack/) for more details on how to install & customize the plugin
 
-![Slack plugin: notification](./nebula-logger-plugins/Slack/images/slack-plugin-notification.png)
+![Slack plugin: notification](./nebula-logger/plugins/slack/.images/slack-plugin-notification.png)
 
 ---
 

config/scratch-orgs/base-scratch-def.json

@@ -12,6 +12,9 @@
         "communitiesSettings": {
             "enableNetworksEnabled": false
         },
+        "pathAssistantSettings": {
+            "pathAssistantEnabled": true
+        },
         "userManagementSettings": {
             "enableEnhancedPermsetMgmt": true,
             "enableEnhancedProfileMgmt": true,

config/scratch-orgs/experience-cloud-scratch-def.json

@@ -15,6 +15,9 @@
         "experienceBundleSettings": {
             "enableExperienceBundleMetadata": true
         },
+        "pathAssistantSettings": {
+            "pathAssistantEnabled": true
+        },
         "userManagementSettings": {
             "enableEnhancedPermsetMgmt": true,
             "enableEnhancedProfileMgmt": true,

docs/apex/configuration/LoggerParameter.md → docs/apex/Configuration/LoggerParameter.md

@@ -458,4 +458,24 @@ List<String>
 
 A list that's retrieved using the parameter if it's not null, otherwise the default list is returned.
 
+#### `matchOnPrefix(String developerNamePrefix)` → `List<LoggerParameter_t>`
+
+matchOnPrefix description
+
+##### Parameters
+
+| Param                 | Description                                                                                 |
+| --------------------- | ------------------------------------------------------------------------------------------- |
+| `developerNamePrefix` | A prefix that has been used in the `DeveloperName` for multiple `LoggerParameter_t` records |
+
+##### Return
+
+**Type**
+
+List&lt;LoggerParameter_t&gt;
+
+**Description**
+
+The list of matching `LoggerParameter_t` records
+
 ---

docs/apex/Configuration/LoggerPlugin.md

@@ -0,0 +1,116 @@
+---
+layout: default
+---
+
+## LoggerPlugin class
+
+The core of the plugin framework, used to create custom Apex & Flow plugins for `LoggerSObjectHandler` and `LogBatchPurger` based on configurations stored in the custom metadata type `LoggerPlugin_t`
+
+---
+
+### Properties
+
+#### `pluginConfiguration` → `LoggerPlugin_t`
+
+---
+
+### Methods
+
+#### `PluginConfigurationSorter(LoggerPlugin_t pluginConfiguration)` → `public`
+
+#### `compareTo(Object compareTo)` → `Integer`
+
+#### `getFilteredPluginConfigurations(List<Schema.SObjectField> populatedFilterFields, Schema.SObjectField sortByField)` → `List<LoggerPlugin_t>`
+
+Filters the configured `LoggerPlugin_t` records based on a list of `SObjectField` - only records that have a value for 1 or more of the specified `populatedFilterFields` will be returned, sorted by the specified `SObjectField` parameter `sortByField`
+
+##### Parameters
+
+| Param                   | Description                                                                                                                   |
+| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
+| `populatedFilterFields` | The list of `SObjectField` to check on each `LoggerPlugin_t` record - filtering logic checks for a non-null value             |
+| `sortByField`           | The `SObjectField` to use to sort the list of matches. The method also uses `DeveloperName` as a secondary field for sorting. |
+
+##### Return
+
+**Type**
+
+List&lt;LoggerPlugin_t&gt;
+
+**Description**
+
+The list of matching `LoggerPlugin_t` records
+
+#### `newBatchableInstance(String apexClassTypeName)` → `Batchable`
+
+Creates an instance of the class `LoggerPlugin.Batchable` based on the provided `LoggerPlugin_t` configuration
+
+##### Parameters
+
+| Param               | Description                                                         |
+| ------------------- | ------------------------------------------------------------------- |
+| `apexClassTypeName` | The name of the Apex class that implements `LoggerPlugin.Batchable` |
+
+##### Return
+
+**Type**
+
+Batchable
+
+**Description**
+
+The dynamically created instance of `LoggerPlugin.Batchable`,
+
+#### `newTriggerableInstance(String apexClassTypeName)` → `Triggerable`
+
+Creates an instance of the class `LoggerPlugin.Triggerable` based on the provided `LoggerPlugin_t` configuration
+
+##### Parameters
+
+| Param               | Description                                                           |
+| ------------------- | --------------------------------------------------------------------- |
+| `apexClassTypeName` | The name of the Apex class that implements `LoggerPlugin.Triggerable` |
+
+##### Return
+
+**Type**
+
+Triggerable
+
+**Description**
+
+The dynamically created instance of `LoggerPlugin.Triggerable`,
+
+#### `sortBy(Schema.SObjectField field)` → `PluginConfigurationSorter`
+
+---
+
+### Inner Classes
+
+#### LoggerPlugin.Batchable interface
+
+Interface used to create plugins that can be used within Logger&apos;s batch job `LogBatchPurger`
+
+---
+
+##### Methods
+
+###### `execute(LoggerPlugin_t configuration, LoggerBatchableContext input, List<SObject> scopeRecords)` → `void`
+
+###### `finish(LoggerPlugin_t configuration, LoggerBatchableContext input)` → `void`
+
+###### `start(LoggerPlugin_t configuration, LoggerBatchableContext input)` → `void`
+
+---
+
+#### LoggerPlugin.Triggerable interface
+
+Interface used to create plugins that can be used within Logger&apos;s trigger handler framework `LoggerSObjectHandler`
+
+---
+
+##### Methods
+
+###### `execute(LoggerPlugin_t configuration, LoggerTriggerableContext input)` → `void`
+
+---

docs/apex/log-management/LogBatchPurger.md → docs/apex/Log-Management/LogBatchPurger.md

@@ -31,7 +31,7 @@ Required by the Database.Batchable interface, this method executes the logic for
 | `batchableContext` | - The context of the current batch job. |
 | `logRecords`       | - The log records to purge.             |
 
-#### `execute(Database.BatchableContext batchableContext, List<SObject> loggerRecords)` → `void`
+#### `execute(Database.BatchableContext batchableContext, List<SObject> scopeRecords)` → `void`
 
 Required by the Database.Batchable interface, this method executes the logic for purging log records.
 
@@ -40,7 +40,7 @@ Required by the Database.Batchable interface, this method executes the logic for
 | Param              | Description                             |
 | ------------------ | --------------------------------------- |
 | `batchableContext` | - The context of the current batch job. |
-| `loggerRecords`    | - The log records to purge.             |
+| `scopeRecords`     | - The log records to purge.             |
 
 #### `finish(Database.BatchableContext batchableContext)` → `void`
 
@@ -52,6 +52,26 @@ Required by the Database.Batchable interface. This method runs after all batch j
 | ------------------ | ------------------------------- |
 | `batchableContext` | - The context of the batch jobs |
 
+#### `setChainedBatchSize(Integer chainedBatchSize)` → `LogBatchPurger`
+
+The `LogBatchPurger` job is designed to run several instances - typically, it runs on `LogEntryTag__c`, then `LogEntry__c`, and finally `Log__c`. This method provides a way to control the batch size used for the chained instances of `LogBachPurger`
+
+##### Parameters
+
+| Param              | Description                                                                    |
+| ------------------ | ------------------------------------------------------------------------------ |
+| `chainedBatchSize` | The batch size to use for any subsequent chained instances of `LogBatchPurger` |
+
+##### Return
+
+**Type**
+
+LogBatchPurger
+
+**Description**
+
+The same instance of `LogBatchPurger`, useful for chaining methods
+
 #### `start(Database.BatchableContext batchableContext)` → `Database.QueryLocator`
 
 Required by the Database.Batchable interface. Collects the records / objects passed in to the batch instance and returns a Databae.QueryLocator reference representing the current iteration.

docs/apex/Log-Management/LogEntryEventHandler.md

@@ -0,0 +1,45 @@
+---
+layout: default
+---
+
+## LogEntryEventHandler class
+
+Processes `LogEntryEvent__e` platform events and normalizes the data into `Log__c` and `LogEntry__c` records
+
+---
+
+### Constructors
+
+#### `LogEntryEventHandler()`
+
+## Default constructor, used by the trigger `LogEntryEvent.trigger`
+
+### Properties
+
+#### `releaseNumber` → `String`
+
+String containing the release number.
+
+#### `releaseVersion` → `String`
+
+String containing the release version.
+
+---
+
+### Methods
+
+#### `getSObjectType()` → `SObjectType`
+
+Returns SObject Type that the handler is responsible for processing
+
+##### Return
+
+**Type**
+
+SObjectType
+
+**Description**
+
+The instance of `SObjectType`
+
+---

docs/apex/Log-Management/LoggerBatchableContext.md

@@ -0,0 +1,40 @@
+---
+layout: default
+---
+
+## LoggerBatchableContext class
+
+Class used by the logging system for batch contextual details
+
+### Related
+
+[LogBatchPurger](LogBatchPurger)
+
+[LoggerPlugin](LoggerPlugin)
+
+---
+
+### Constructors
+
+#### `LoggerBatchableContext(Database.BatchableContext batchableContext, Schema.SObjectType sobjectType)`
+
+Constructor used to set the 2 properties
+
+##### Parameters
+
+| Param              | Description                                                                                       |
+| ------------------ | ------------------------------------------------------------------------------------------------- |
+| `batchableContext` | The instance of `Database.BatchableContextbatchableContext`, provided by the platform at run-time |
+| `sobjectType`      | The `SObjectType` that will be queried & purged                                               |
+
+---
+
+### Properties
+
+#### `batchableContext` → `Database.BatchableContext`
+
+#### `sobjectType` → `Schema.SObjectType`
+
+#### `sobjectTypeName` → `String`
+
+---

docs/apex/Log-Management/LoggerEmailSender.md

@@ -0,0 +1,35 @@
+---
+layout: default
+---
+
+## LoggerEmailSender class
+
+Builds and sends email notifications when internal exceptions occur within the logging system
+
+---
+
+### Methods
+
+#### `sendErrorEmail(Schema.SObjectType sobjectType, List<Database.SaveResult> saveResults)` → `void`
+
+Sends an error email notification to the org&apos;s list of Apex Exception Email recipients, configured under Setup --&gt; Email --&gt; Apex Exception Email
+
+##### Parameters
+
+| Param         | Description                                                    |
+| ------------- | -------------------------------------------------------------- |
+| `sobjectType` | The SObjectType of records being saved.                        |
+| `saveResults` | The list of Database.SaveResult instances to use in the email. |
+
+#### `sendErrorEmail(Schema.SObjectType sobjectType, List<Database.UpsertResult> upsertResults)` → `void`
+
+Sends an error email notification to the org&apos;s list of Apex Exception Email recipients, configured under Setup --&gt; Email --&gt; Apex Exception Email
+
+##### Parameters
+
+| Param           | Description                                                      |
+| --------------- | ---------------------------------------------------------------- |
+| `sobjectType`   | The SObjectType of records being saved.                          |
+| `upsertResults` | The list of Database.UpsertResult instances to use in the email. |
+
+---