RFC - Agnostic Lumberjack - Draft 1

[NachoVazquez]

RFC - Agnostic Lumberjack - Draft 1

TL;DR

This RFC proposes a new lumberjack package, agnostic to the underlying JavaScript framework. This package will power the @ngworker/lumberjack package and facilitate the creation of logging libraries for various frameworks beyond Angular, thereby providing a standard interface for logging.

This document will define and discuss the API of the agnostic lumberjack package.

Simultaneously, we must address the naming conventions for current and future packages as we introduce new packages.

Need

The need for a logging library agnostic to the underlying JavaScript framework emerged when we began using other frameworks for our jobs and side projects.

Developers typically use a specific client for their log storage provider, such as Logz.io, DataDog, or Sentry. This arrangement works fine if you use only one provider. However, using multiple frameworks requires a different client, leading to consistency and data mismatch issues.

Our goal is to create an agnostic logging library that offers native clients for all the frameworks that the community desires. Users can select the client they wish to use through our plugin system.

Success Criteria

By developing an agnostic Lumberjack library, we should be able to convert @ngworker/lumberjack into a framework (Angular) shell that employs the underlying library while maintaining its current features. The only exception is that many types and features previously exported by @ngworker/lumberjack will be moved to the new agnostic lumberjack package.

We should be able to create a new client for any framework, offering similar features to the Angular client. These features include:

• Offering multiple drivers (plugins) to send the logs to the desired destination.
• Utilizing the same configuration currently available for the core system and drivers, such as levels, format functions, etc.

Approach

To accomplish this, we will develop a new package that encompasses all the core logic of the existing lumberjack library, independent of the underlying JavaScript framework.

We will employ factory functions instead of classes to provide a more functional approach to the library and circumvent this binding issues.

All non-Angular related logic currently in the @ngworker/lumberjack package will be transferred to this new package.

Technical Design

The proposed public API for the new library is as follows:

/*
 * Public API surface of @webworker/lumberjack
 */

// Configuration
export { LumberjackConfig } from './lib/configuration/lumberjack.config';
export { LumberjackLogDriverConfig } from './lib/configuration/lumberjack-log-driver.config';
export { LumberjackOptions } from './lib/configuration/lumberjack.options';
export { defaultDevelopmentLevels } from './lib/configuration/default-development-levels';
export { defaultProductionLevels } from './lib/configuration/default-production-levels';
export { createLumberjackConfig } from './lib/configuration/create-lumberjack-config';

// Logs
export { LumberjackConfigLevels } from './lib/logs/lumberjack-config-levels';
export { LumberjackLevel } from './lib/logs/lumberjack-level';
export { LumberjackLog } from './lib/logs/lumberjack.log';
export { LumberjackLogLevel } from './lib/logs/lumberjack-log-level';
export { LumberjackLogPayload } from './lib/logs/lumberjack-log-payload';

// Log drivers
export { LumberjackLogDriver } from './lib/log-drivers/lumberjack-log-driver';
export { LumberjackLogDriverLog } from './lib/log-drivers/lumberjack-log-driver.log';

// Formatting
export { LumberjackFormatFunction } from './lib/formatting/lumberjack-format-function';
export { lumberjackFormatLog } from './lib/formatting/lumberjack-format-log';
export { utcTimestampFor } from './lib/formatting/utc-timestamp-for';

// Logging
export { createLumberjack } from './lib/logging/create-lumberjack';
export { Lumberjack } from './lib/logging/lumberjack';
export { createLumberjackLogFactory } from './lib/logging/create-lumberjack-log-factory';
export { LumberjackLogFactory } from './lib/logging/lumberjack-log-factory';
export { LumberjackLogBuilder } from './lib/logging/lumberjack-log.builder';

Let's dissect it to understand each component's purpose.

Configuration

The LumberjackConfig, LumberjackLogDriverConfig, LumberjackOptions, defaultDevelopmentLevels, and defaultProductionLevels are existing configuration files that will remain unchanged. They will be migrated to the agnostic library to make such configurations universally accessible to all framework client APIs.

The createLumberjackConfig is a new function that seamlessly integrates the default and custom user configurations.

It follows the same patterns in the @ngworker/lumberjack package when configuring Lumberjack through their modules and provideLumberjack functions.

/**
 * Helps combine the defaults Lumberjack configurations with custom developer configurations.
 * @param isProductionEnvironment - Lumberjack uses different default log levels based on its running environment.
 * @param options - LumberjackOptions that overwrite the default configuration.
 * @returns - The combination of default configs and custom overwrites.
 */
export function createLumberjackConfig<TPayload extends LumberjackLogPayload | void = void>(
  isProductionEnvironment: boolean,
  options: LumberjackOptions = {}
): LumberjackConfig<TPayload> {
  return {
    format: lumberjackFormatLog<TPayload>,
    levels: isProductionEnvironment ? defaultProductionLevels : defaultDevelopmentLevels,
    ...options,
  } as LumberjackConfig<TPayload>;
}

One point of discussion is whether we want to default the isProductionEnvironment to use the NODE_ENV environment variable as the default.

Logs

LumberjackConfigLevels, LumberjackLevel, LumberjackLog, LumberjackLogLevel, and LumberjackLogPayload are unchanged existing types and functions that will be extracted into the new package.

Log drivers

The LumberjackLogDriver and LumberjackLogDriverLog are unchanged existing interfaces that will be extracted into the new agnostic library.

Formatting

The lumberjackFormatLog is the default Lumberjack function to format logs. It will be extracted into the new package. The clear benefit of exporting this function is to use it in tests to validate the output of logging a log. However, that benefit could be achieved by providing a custom LumberjackFormatFunction on the configuration.

It is open to discussion if the lumberjackFormatLog function should belong to the public API.

The type for all possible Lumberjack format functions is exposed as LumberjackFormatFunction.

It is up to the discussion if the utcTimestampFor is going to remain as part of the public API of the agnostic library. Its only use outside of the agnostic core is in some Angular library tests.

Logging

The LumberjackLogBuilder is the existing builder class and will be extracted into the new package to assist with log creation.

The createLumberjackLogFactory is a new factory function that extracts the functionality previously found in the LumberjackLogFactory class. The LumberjackLogFactory class still exists, but it was renamed to LumberjackLogFactoryService. The LumberjackLogFactoryService is now a wrapper of this new factory function. The type LumberjackLogFactory was introduced in the agnostic library and represented the structure of the returned object by the createLumberjackLogFactory.

The createLumberjack is the new factory function that extracts the functionality previously found in the LumberjackService class. This is the core of the agnostic library and what all the new clients will use to access the Lumberjack core system. The LumberjackService is now a wrapper for this new factory function. The type Lumberjack was introduced in the agnostic library and represented the structure of the returned object by the createLumberjack.

Examples of use

Vanilla JavaScript

Since this library is agnostic of any framework, it can be used in any JavaScript project, including Node.js environments.

Examples of Lumberjack usage outside of Angular

Disclaimer: The design of the new clients will happen in a different document. The following examples are early ideas.

Express with simple API

import express from 'express';

import {
  createLumberjack,
  createLumberjackConfig,
  LumberjackLevel,
} from '@webworker/lumberjack';

const host = process.env.HOST ?? 'localhost';
const port = process.env.PORT ? Number(process.env.PORT) : 3000;

const app = express();

const lumberjack = createLumberjack({
  drivers: [...],
  config: createLumberjackConfig(process.env.NODE_ENV === 'production'),
});

const getUnixEpochTicks = () => new Date().valueOf();
const scope = 'Forest App';

app.get('/', (req, res) => {
  lumberjack.log({ message: 'Hello Forest', scope, level: LumberjackLevel.Info, createdAt: getUnixEpochTicks() });
  res.send({ message: 'Hello Forest' });
});

app.listen(port, host, () => {
  console.log(`[ ready ] http://${host}:${port}`);
});

Express with log builder factory

import express from 'express';

import {
  createLumberjack,
  createLumberjackConfig,
  createLumberjackLogFactory,
  LumberjackLevel,
} from '@webworker/lumberjack';

const host = process.env.HOST ?? 'localhost';
const port = process.env.PORT ? Number(process.env.PORT) : 3000;

const app = express();

const lumberjack = createLumberjack({
  drivers: [],
  config: createLumberjackConfig(process.env.NODE_ENV === 'production'),
});

const lumberjackLogFactory = createLumberjackLogFactory();
const scope = 'Forest App';

app.get('/', (req, res) => {
  const log = lumberjackLogFactory.createCriticalLog('The forest is on fire!').withScope(scope).build();
  lumberjack.log(log);
  res.send({ message: 'The forest is on fire!' });
});

app.listen(port, host, () => {
  console.log(`[ ready ] http://${host}:${port}`);
});

JSX Frameworks

The following code shows how to use the agnostic library to create an JSX framework Lumberjack client like React.

This code has not been tested and is only for demonstration purposes.

import * as React from 'react';

const LumberjackContext = React.createContext();

function LumberjackProvider({ children, drivers, config }) {
  const lumberjack = createLumberjack({ drivers, config });
  return <LumberjackContext.Provider value={lumberjack}>{children}</LumberjackContext.Provider>;
}

function useLumberjack() {
  const context = React.useContext(LumberjackContext);
  if (context === undefined) {
    throw new Error('useLumberjack must be used within a LumberjackProvider');
  }
  return context;
}

export { LumberjackProvider, useLumberjack };

Then we could provide it like this:

export function RootComponent() {
  return (
  <LumberjackProvider drivers=[...] config={createLumberjackConfig(process.env.NODE_ENV === 'production')}>
  // ... your components
  </ LumberjackProvider>
  )
}

And to use it:

export function SomeComponent() {
  const lumberjack = useLumberjack();

  useEffect(() => {
    lumberjack.log({...})
  }, []);
}

Breaking Changes

Relocation of files

The following lists the types, classes, and functions moved from the @ngworker/lumberjack package to the new agnostic library.

• LumberjackConfig
• LumberjackLogDriverConfig
• LumberjackOptions
• defaultDevelopmentLevels
• defaultProductionLevels
• LumberjackConfigLevels
• LumberjackLevel
• LumberjackLog
• LumberjackLogLevel
• LumberjackLogPayload
• LumberjackLogDriver
• LumberjackLogDriverLog
• lumberjackFormatLog
• utcTimestampFor
• LumberjackLogBuilder

Renaming of the LumberjackLogFactory

With the creation of the createLumberjackLogFactory on the new agnostic library, we also created a new type to represent the object that such a function returns. The type received the name LumberjackLogFactory,, which conflicts with the existing @ngworker/lumberjack service.

Following Angular naming conventions, we renamed the service to LumberjackLogFactoryService.

Naming Considerations

We name ourselves and our organization the ngworker, clearly referencing Angular and the ng prefix. This is reflected in the lumberjack package name @ngworker/lumberjack.

However, with the introduction of an agnostic framework and the possibility of using it in other frameworks, we must consider what conventions to follow when naming the new package.

The following are some suggestions, but we are open to considering other options.

@webworker/lumberjack, @ngworker/lumberjack, @reactworker/lumberjack, etc

This variant specifies the framework's name using the prefix/organization scope.

Its main benefit is that it would keep the identity of the original package, and it would be easy to identify the framework being used.

The main drawback is that the ownership of the packages would need to be clarified. It would also be confusing to have the packages living under the @ngworker GitHub organization but not being related to Angular necessarily.

Renaming the organization-scope

This option would look like @lumberjack/core, @lumberjack/angular, @lumberjack/react, etc.

The benefit of this approach is that all packages get grouped under the same organization, and it is clear that they are related.

The main drawback is that @ngworker/lumberjack would have to cease to exist, or it should be deprecated and point to the new organization.

Out of Scope

This effort does not involve creating framework clients for Lumberjack. That will be addressed in the following work.

Implementation

• feature: extract lumberjack core into an agnostic library #154

[LayZeeDK]

npm scope and package names

I would prefer it if we could have the packages in the same npm scope. The @lumberjack npm scope is unavailable because there is an npm user with the lumberjack handle.

Maybe something like @lumberjackjs.

What do you all think?

[NachoVazquez]

Yeah, I like simpler. For drivers could be as simple as adding the suffix driver.

Something like @lumberjackjs/http-driver. This assumes that drivers are agnostic.

For non-agnostic drivers, we could use a prefix with the framework, for example, @lumberjackjs/angular-http-driver or with a / separator like @lumberjackjs/angular/http-driver. I'm not sure this last one is possible outside of secondary entry points but I think it is.

[LayZeeDK]

For non-agnostic drivers, we could use a prefix with the framework, for example, @lumberjackjs/angular-http-driver or with a / separator like @lumberjackjs/angular/http-driver. I'm not sure this last one is possible outside of secondary entry points but I think it is.

This would be a secondary entry point of an npm package named @lumberjackjs/angular.

[LayZeeDK]

Something like @lumberjackjs/http-driver. This assumes that drivers are agnostic.

Not @lumberjackjs/http-log-driver?

[NachoVazquez]

Not @lumberjackjs/http-log-driver?

That also works.

[LayZeeDK]

Decision

The npm scope is @lumberjackjs as we want to keep Lumberjack packages within the same npm scope and the @lumberjack npm scope is unavailable.

The initial packages and secondary entry points will be the following.

• @lumberjackjs/angular
• @lumberjackjs/angular-http-driver
• @lumberjackjs/core
  • @lumberjackjs/core/console-driver

We decided that the console API is so universal and essential that we want to include it in the @lumberjack/core package but as a secondary entry point.

We decided that HTTP log drivers like Angular HttpClient drivers and fetch drivers are shipped as separate packages, not secondary entry points as not all developers use them and one might one to use one or the other and not pay with unused dependencies or additional bundle size.

The following are examples of potential future experimental packages.

• @lumberjackjs/cloudflare-workers
• @lumberjackjs/fetch-driver
• @lumberjackjs/node
• @lumberjackjs/qwik
• @lumberjackjs/react
• @lumberjackjs/solid
• @lumberjackjs/vue

[LayZeeDK]

Common log drivers

I think we should ship a console log driver with the core library. I also suggest that we ship a fetch-based HTTP log driver with the core library.

[NachoVazquez]

I love the idea. I would ask if we want to include the drivers with the core library or publish them as separate packages.

[LayZeeDK]

I think it should depend on how often framework-specific common log drivers would rely on the common core log drivers and how often they would have to implement their own versions like the Angular HTTP log driver which wouldn't use the fetch-based HTTP log driver behind-the-scenes.

[NachoVazquez]

It makes sense. At this point, I would prefer to have them as separate packages for consistency. Treat all drivers the same and avoid the interpretation of official vs. non-official plugins, which we don't necessarily will follow.

[LayZeeDK]

Decision

A framework-agnostic console driver is part of the secondary entry point @lumberjackjs/core/console-driver

Possible future package: @lumberjackjs/fetch-driver

We decided that the console API is so universal and essential that we want to include it in the @lumberjack/core package but as a secondary entry point.

We decided that HTTP log drivers like Angular HttpClient drivers and fetch drivers are shipped as separate packages, not secondary entry points as not all developers use them.

[LayZeeDK]

Class and function names

I heavily dislike the Service suffix in class names. It says nothing about what a class does. If anything, it's an indicator that a class has too many responsibilities and/or that the developers are poor at naming. I am already unhappy with the name LumberjackService in the @ngworker/lumberjack package.

LumberjackLogFactoryService

Come on, this is very bad.

createLumberjackLogFactory

Functions with a create prefix in their name are factory functions. Classes with a Factory suffix in their names are factory classes. What is createLumberjackLogFactory? A factory function for a factory class instance? That would be a factory class constructor, no?

[NachoVazquez]

Yes, my naming game was not in its prime there.

[LayZeeDK]

Don't worry, I still love you 🫶

[NachoVazquez]

I will return with some suggestions tomorrow, but if you have any, please let us know.

[NachoVazquez]

The problem is that the createLumberjackLogFactory is a factory function that returns an object with several LumberjackLogBuilder factory functions.

I used the create prefix to indicate that this function is a factory function.

And the suffix Factory indicates that the factory function returns other factory functions.

The most accurate name I can think of right now is createLumberjackLogBuilderFactories

[LayZeeDK]

Decision

@ngworker/lumberjack had LumberjackLogFactory that requires a LumberjackTimeService instance. The Lumberjack Core version of this should be individual functions that are curried so that consumers can use partial application to pass a LumberjackTimeService to return a function that accepts a log message.

The non-class version of a cohesive group of functions is an ECMAScript Module exporting pure functions. Pure function versions of methods that rely on class collaborators are curried functions that accept the collaborators.

[LayZeeDK]

Examples

@NachoVazquez

Could you add examples showing JSX-style framework usage of useLumberjack and Vanilla JavaScript usage of declarative loggers?

[NachoVazquez]

Definitely! Should I create a copy of the Draft before making modifications and bumping its version?

[LayZeeDK]

That's not necessary. Previous versions are available from the Edited dropdown menu.

[NachoVazquez]

Amazing!

[NachoVazquez]

We should not include the API design for the JSX frameworks in this discussion. Those APIs need to be carefully designed.

However, I completed the snippets I proposed.

The following is an example of using the agnostic API with Express.

Examples of Lumberjack usage outside of Angular

Disclaimer: The design of the new clients will happen in a different document. The following examples are early ideas.

VanillaJS

Express with simple API

import express from 'express';

import {
  createLumberjack,
  createLumberjackConfig,
  LumberjackLevel,
} from '@webworker/lumberjack';

const host = process.env.HOST ?? 'localhost';
const port = process.env.PORT ? Number(process.env.PORT) : 3000;

const app = express();

const lumberjack = createLumberjack({
  drivers: [...],
  config: createLumberjackConfig(process.env.NODE_ENV === 'production'),
});

const getUnixEpochTicks = () => new Date().valueOf();
const scope = 'Forest App';

app.get('/', (req, res) => {
  lumberjack.log({ message: 'Hello Forest', scope, level: LumberjackLevel.Info, createdAt: getUnixEpochTicks() });
  res.send({ message: 'Hello Forest' });
});

app.listen(port, host, () => {
  console.log(`[ ready ] http://${host}:${port}`);
});

Express with log builder factory

import express from 'express';

import {
  createLumberjack,
  createLumberjackConfig,
  createLumberjackLogFactory,
  LumberjackLevel,
} from '@webworker/lumberjack';

const host = process.env.HOST ?? 'localhost';
const port = process.env.PORT ? Number(process.env.PORT) : 3000;

const app = express();

const lumberjack = createLumberjack({
  drivers: [],
  config: createLumberjackConfig(process.env.NODE_ENV === 'production'),
});

const lumberjackLogFactory = createLumberjackLogFactory();
const scope = 'Forest App';

app.get('/', (req, res) => {
  const log = lumberjackLogFactory.createCriticalLog('The forest is on fire!').withScope(scope).build();
  lumberjack.log(log);
  res.send({ message: 'The forest is on fire!' });
});

app.listen(port, host, () => {
  console.log(`[ ready ] http://${host}:${port}`);
});

[LayZeeDK]

Lumberjack Core Configuration API

I think that the following are implementation details that shouldn't be part of a public API.

• defaultDevelopmentLevels
• defaultProductionLevels
• lumberjackFormatLog
• utcTimestampFor

[NachoVazquez]

You are correct!

[LayZeeDK]

Declarative Lumberjack Core logging API

LumberjackLogger and ScopedLumberjackLogger are essential and the recommended API for using Lumberjack in an Angular application. The Lumberjack Core API should also have a declarative logging API.

[NachoVazquez]

This is an interesting discussion. Should we add the declarative logging API to the core or keep it minimal and add the declarative logging APIs for the SDK?

[LayZeeDK]

Save the log drivers

I don't feel good about entirely removing the concept and wording of log drivers. After all, log driving is an actual activity for lumberjacks.

[NachoVazquez]

Yes, I was refactoring the code and had feelings about it.

The question is: Should we add it to just documentation, or should we put it back everywhere and stop using Driver independently?

[NachoVazquez]

Closing this discussion as we decided to move in a different direction, see #188