What's the difference between box:: and import::?

[king-of-poppk]

See https://cran.r-project.org/web/packages/import/vignettes/import.html.

[klmr]

I’ve been meaning to write a comparison article but doing this in a fair way is hard because (a) I am obviously biased and (b) I admittedly don’t know ‘import’ nearly as well as ‘box’. Having said that, it is my conviction that the module system implemented by ‘box’ is strictly superior to that implemented by ‘import’. Furthermore, as far as I am aware, the functionality provided by ‘box’ is a superset of the functionality provided by ‘import’: it does everything¹ ‘import’ does, plus more.

But let’s start at the beginning. ‘box’ and ‘import’ have a large overlap in functionality and aim to solve some of the same problems, but I think the overarching aims differ:

• The primary aim of ‘import’ is to prevent name clashes by controlling which names get attached when a package is loaded. In fact, for a long time this was the only functionality ‘import’ provided.

• The primary aim of ‘box’ is to implement a modern module system for R which intends to rival packages as the primary means of code reuse. To be clear, preventing name clashes is also an aim of ‘box’, but that’s just one in a large list of issues that I am seeing with R’s package system.

Over time, the two packages have converged in functionality. In particular, ‘import’ has gained the ability to load code from scripts rather than packages, which is similar to (but more limited in scope than) the module system from ‘box’. And ‘box’ has gained the ability to import packages in addition to modules because I realised that my stated aim of replacing the existing package system was too ambitious.

So what does ‘box’ provide that ‘import’ does not provide?

1. Global module library
2. Relative imports that actually work
3. Module documentation via ‘roxygen2’
4. Strict module boundaries via export declaration
5. Scope-local imports
6. Circular imports

(Note that this is a non-exhaustive list; I do want to keep it short.)

Here they are in more detail:

Global module library

‘box’ knows two types of modules: project-local modules and global modules.

Project-local modules are located in a subfolder of the current project. Global modules are located in one or more a central installation locations, akin to the R package library (that’s why you can use an R package in your code without having to reinstall it for each project). The R package library paths are configured via .libPaths(). ‘box’ has an equivalent mechanism to set the module library paths (via options('box.path')). ‘import’ has nothing equivalent. It could import centrally-installed modules but this requires hard-coding the path of that central location, which is a complete no-go.

Relative imports that actually work

Supporting project-local modules — and in particular hierarchical, nested modules — requires that these modules can be found by the import mechanism. ‘box’ does a lot of work behind the scenes to ensure that box::use(./mod) does the right thing and finds the specified module relative to the currently executing code. import::from(mod.R) does absolutely nothing to ensure this: it attempts to load the path as-is, and if the module is not in the current working directory, you’re out of luck. This means that the following simple R application does not work when implemented with ‘import’.

First, here is the folder structure:

.
└── app
    ├── app.r
    └── utils.r

And here is the (working) ‘box’ implementation:

app/app.r

box::use(./utils[echo])
echo('Hello!')

app/utils.r

#' @export
echo = function (x) message('Message: ', x)

Now I can run this application from anywhere and it works, e.g. if I am in the parent directory of the app folder:

$ Rscript app/app.r

Message: Hello!

This should be nothing special. Alas, ‘import’ fundamentally does not support this. In principle the only change we should need to make is to app/app.r:

import::from(utils.r, echo)
echo('Hello!')

Attempting to execute the application fails (unless the current working directory happens to be app):

$ Rscript app/app.r

Error in loadNamespace(name) : there is no package called ‘utils.r’
Calls: <Anonymous> ... loadNamespace -> withRestarts -> withOneRestart -> doWithOneRestart
Execution halted

The ‘import’ documentation recommends using here::here to specify import paths. But, firstly, there’s no good reason for the user to have to do this work rather than ‘import’; and secondly (and more importantly!) it still fails if we replace the import::from line above with:

import::from(utils.r, echo, .directory = here::here('.'))

It fails because here::here simply does not support this use-case. It is designed for and only works inside R projects following a narrowly defined structure that is not suitable for anything except notebook-style data analysis projects.

Furthermore, in an attempt to alleviate the issue ‘import’ repeats a mistake of the source function from base R: by providing a chdir parameter (which defaults to TRUE): changing the working directory is not what we want to do, and is rarely appropriate. The working directory should be treated as user input, and left alone: the user expects to be in control of it, e.g. when providing a path relative to the current working directory as an input to the script. Consider the case where app.r expects a user-provided path to a file, and uses a nested submodule to read this file. Well, if the user provides a relative path, that submodule wouldn’t find the file because ‘import’ changed the working directory. ‘box’ modules do not have this problem.

Module documentation via ‘roxygen2’

‘box’ modules can contain ‘roxygen2’ documentation which is displayed via the box::help function. ‘import’ does not support this.

Strict module boundaries via export declarations

Information hiding is an important principle of software engineering and a core principle of all good module systems. ‘box’ modules allow distinguishing between the public interface of a module (which the user is encouraged to interact with) and its private implementation (which is hidden from the user). In particular, by default all names definied inside a module are private except those explicitly exported via an #' @export annotation.

‘import’ modules do not support information hiding: everything defined inside a module is exposed to the user, unless the author jumps through hoops by defining their own local scope.

Scope-local imports

When attaching names, box::use() always attaches them inside the narrowest local scope from which it is called, e.g. inside a function or a module. It never pollutes the global search path, unless called from the global environment.

‘import’ supports this as well — but only when using import::here() or import::from(.into = environment()), not by default for import::from().

Cyclic imports

‘box’ supports cyclic (circular) imports; that is, a module a can import a module b which in turn imports a (with some limitations). ‘import’ does not support this.

Admittedly this is not often required, many module systems in other languages do not support it (or only to a very limited extent), and there are ways to work around such a limitation. However, I have time and again run into situations where a circular module dependency drastically simplified the design, and where lack of support made the code more complex and brittle.

Other considerations

1. The ‘box’ module system is the result of years-long research of module system design. It incorporates many learnings from the design discussions of other state-of-the-art module systems (particularly JavaScript, Rust and Python). Every aspect of the module behaviour in ‘box’ is the result of a careful and conscious decision.

   From the outside it doesn’t seem as if the same rigour went into the design of the ‘import’ module system. I admit that this impression may be wrong but I find it hard to reconcile several glaring issues (see above) with a rigorous design.

2. box::use provides a full-featured eDSL to declare imports, which leads to better readability. “Better” in this context is obviously subjective but the box::use declaration syntax is the result of several iterations with user interviews and usability analysis. ‘import’ is hindered by the lack of a dedicated DSL and needs to provide multiple distinct import functions (from, here, into) where one should suffice, and options via arguments, which start with . to disambiguate them from imported names. Some people describe this kind of non-functional syntax as syntactic salt, which expresses the fact that (unlike syntactic sugar) the user has to deal with irrelevant details that a more expressive syntax would take care of. Good notation is valuable.

---

¹ I need to carefully qualify this. For example, ‘import’ allows specifying a target import environment which is attached to the search path. ‘box’ doesn’t. However, the reason ‘import’ provides this is because it is required to work around limitations in how R looks up imports from packages. ‘box’ circumvents this problem differently, by avoiding to pollute the global search path entirely via scope-local imports (see above).

Another limitation is that ‘box’ currently does not (yet) provide an equivalent to the .except parameter from ‘import’. It also quite intentionally does not support anything like the .character_only parameter, but a future version might support value injection with the tidy evaluation syntax.