Goals for the backends

[digikar99]

Hi!

I have been wanting to separate out the backend parts from the high-level parts, so that the same backend could be used by multiple high level numerical libraries - magicl, numcl, petalisp, numericals and more, each with their own ever-so-differing high level goals and approaches.

I currently have CL-CBLAS and CL-BMAS, and was comparing CL-CBLAS with MAGICL's BLAS backend. One of the differences between the CL-AUTOWRAP generated CBLAS:DCOPY and MAGICL.BLAS-CFFI::%%DCOPY I noted is that the latter has all parameters as pointers, whereas the former has arrays as pointers, but the n and strides as integers. Is there a reason for this decision?

In addition, is it feasible to separate out the backend into a separate system/package independent of the high-level interface?

[stylewarning]

In what way are you interested in separating the backends from the frontend? The frontend is in magicl/core, which is just plain ol' Lisp code. Indeed, it does have Lisp generic functions which the backends "hook" into in order to install their own code. The backends are all derived from functions implemented in magicl/ext-* systems. One can use those directly if one pleases; magicl/core isn't necessary at all. Having them as separate systems was done to promote modularity. Though it was never anticipated that we might have more than one frontend, I wouldn't say the currently architecture forbids it.

Would you want to excise magicl/core just for space economy (i.e., not compiling code you don't need)?

In principle we could have two systems per extension:

• magicl/core, which contains all the hooks
• magicl/blas, which just contains the BLAS stuff without hooking into magicl/core
• magicl/ext-blas, which would do the work of hooking magicl/blas into magicl/core.

I'd only want to do this if there would be actual users that would profit from such a structure. Otherwise, at least as it pertains to the MAGICL project, it's over-modularizing without much purpose.

[digikar99]

Sorry for the delayed response, been busy with IRL things!

Would you want to excise magicl/core just for space economy (i.e., not compiling code you don't need)?

Both space economy, or put another way, clean dependencies from the POV of community, where you don't load what you do not use. And yes, the idea of two systems per extension is what seems ideal to me - not even magicl/blas but just blas. This blas system would live in a separate repository, as also separate repositories for lapack and expokit; while magicl/ext-blas magicl/ext-lapack, magicl/ext-expokit that contains code that hooks blas into magicl/core would continue to live here.

(EDIT: As far as space economy itself is concerned, it doesn't really matter, a tar.gz SBCL base image is about 10 MB, with magicl-core, it is 12 MB; and with complete magicl, it is 19MB. So, the task of separating out the systems is more of an aesthetic rather than anything of practical significance.)

I'd only want to do this if there would be actual users that would profit from such a structure. Otherwise, at least as it pertains to the MAGICL project, it's over-modularizing without much purpose.

Agree there. Currently, other than magicl, only at most one other user (library developers) is interested in this. It would make more sense to undertake this only if there are other developers (+ possibly end users of those libraries) interested in this approach.