Future of VegaFusion and Altair

[jonmmease]

Now that Altair 5 is out (thanks again to @mattijn @joelostblom and @binste for all of the work to make this a reality!), I wanted to start a discussion on the future relationship between VegaFusion and Altair. This is also in response to @joelostblom's question in #3053.

Even though VegaFusion is a cool piece of technology, it's not really something an end user of Altair should need to think about. From my perspective, It would be great if the capabilities added by VegaFusion were just seen as capabilities of Altair itself to the casual user.

Today, the VegaFusion Python library consists of two parts. A low-level runtime that deals exclusively with dictionaries of Vega specs and has no dependency on Altair (e.g. vf.runtime.pre_transform_spec). And higher level entry points that integrate with Altair using vl-convert (e.g. the mime renderer, save functions, and transformed_data function).

A question for the Altair community is whether we would be interested in making VegaFusion an optional dependency and then port this high-level functionality into Altair itself.

Example Integrations

Here are a couple of examples of what the more integrated functionality could look like:

The existing renderers could get a vegafusion kwarg to control whether serverside transformation is used (and it could default to True if VegaFusion is installed).

alt.renderers.enable("mimetype", vegafusion=True)

or for the VegaFusion widget renderer

alt.renderers.enable("widget").

Similarly, the Chart.save method could have a vegafusion kwarg to control whether transformations are applied before saving the chart:

Chart.save("chart.html", vegafusion=True)

Charts could have a .transformed_data() method alongside their existing .data property to access the chart's data post transformation:

chart.transformed_data()

And the vegafusion[embed] package specification could be added to an extra_requires group to install automatically with pip install altair[full].

Always optional

As with vl-convert, it's important that a VegaFusion dependency always be optional since VegaFusion is a compiled library and won't be available everywhere that Altair currently works (e.g. in pyodide).

Development and Maintenance

If we want to go this direction, I'd be happy to do the bulk of the integration work and to join as an Altair maintainer.

[joelostblom]

From my perspective, It would be great if the capabilities added by VegaFusion were just seen as capabilities of Altair itself to the casual user.

I agree with this, I think it is important that the out of the box experience includes the most crucial conveniences for users and I think smaller sized charts by default is one of these and aligns closely with what the user would likely expect when making a chart (i. e. only the data required to display the chart is included in the spec unless explicitly instructed to include all data)

The existing renderers could get a vegafusion kwarg to control whether serverside transformation is used (and it could default to True if VegaFusion is installed).

I think this sounds like a good solution to me. I wonder if the kwarg should be called vegafusion or something more descriptive like transform_data (but it might be hard to find a name that encompasses everything that vegafusion does and that might also evolve over time, so vegafusion might be the safest choice).

A question for the Altair community is whether we would be interested in making VegaFusion an optional dependency and then port this high-level functionality into Altair itself... If we want to go this direction, I'd be happy to do the bulk of the integration work and to join as an Altair maintainer.

On the top of my head, I would be in favor of this. Could you give a rough indication of the amount of code that would be integrated into Altair, is it a few hundred lines or thousands? And thank you for offering to do the maintenance of this part of the code for the library, I personally think that sounds like the ideal solution.

And the vegafusion[embed] package specification could be added to an extra_requires group to install automatically with pip install altair[full].

I think this would be great, and having a similar conda package would be convenient as well, but I am not sure if conda supports this functionality now or if we would need to create a meta package.

As with vl-convert, it's important that a VegaFusion dependency always be optional since VegaFusion is a compiled library and won't be available everywhere that Altair currently works (e.g. in pyodide).

Thanks for noting this, makes sense!

[jonmmease]

Thanks for taking the time to consider and respond @joelostblom!

Could you give a rough indication of the amount of code that would be integrated into Altair, is it a few hundred lines or thousands?

I'd estimate around 500 lines of Python, plus tests. It would be a subset of the logic in https://github.com/hex-inc/vegafusion/tree/main/python/vegafusion/vegafusion.

[joelostblom]

And a follow up question, if we go this route, should the VegaFusion widget also be part of the extra_requires dependencies that are installed with altair[full] or is it meaningful to keep it as a separate package like the data server is kept separately now.

[jonmmease]

In the near term, it probably makes sense to keep vegafusion-jupyter as a separate package, and I could see including it in altair[full].

[binste]

I've been using VegaFusion for a while now, mainly the mimetype renderer, and i also find it a very very "cool piece of technology"! :) It would be great to have it tighter integrated into Altair so that many more users can reap the benefits of this. +1 on everything said so far (with the caveat regarding enabling it by default, see below) and thank you @jonmmease for offering to do the integration! The example integration sounds good to me.

I'm wondering how we can enable the mimetype renderer by default if VegaFusion is installed without breaking too much. I think it's definitely worth trying and it's something to be looked at during the implementation. But just as an example, let's say an application creates an Altair chart, converts it to json with chart.to_json(), and then passes that Vega-Lite spec to a frontend so it can be rendered (think of e.g. Streamlit). With VegaFusion enabled, the json produced by chart.to_json() is currently still a VL spec but data is replaced with a vegafusion dataset URI which is no longer a self-contained VL spec that can be rendered. If VegaFusion would do the full transformation it would need to return a Vega spec which I'd find an unexpected behavior. Maybe a global "off" switch is enough with a future warning for a few releases if chart.to_json/to_dict() is used without explicitly specifying the vegafusion argument. I think it's definitely worth trying, would be great if it can be done!

In any case, very much looking forward to this! Happy to support with the implementation and review.

[jonmmease]

Thanks for the kind words @binste!

Yeah, using VegaFusion by default may not be a good idea (especially to start with). I think having it be an option that you turn on (either globally or at the renderer level) would be a great start. Describing how to enable it in the MaxRowsError message would still make it pretty discoverable for those who would benefit!

[jonmmease]

@mattijn, do you have thoughts on the topic?

[joelostblom]

+1 to that plan! In terms of a place in the docs, I think it would be useful to add a section to the data transformation landing page on how to view transformed data. That section could start with a brief example of doing it with only Altair by creating a table via mark_text, and then show that it is much easier to have VegaFusion installed since you then can access transformed_data and essentially include the section you already have in the VegaFusion docs plus some more advanced example of what can be done now that we can programatically access the data. What do you think about that?

Another spot could be the "Data transforms" section, and I think maybe we should add something there about the new vegafusion transform options regardless.

To make the vegafusion advantages more discoverable, I think we should add sentences in 1-2 places in the docs that advertise these new sections (both for the transformed data and the fact that vf makes it easier to work with large datasets, but "by default" and via the widget if needed)

[binste]

+1 as well, having .transformed_data would be very useful and should not break anything for users so it's a great start. Looking forward to it and happy to review! And great to have you @jonmmease onboard as a maintainer, welcome :)

[jonmmease]

Let me also add you as maintainer to the altair-viz in case that makes things easier for the integration activities.

Thanks @mattijn, I appreciate the trust.

Thanks for the documentation suggestions @joelostblom

I'll keep you all posted as I start making progress on this!

[jonmmease]

Initial transformed_data PR: #3081

[jonmmease]

merged, with Chart._transformed_data() as a private method for now. I'll start working on documentation updates over the next few days.

[jonmmease]

I've started playing some with how to integrate VegaFusion into the Altair rendering pipeline, and I think I have a pretty clean solution to propose.

First, add a new data transformer named "vegafusion". This data transformer will extract the dataframe-like data sources into a private WeakValueDictionary and replace them with special URLs that VegaFusion understands (basically "table://{uuid}").

Other parts of Altair will check whether the active data transformer is named "vegafusion", and if so they will:

1. compile the vega-lite spec to vega
2. retrieve the extracted data sources form the private dictionary above
3. Invoke VegaFusion's pre_transform_spec method with the Vega spec and the inline datasets to produce a new Vega spec with data transformations applied

(In practice this will all be wrapped up in a utility method)

So far, I've thought of three places to insert this logic:

1. When rendering a chart (using any built-in renderer)
2. When saving a chart (to any supported format)
3. When converting a chart to a dictionary (to_dict) or JSON (to_json) if the format argument is set to "vega". (If the format is "vega-lite", then leave as-is).

A nice consequence of this approach is that a user will only need to run:

alt.data_transformers.enable("vegafusion")

And then Altair will automatically use VegaFusion everywhere it makes sense.

Let me know what you think! If this general direction makes sense I'll work on a PR and we can discuss more there. Thanks!

[mattijn]

Sounds good! I like the approach of using an altair.data_transformer() here and I understand that you will have to introduce some logic at a few places to accommodate for this data-transformer.