Appification / Final Push

[tychedelia]

This code is probably going to be pretty impossible to review in the diff. Highly suggest checking it out and just playing around with it.

Changes

Draw materials

The Draw instance is now generic over a bevy material. By default, we expose a DefaultNannouMaterialthat is what most users will interact with. This material is a material extension over the bevy StandardMaterial, which is their default PBR material and contains a variety of useful properties that users can now use.

Rendering materials

The rendering algorithm is significantly complicated by making Draw generic. Every time a user mutates their drawing in a way that requires a new material, this requires storing that material instance within our draw state. Further, we support changing that material's type (as described below) for any given draw instance or drawing, which transitions the type of the draw instance from Draw<M> to Draw<M2>. Because these cloned / mutated draw instances still point to the same "logical" draw, we need a way to type erase materials within our draw state.

This is accomplished in the following manner:

• We store a last_material within state that tracks the id of the last known material, which works similarly to tracking context.
• Every time a new drawing is started, a None slot is added to draw_commands which can be used if the material is later changed for that drawing.
• If last_material has changed, we push a DrawCommand::Material(UntypedAssetId). Here, UntypedAssetId is the mechanism through which we achieve type erasure. Inside the draw state, we store a HashMap<UntypedAssetId, Box<dyn Any + Send + Sync>>, which contains all the materials that are used in our drawing and will be downcast later when processed for rendering.
• Inside an individual Drawing, we now store a copy on write pointer type DrawRef that can either be a reference to a parent draw instance OR a owned clone of that draw instance with a different material parameter. The state of ref is used to determine on drop whether we mutated the material in our drawing, and thus whether a new material needs to be inserted into draw_commands at the index slot we created in advance when the drawing was created.
• In a separate bevy system that runs after our drawing, we iterate through the materials stored in State and try to downcast them into a given registered material type. We then add the material as an asset to the bevy world so it can be used by our mesh rendering system.
• When rendering a drawing, we iterate through the stack of draw commands and use our material command as a signal to render into a new mesh. What this means in practice is that if a user never switches materials by calling a mutating material method, their entire drawing will render into the same mesh. In other words, the number of meshes == the number of materials.

Custom materials

A variety of new possibilities are opened up by allowing users to create their own materials. The bevy Material allows easy access to writing a vertex or fragment shader for a given drawing, which means user's can easily include any arbitrary data they want when writing a custom shader for a given drawing primitive. What this means in practice is that rather than writing manual wgpu code, the user will write something like the following in order to render a fullscreen shader:

let win = app.window_rect();
    draw
        .material(CustomMaterial::default())
        .rect()
       .x_y(win.x(), win.y());

Removal of Texture drawing primitive

Previously, a texture was rendered in nannou using a special primitive that instructed the fragment shader to sample a given texture. This has primitive has been removed. Users now should use the material method texture that sets the bevy material's texture to a given Handle<Image>, which represents an asset handle to a wgpu texture. Similarly to above, this means that you now will render a simple texture as a `rect:

use nannou::prelude::*;

fn main() {
    nannou::app(model).run();
}

struct Model {
    texture: Handle<Image>,
}

fn model(app: &App) -> Model {
    app.new_window().size(512, 512).view(view).build();
    let texture = app.assets().load("images/nature/nature_1.jpg");
    Model { texture }
}

fn view(app: &App, model: &Model) {
    let draw = app.draw();
    draw.background().color(BLACK);
    let win = app.window_rect();
    draw.rect().x_y(win.x(), win.y()).texture(&model.texture);
}

In the model function, we now use bevy's AssetLoader exposed via the world in order to load images as an asset handle. These handles can then be used to load an image into cpu memory to do things like configure the given sampler for an image. In general, this provides a lot more flexibility and should allow users to do more powerful things with textures without having to drop into our "points" API.

Mesh UVs

In order to render textures and do anything interesting with custom shaders, it is now important that every drawing primitive has texture coordinates, and these are a required field in our vertex layout. Drawing primitives have been changed to provide a set of default UVs where sensible. Additionally, a points_vertex method has been added to our "points" API that allows the user to provide explicit UVs in additional to position and color data. Our "full" vertex type is now always (Vec2, Color, Vec2).

Additionally, a new SetTexCoords property trait has been added that allows users to change the UVs for a given drawing primitive. For example, our rect has a sensible [0.0, 0.0] to [1.0, 1.0] set by default, but users may want to sample from a smaller part of a texture:

let area = geom::Rect::from_x_y_w_h(0.5, 0.5, 1.0, 1.0);
draw.rect()
            .x_y(x2, y2)
            .w_h(w, h)
            .texture(&texture)
            .area(area);

The area method (final name tbd) here of SetTexCoords allows providing a geom::Rect that updates the UVs. In this way, while we make a best effort to provide sensible default values, users can override those values which may be particularly useful for things like triangles or circles, etc.

There are still a few paths where we may fall back to a default Vec2::ZERO, but this can hopefully be eliminated over time. We also still provide the points_colored API for instances where users truly don't care about their texture coordinates, but anyone working with textures or custom shaders will need to provide their own.

App changes

Our App instance now wraps bevy's world as a Rc<RefCell<UnsafeWorldCell<'w>>>. This interior mutability allows us to access potentially mutating methods on the bevy world without having to use &mut Self on our App instance.

App methods

All of App's methods now borrow our internal reference to world and return that data to a user. Sometimes, this means returning an instance of a bevy resource directly (e.g. Time or AssetLoader). Due to the use of interior mutability, it's theoretically possible for the user to cause a panic, although I have not experienced that yet. We'll want to be careful and watch to ensure that this isn't easy to do.

Creating App in internal systems

Because we store our nannou bookkeeping state (i.e. windows, event handler function pointers, the user's model) in the bevy world, there's a bit of unsafety and complexity around how to then provide an instance of App to the user that wraps that world.

Each system that runs a given event handler declares the dependencies it requires to be extracted from the world before creating the App instance to pass to the user's handler. For example:

#[allow(clippy::type_complexity)]
fn key_events<M>(
    world: &mut World,
    state: &mut SystemState<(
        EventReader<KeyboardInput>,
        Query<&WindowUserFunctions<M>>,
        NonSendMut<M>,
    )>,
) where
    M: 'static

This SystemState is then used to unsafely extract the state and create an App instance:

fn get_app_and_state<'w, 's, S: SystemParam + 'static>(
    world: &'w mut World,
    state: &'s mut SystemState<S>,
) -> (App<'w>, <S as SystemParam>::Item<'w, 's>) {
    state.update_archetypes(world);
    let app = App::new(world);
    let param = unsafe { state.get_unchecked_manual(*app.world.borrow_mut()) };
    (app, param)
}

This is dangerous! The primary invariant we must uphold here is that we never expose methods on App that would allow mutable reference to the state we extract from world here. In many cases this is fine, because we are extracting internal nannou state that user's don't need to know about, but is a potential source of soundness issues.

TODO:

- [ ] Figure out how to interlace draw cmds where primitives may mutate their own material.
* Actually this is okay. If a drawing changes it's material, we probably do want it to have a higher z-value as well as reset the mesh/material of its parents. I think this behavior makes more sense atm.

• Add all material methods.
• Ensure UVs are correctly calculated for all primitives.
• Add new color spaces.
• Gate nightly features. (https://github.com/dtolnay/rustversion?)
• Feature flag egui deps.
  - [ ] Document / address send bounds?
  • Have removed this for now
• Document / fix UpdateMode examples.