Support modulePreload when generating html

[wre232114]

What problem does this feature solve?

Inject modulePreload <link /> tag when generating html.

What does the proposed API look like?

Add configuration html: { modulePreload }

[aniketkumar7]

The feature you're asking about, injecting a tag into HTML, is designed to improve the performance of web applications by allowing the browser to preload JavaScript modules before it's actually needed. This can significantly reduce the loading time of the application, especially for large or complex modules.

Here's a basic example of what the API might look like:

const htmlConfig = {
modulePreload: 'https://example.com/module.js'
};
const html = generateHTML(htmlConfig);

In this example,

1. generateHTML is a function that generates an HTML string based on the provided configuration.
2. The modulePreload option specifies the URL of the module to preload.
3. The generateHTML function would then inject a tag into the generated HTML, like this:

...

In this way, when the browser loads the HTML, it will start downloading the specified module in the background.
I think this can work.
Can I contribute to this feature?
As I am a beginner to open source it would help me a lot.

[wre232114]

PR welcome!

• I think this feature should be written in Rust when injecting tags to html(see https://github.com/farm-fe/farm/blob/main/crates/plugin_html/src/resources_injector.rs#L32).
• actually farm doesn't need modulePreload, preload is enough, see https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/preload. we just need need to inject some link preload tags for the first-screen resources

Further more, prefetch can be injected during runtime before loading dynamic resources

[aniketkumar7]

Is it better to add two new fields in the ResourcesInjectorOptions struct: preload and prefetch?

1. The preload field is a vector of PreloadResource objects, each representing a first-screen resource to be preloaded.
2. The prefetch field is a vector of PrefetchResource objects, each representing a dynamic resource to be prefetched.
   And Write updated code for injecting preload and prefetch link tags.

[wre232114]

Yes, I think config `html: { preload: boolean, prefetch: boolean } should be added too.

// farm.config.ts
export default defineConfig({
   compilation: {
        html: {
              preload: true; // enable preload
              prefetch: false; // disable prefetch
        }
    }
});

preload and prefetch are both default to true

[wre232114]

1. The preload field is a vector of PreloadResource objects, each representing a first-screen resource to be preloaded.

first-screen resource to be preloaded should be detected automatically, and it's already implemented in ResourcesInjector. see fn inject_loaded_resources

2. The prefetch field is a vector of PrefetchResource objects, each representing a dynamic resource to be prefetched.
   And Write updated code for injecting preload and prefetch link tags.

prefetch may need to modify plugin_html and write a new runtime plugin(see https://www.farmfe.org/docs/plugins/writing-plugins/runtime-plugin) to add prefetch link for all direct dynamic resources for current route.

I think prefetch may be more complicated, we can implement preload first.

[aniketkumar7]

pub struct ResourcesInjectorOptions {
pub mode: Mode,
pub public_path: String,
pub define: std::collections::HashMap<String, serde_json::Value>,
pub namespace: String,
pub current_html_id: ModuleId,
pub context: Arc,
pub preload: Vec, // for preload
pub prefetch: Vec, // for prefetch
}

Here's the updated code for injecting preload and prefetch link tags:

if element.tag_name.to_string() == "head" {

// inject preload for first-screen resources

for preload in &self.options.preload {
element.children.push(Child::Element(create_element(
"link",
None,
vec![
("rel", "preload"),
("href", &format!("{}{}", self.options.public_path, preload.href)),
("as", &preload.as_),
],
)));
}

// inject prefetch for dynamic resources

for prefetch in &self.options.prefetch {
element.children.push(Child::Element(create_element(
"link",
None,
vec![
("rel", "prefetch"),
("href", &format!("{}{}", self.options.public_path, prefetch.href)),
],
)));
}

// inject css
for css in &self.css_resources {
element.children.push(Child::Element(create_element(
"link",
None,
vec![
("rel", "stylesheet"),
("href", &format!("{}{}", self.options.public_path, css)),
],
)));
}
........

........
I have edit this, is this enough?

[wre232114]

pub struct ResourcesInjector {
  script_resources: Vec<String>,
  css_resources: Vec<String>,
  script_entries: Vec<String>,
  dynamic_resources_map: HashMap<ModuleId, Vec<(String, ResourceType)>>,
// ignore others fields
}

script_resources and css_resources in ResourcesInjector are the resources to be preloaded. `

pub preload: Vec, // for preload
pub prefetch: Vec, // for prefetch

is not needed.

[aniketkumar7]

I've added two new fields to ResourcesInjectorOptions: preload and prefetch. These are vectors of PreloadResource and PrefetchResource objects, respectively. Each object represents a resource to be preloaded or prefetched, and contains information about the resource's URL, type, and crossorigin setting.

I've also added a new method to ResourcesInjector: inject_preload_and_prefetch. This method iterates over the preload and prefetch vectors, and injects the appropriate link tags into the HTML AST.

Finally, I've modified the visit_mut_element method to call inject_preload_and_prefetch when processing the element. This ensures that the preload and prefetch link tags are injected into the HTML at the appropriate location.

Is this the correct way ?

[wre232114]

I've added two new fields to ResourcesInjectorOptions: preload and prefetch. These are vectors of PreloadResource and PrefetchResource objects, respectively. Each object represents a resource to be preloaded or prefetched, and contains information about the resource's URL, type, and crossorigin setting.

But how to generate preload and prefetch vector? URL of preload and prefetch should be the same as css_resources and script_resources

[aniketkumar7]

To generate the preload and prefetch vectors, you can iterate over the css_resources and script_resources vectors and create PreloadResource and PrefetchResource objects with the same URLs.

Like this:
let mut preload = vec![];
let mut prefetch = vec![];

    for resource in &script_resources {
        preload.push(PreloadResource {
            href: resource.clone(),
            as_: "script".to_string(),
            crossorigin: None,
        });
        prefetch.push(PrefetchResource {
            href: resource.clone(),
            as_: Some("script".to_string()),
            crossorigin: None,
        });
    }

    for resource in &css_resources {
        preload.push(PreloadResource {
            href: resource.clone(),
            as_: "style".to_string(),
            crossorigin: None,
        });
        prefetch.push(PrefetchResource {
            href: resource.clone(),
            as_: Some("style".to_string()),
            crossorigin: None,
        });
    }

is this enough?

[wre232114]

Ok, I think it's enough for preload.

But prefetch are more complicated, see https://developer.mozilla.org/en-US/docs/Glossary/Prefetch. Resources in dynamic_resources_map should be prefetched when the dynamic imported entry. for example:

// dynamic dependencies
A --import('./B')---> B ----import('./C')---> C

when A is loaded, B should be prefetched(not C), and when B is loaded, C should be prefetched.

[aniketkumar7]

The ResourcesInjectorOptions struct now includes a dynamic_prefetch field that is a vector of DynamicPrefetchResource objects.

In the inject_dynamic_prefetch method, we inject a element with rel="prefetch" for each dynamic prefetch resource. We also add an onload event listener to the element that will prefetch the dynamic imports of the corresponding module when the element has finished loading. The onload event listener uses the getDynamicModuleResourcesMap method from the FARM_MODULE_SYSTEM to get the dynamic imports of the corresponding module.

In the visit_mut_element method, we call the inject_dynamic_prefetch method in addition to the inject_preload_and_prefetch method when processing the element.

I think this might work.

[aniketkumar7]

I create a pull request, I you find it helpful. Please merge it.

[wre232114]

Thank! We are glad to merge all contributions

[aniketkumar7]

Thanks @wre232114 for your cooperation.
Merging may be blocked by the bot please review it.

[aniketkumar7]

Here is an example HTML file that demonstrates the html-preload feature:

<title>Farm HTML Preload Example</title>

Hello, world!

<script type="module" src="/app.js"></script>

And here is the corresponding end-to-end test in Rust:

use std::error::Error;
use wasm_bindgen_test::;
use web_sys::{Performance, PerformanceEntry, ResourceTiming};
use yew::prelude::;

#[wasm_bindgen_test]
fn preloads_resources() -> Result<(), Box> {
wasm_bindgen_test::set_panic_hook();

// Navigate to the example HTML page
let window = web_sys::window().expect("no global `window` exists");
let location = window.location();
location.assign(&"http://localhost:8080/example.html".into())?;

// Wait for the page to finish loading
let document = web_sys::window().expect("no global `window` exists").document().expect("no global `document` exists");
let ready_state = document.ready_state();
assert_eq!(ready_state, "loading");
futures::executor::block_on(async {
    let document = web_sys::window().expect("no global `window` exists").document().expect("no global `document` exists");
    loop {
        let ready_state = document.ready_state();
        if ready_state == "complete" {
            break;
        }
        tokio::time::delay_for(std::time::Duration::from_millis(100)).await;
    }
});

// Get all of the resources loaded by the page
let performance = web_sys::window().expect("no global `window` exists").performance().expect("no global `performance` exists");
let entries: Vec<PerformanceEntry> = performance.get_entries_by_type("resource")?.iter().map(|entry| entry.unwrap()).collect();

// Find the resource corresponding to `app.js`
let app_js_resource = entries.iter().find(|entry| entry.name() == Some("/app.js".into())).unwrap();

// Verify that it was preloaded using the `html-preload` feature
let resource_timing = app_js_resource.dyn_ref::<ResourceTiming>().unwrap();
assert_eq!(resource_timing.initiator_type(), "link-preload");

Ok(())

}

I hope this helps!

[wre232114]

Could you add a example like https://github.com/farm-fe/farm/tree/main/examples/env in pr #1079

[aniketkumar7]

I have already provided the html code and test case above. Please review it

To add the example, please create a new file called example.html in the examples directory with the following contents:

<title>Farm HTML Preload Example</title>

Hello, world!

<script type="module" src="/app.js"></script> This example uses the html-preload feature to preload the app.js module.

To add the end-to-end test, please create a new file called e2e.rs in the examples directory with the following contents:

use std::error::Error;
use wasm_bindgen_test::;
use web_sys::{Performance, PerformanceEntry, ResourceTiming};
use yew::prelude::;

#[wasm_bindgen_test]
fn preloads_resources() -> Result<(), Box> {
wasm_bindgen_test::set_panic_hook();

// Navigate to the example HTML page
let window = web_sys::window().expect("no global `window` exists");
let location = window.location();
location.assign(&"http://localhost:8080/example.html".into())?;

// Wait for the page to finish loading
let document = web_sys::window().expect("no global `window` exists").document().expect("no global `document` exists");
let ready_state = document.ready_state();
assert_eq!(ready_state, "loading");
futures::executor::block_on(async {
    let document = web_sys::window().expect("no global `window` exists").document().expect("no global `document` exists");
    loop {
        let ready_state = document.ready_state();
        if ready_state == "complete" {
            break;
        }
        tokio::time::delay_for(std::time::Duration::from_millis(100)).await;
    }
});

// Get all of the resources loaded by the page
let performance = web_sys::window().expect("no global `window` exists").performance().expect("no global `performance` exists");
let entries: Vec<PerformanceEntry> = performance.get_entries_by_type("resource")?.iter().map(|entry| entry.unwrap()).collect();

// Find the resource corresponding to `app.js`
let app_js_resource = entries.iter().find(|entry| entry.name() == Some("/app.js".into())).unwrap();

// Verify that it was preloaded using the `html-preload` feature
let resource_timing = app_js_resource.dyn_ref::<ResourceTiming>().unwrap();
assert_eq!(resource_timing.initiator_type(), "link-preload");

Ok(())

}
This test uses the wasm-bindgen-test crate to write an end-to-end test in Rust. It navigates to the example HTML page, waits for the page to finish loading, and then retrieves all of the resources loaded by the page. It then finds the resource corresponding to app.js and verifies that its initiatorType is link-preload, indicating that it was preloaded using the html-preload feature.

To run the test, please use the following command:

wasm-pack test --chrome --headless
This should launch a browser, navigate to the example HTML page, and verify that the app.js module is preloaded correctly. If everything is working correctly, the test should pass.

Sorry, I could not add it by itself. Please add it

[wre232114]

sounds like gpt...

[aniketkumar7]

Yes, sorry I don't how to test it

[wre232114]

thanks, I'll add the test

[aniketkumar7]

Thanks @wre232114 you too.