Render VLC content in tauri window

[hellozyemlya]

So, I trying to use vlc-rs to create some tauri based video player that can handle many formats. Problem here - vlc takes ns_object to render content on, and obviously doing its rendering completely outside of any tauri lifecycle, and if I just feed webview window nsobject to vlc, it will just overdraw everything tauri rendered.

I tried another approach - create two separate windows:

// Prevents additional console window on Windows in release, DO NOT REMOVE!!
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]

use std::sync::Mutex;

use tauri::{LogicalPosition, LogicalSize, Manager, WebviewUrl};
use vlc::{Instance, Media, MediaPlayer};

// Learn more about Tauri commands at https://tauri.app/v1/guides/features/command
#[tauri::command]
fn greet(name: &str) -> String {
    format!("Hello, {}! You've been greeted from Rust!", name)
}

fn main() {
    tauri::Builder::default()
        .setup(|app| {
            let width = 800.;
            let height = 600.;
            let window = tauri::window::WindowBuilder::new(app, "root")
                .inner_size(width, height)
                .build()?;

            let video_view = tauri::window::WindowBuilder::new(app, "video")
                .inner_size(width, height)
                .parent(&window).unwrap()
                .build()?;

            let instance = Instance::new().unwrap();
            let md = Media::new_path(&instance, "/sample.mkv").unwrap();
            let mdp = MediaPlayer::new(&instance).unwrap();
            mdp.set_nsobject(video_view.ns_view().unwrap());
            mdp.set_media(&md);
            mdp.play().unwrap();
            app.manage(Mutex::new(mdp));

            let main_view = window.add_child(
                tauri::webview::WebviewBuilder::new("content", WebviewUrl::App(Default::default()))
                    .auto_resize(),
                LogicalPosition::new(0., 0.),
                LogicalSize::new(width, height),
            )?;
            Ok(())
        })
        .invoke_handler(tauri::generate_handler![greet])
        .build(tauri::generate_context!())
        .expect("error while building tauri application")
        .run(|app_handle, event| {
        });
}

but I want to have video as a background, and webview as an overlay to video. Is there some ways to achieve that?
I guess I need two separate nsobjects in single window, one for video another for webview, but is that possible to achieve it?
Or maybe it is possible to create 1 root window, and two child windows that will be directly "embedded" to one root window and properly rendered?

Thanks in advance, tauri is a grate project !

[hellozyemlya]

Found out I can do following thing:

// Prevents additional console window on Windows in release, DO NOT REMOVE!!
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]

use std::sync::Mutex;

use tauri::{LogicalPosition, LogicalSize, Manager, Url, WebviewUrl};
use vlc::{Instance, Media, MediaPlayer};

// Learn more about Tauri commands at https://tauri.app/v1/guides/features/command
#[tauri::command]
fn greet(name: &str) -> String {
    format!("Hello, {}! You've been greeted from Rust!", name)
}

fn main() {
    tauri::Builder::default()
        .setup(|app| {
            let width = 800.;
            let height = 600.;
            let window = tauri::window::WindowBuilder::new(app, "root")
                .inner_size(width, height)
                .build()?;

            // let video_view = tauri::window::WindowBuilder::new(app, "video")
            //     .inner_size(width, height)
            //     .parent(&window).unwrap()
            //     .build()?;

            let video_view = window.add_child(
                tauri::webview::WebviewBuilder::new("video", WebviewUrl::External(Url::parse("https://google.com").unwrap()))
                    .auto_resize(),
                LogicalPosition::new(0., 0.),
                LogicalSize::new(width, height),
            )?;
            let instance = Instance::new().unwrap();
            let md = Media::new_path(&instance, "/sample.mkv").unwrap();
            let mdp = MediaPlayer::new(&instance).unwrap();
            mdp.set_nsobject(video_view.window().ns_view().unwrap());
            mdp.set_media(&md);
            mdp.play().unwrap();
            app.manage(Mutex::new(mdp));

            let main_view = window.add_child(
                tauri::webview::WebviewBuilder::new("content", WebviewUrl::App(Default::default()))
                    .transparent(true)
                    .auto_resize(),
                LogicalPosition::new(0., 0.),
                LogicalSize::new(width, height),
            )?;
            Ok(())
        })
        .invoke_handler(tauri::generate_handler![greet])
        .build(tauri::generate_context!())
        .expect("error while building tauri application")
        .run(|app_handle, event| {});
}

but it does not help, vlc overdraw everything.

[hellozyemlya]

I suspect the way how vlc renders in ns_view is not very friendly to other ns_views...

[dank074]

I suspect the way how vlc renders in ns_view is not very friendly to other ns_views...

You are creating two webviews, not two windows. Webviews are rendered on the provided window, so what your code is actually doing is rendering the two webviews, plus VLC, on the same window.

You need to create 2 windows using tauri::window::WindowBuilder::new, parent and child. Make sure you set the .parent property for the child window in the builder. Render VLC on the parent, and on the child window render the webview by doing child_window.add_child(webview builder here), this way, the webview will have a higher z than the video and it will render on top of it. You will also have to add an event listener to parent to move/resize the child every time parent is moved/resized.