Skip to content

Commit

Permalink
feat: <EntitySelector> component to modify specific entities (#238)
Browse files Browse the repository at this point in the history
  • Loading branch information
hannojg authored Jul 18, 2024
1 parent e11899b commit 865aec6
Show file tree
Hide file tree
Showing 15 changed files with 297 additions and 58 deletions.
2 changes: 1 addition & 1 deletion docs/docs/guides/CAMERA.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ import { Camera } from 'react-native-filament'

<Camera
position={[0, 0, 10]}
target={[0, 0, 0]}
target={[0, 0, 0]} // "Look at" point
/>
```

Expand Down
118 changes: 117 additions & 1 deletion docs/docs/guides/IMAGES.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,120 @@ sidebar_label: Images
slug: /guides/images
---

Documentation is added soon!
import useBaseUrl from '@docusaurus/useBaseUrl'

<div class="image-container">
<svg xmlns="http://www.w3.org/2000/svg" width="283" height="535">
<image href={useBaseUrl("img/demo-backgroundimage.jpg")} x="18" y="33" width="247" height="469" />
<image href={useBaseUrl("img/frame.png")} width="283" height="535" />
</svg>
</div>

There are two types of images to consider:

- Background images
- Images that will render within the 3D scene. Applied on a 3D shape. Often called a texture.

## Background images

You can only have one background image per scene. The background image is a 2D image that is rendered behind the 3D scene. It is not affected by the camera or lights and will
always fill the whole view.

```tsx
import { BackgroundImage, Camera, DefaultLight } from 'react-native-filament'
import BackgroundImageMaterial from './assets/background_image.filamat'

const imageResource = require('./assets/background.jpg')

function Renderer() {
return (
<FilamentView style={styles.container}>
<Camera />
<DefaultLight />

<BackgroundImage source={imageResource} materialSource={BackgroundImageMaterial} resizeMode="cover" />
</FilamentView>
)
}
```

### Materials

As you may noticed, we imported a `.filamat` file and passed it as a prop to the `BackgroundImage` component. A material in filament "[defines the visual appearance of a surface.](https://google.github.io/filament/Materials.html#overview/coreconcepts)".
To completely describe and render a surface, a material provides the following information:

- [Material model](https://google.github.io/filament/Materials.html#materialmodels)
- Set of use-controllable named parameters
- [Raster state](https://google.github.io/filament/Materials.html#materialdefinitions/materialblock/rasterization:culling) (blending mode, backface culling, etc.)
- [Vertex shader code](https://google.github.io/filament/Materials.html#materialdefinitions/vertexblock)
- [Fragment shader code](https://google.github.io/filament/Materials.html#materialdefinitions/fragmentblock)

For rendering a background image you can use the pre-compiled material from the exampels:

- [⬇️ Download `background_image.filamat`](https://github.com/margelo/react-native-filament/blob/main/package/example/Shared/assets/background_image.filamat)

This material receives an image texture as input, which the `BackgroundImage` component will automatically set using the image you've provided.

:::warning
Only `.png` and `.jpg` images are supported yet. [Support for `.webp` is coming soon](https://github.com/google/filament/issues/7962).
:::

If you're curious how materials are defined you can have a [look at the material definition](https://github.com/margelo/react-native-filament/blob/e11899be9e9ae4b6417215e583891ef502211b16/package/example/Shared/assets/background_image.mat#L3-L6) for the background material.
Compiled materials are `.filamat` files, where the definition files are human-readable `.mat` files.

<div class="image-container">
<svg xmlns="http://www.w3.org/2000/svg" width="283" height="535">
<image href={useBaseUrl("img/demo-change-textures.gif")} x="18" y="33" width="247" height="469" />
<image href={useBaseUrl("img/frame.png")} width="283" height="535" />
</svg>
</div>

Materials are thus a way to provide custom shaders.

## Applying textures

For applying images as textures to 3D shapes need to have:

- A 3D module, that uses a material with a texture parameter (by default if your model/material is using an image texture)
- Know the entity of your model that's using the material
- Know the name of the material

For example, let's say we have [this 3D model of a rocket](https://www.cgtrader.com/free-3d-models/space/spaceship/toy-rocket-4k). When we open it in blender we see:

<img src={useBaseUrl('img/blender-explanation-materials.png')} />

- In the top right corner you see the entities: `Engine`, `Tip`, `Wings`, etc.
- You can see that an entity is associated with a material.
- You can see the materials name is "Toy Ship", and you can see it uses a texture.

| The base texture looks like this: | Variant of the base color texture : |
|---------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------|
| <img src={useBaseUrl('img/example-Toy_Rocket_Toy Ship_BaseColor.jpg')} /> | <img src={useBaseUrl('img/example-Toy_Rocket_Toy Ship_BaseColorBlue.jpg')} /> |

Now we can apply the new texture to the model:

```tsx
import RocketGlb from '~/assets/rocket.glb'
const baseColorBlueImage = require('~/assets/rocket_BaseColor_Blue.png')

function Renderer() {
// Load the texture as a buffer
const blueBaseColorBuffer = useBuffer({ source: baseColorBlueImage })

// The name of the material on the entity we want to change
const materialName = 'Toy Ship'

return (
<FilamentView style={styles.filamentView}>

<Model source={RocketGlb}>
{blueBaseColorBuffer != null && (
<>
{/* Inside the <Model> we can use the <EntitySelector> to apply modifiers */}
<EntitySelector byName="Tip" textureMap={{ materialName, textureSource: blueBaseColorBuffer }} />
<EntitySelector byName="Wings" textureMap={{ materialName, textureSource: blueBaseColorBuffer }} />
</>
)}
</Model>
</FilamentView>
```
Binary file added docs/static/img/blender-explanation-materials.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/demo-backgroundimage.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/demo-change-textures.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions package/android/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ add_library(
../cpp/core/RNFAABBWrapper.cpp
../cpp/core/RNFBoxWrapper.cpp
../cpp/core/RNFMaterialInstanceWrapper.cpp
../cpp/core/RNFNameComponentManagerWrapper.cpp
../cpp/core/RNFRenderableManagerImpl.cpp
../cpp/core/RNFRenderableManagerImpl.DebugHelpers.cpp
../cpp/core/RNFRenderableManagerWrapper.cpp
Expand Down
33 changes: 33 additions & 0 deletions package/cpp/core/RNFNameComponentManagerWrapper.cpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
//
// Created by Hanno Gödecke on 18.07.24.
//
#include "RNFNameComponentManagerWrapper.h"

namespace margelo {

void NameComponentManagerWrapper::loadHybridMethods() {
registerHybridMethod("getEntityName", &NameComponentManagerWrapper::getEntityName, this);
}

// TODO: This code is similar to EntityNameMap AnimatorWrapper::createEntityNameMap, consider refactoring
std::optional<std::string>
NameComponentManagerWrapper::getEntityName(std::shared_ptr<EntityWrapper> entityWrapper) {
if (!entityWrapper) {
throw std::invalid_argument("EntityWrapper is null");
}

Entity entity = entityWrapper->getEntity();
bool hasEntity = pointee()->hasComponent(entity);
if (!hasEntity) {
Logger::log(TAG, "Entity wasn't found in NameComponentManager");
return std::nullopt;
}

NameComponentManager::Instance instance = pointee()->getInstance(entity);
if (!instance || !instance.isValid()) {
Logger::log(TAG, "Instance wasn't found in NameComponentManager");
return std::nullopt;
}
return pointee()->getName(instance);
}
}
6 changes: 5 additions & 1 deletion package/cpp/core/RNFNameComponentManagerWrapper.h
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
#pragma once

#include "jsi/RNFPointerHolder.h"
#include "core/utils/RNFEntityWrapper.h"
#include <utils/NameComponentManager.h>

namespace margelo {
Expand All @@ -16,14 +17,17 @@ class NameComponentManagerWrapper : public PointerHolder<NameComponentManager> {
explicit NameComponentManagerWrapper(std::shared_ptr<NameComponentManager> nameComponentManager)
: PointerHolder(TAG, nameComponentManager) {}

void loadHybridMethods() override {};
void loadHybridMethods() override;

std::shared_ptr<NameComponentManager> getManager() {
return pointee();
}

private:
static auto constexpr TAG = "NameComponentManagerWrapper";

private: // Exposed JS method
std::optional<std::string> getEntityName(std::shared_ptr<EntityWrapper> entityWrapper);
};

} // namespace margelo
44 changes: 22 additions & 22 deletions package/example/AppExamplePaper/ios/Podfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -935,7 +935,7 @@ PODS:
- React-Mapbuffer (0.74.1):
- glog
- React-debug
- react-native-filament (1.3.1):
- react-native-filament (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -949,16 +949,16 @@ PODS:
- React-featureflags
- React-graphics
- React-ImageManager
- react-native-filament/camutils (= 1.3.1)
- react-native-filament/filamat (= 1.3.1)
- react-native-filament/filament (= 1.3.1)
- react-native-filament/gltfio_core (= 1.3.1)
- react-native-filament/image (= 1.3.1)
- react-native-filament/ktxreader (= 1.3.1)
- react-native-filament/math (= 1.3.1)
- react-native-filament/tsl (= 1.3.1)
- react-native-filament/uberz (= 1.3.1)
- react-native-filament/utils (= 1.3.1)
- react-native-filament/camutils (= 1.4.1)
- react-native-filament/filamat (= 1.4.1)
- react-native-filament/filament (= 1.4.1)
- react-native-filament/gltfio_core (= 1.4.1)
- react-native-filament/image (= 1.4.1)
- react-native-filament/ktxreader (= 1.4.1)
- react-native-filament/math (= 1.4.1)
- react-native-filament/tsl (= 1.4.1)
- react-native-filament/uberz (= 1.4.1)
- react-native-filament/utils (= 1.4.1)
- react-native-worklets-core
- React-NativeModulesApple
- React-RCTFabric
Expand All @@ -967,7 +967,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/camutils (1.3.1):
- react-native-filament/camutils (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -990,7 +990,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/filamat (1.3.1):
- react-native-filament/filamat (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1014,7 +1014,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/filament (1.3.1):
- react-native-filament/filament (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1038,7 +1038,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/gltfio_core (1.3.1):
- react-native-filament/gltfio_core (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1063,7 +1063,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/image (1.3.1):
- react-native-filament/image (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1086,7 +1086,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/ktxreader (1.3.1):
- react-native-filament/ktxreader (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1110,7 +1110,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/math (1.3.1):
- react-native-filament/math (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1132,7 +1132,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/tsl (1.3.1):
- react-native-filament/tsl (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1154,7 +1154,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/uberz (1.3.1):
- react-native-filament/uberz (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand All @@ -1179,7 +1179,7 @@ PODS:
- ReactCommon/turbomodule/bridging
- ReactCommon/turbomodule/core
- Yoga
- react-native-filament/utils (1.3.1):
- react-native-filament/utils (1.4.1):
- DoubleConversion
- glog
- hermes-engine
Expand Down Expand Up @@ -1777,7 +1777,7 @@ SPEC CHECKSUMS:
React-jsitracing: 233d1a798fe0ff33b8e630b8f00f62c4a8115fbc
React-logger: 7e7403a2b14c97f847d90763af76b84b152b6fce
React-Mapbuffer: 11029dcd47c5c9e057a4092ab9c2a8d10a496a33
react-native-filament: 0c55d396e788695c7a7884bb3a9f5d0957b0d74c
react-native-filament: 04ef2c32e88eec8f22b545029962f2b4af48e059
react-native-safe-area-context: dcab599c527c2d7de2d76507a523d20a0b83823d
react-native-video: 9f29aee2feb9e42da24cb40b70d005d38a1a132f
react-native-worklets-core: f51430dd07bf5343d4918d28a4bb00fe8f98b982
Expand Down
Loading

0 comments on commit 865aec6

Please sign in to comment.