[Perfomance] Frame drops

[Francescu]

Hi guys,

The lib is awesome, but we get performance issue regarding frame drops.

Do you think recording in a buffer first and perform the actual write after the recording would solve the issue?

[okaris]

I believe the performance issues are caused by rendering the scene twice. First is for ARKit and the second render is for the recorder. I have tried to work around this issue by pulling the already rendered frame from sceneview but there were some complex color space problems, which Apple does not give up information about. They also suggested that approach should not be used.

Given these I think writing to disk is not the problem. Since iPhones operate on SSD's the write speeds are close to RAM and that is highly enough for the task at hand.

I would suggest that you check the performance issues part in the readme. Those might help. And also you can try setting the recording to a lower resolution.

We are actively working on this repo to overcome performance issues. Contributions are always welcome!

[Francescu]

I believe the performance issues are caused by rendering the scene twice. First is for ARKit and the second render is for the recorder. I have tried to work around this issue by pulling the already rendered frame from sceneview but there were some complex color space problems, which Apple does not give up information about. They also suggested that approach should not be used.

Ok, thanks for the explanation. So, I get that most of the performance hurts come from the double rendering. Do you think a reversed could approach work? E.g disabling the sceneView rendering and displaying images we get from the snapshot method instead?

Given these I think writing to disk is not the problem. Since iPhones operate on SSD's the write speeds are close to RAM and that is highly enough for the task at hand.

I noticed that we had a lot of crash before you updated safeguard (like videoInput.isReadyForMoreMediaData and CMTIME_IS_VALID). I though it meant that we drop frames because of the writing itself.

I would suggest that you check the performance issues part in the readme. Those might help. And also you can try setting the recording to a lower resolution.

Yes I did, but it didn't affect performance so much. Plus, we don't want to lose visual appeal or resolution as it's very important to our product.

We are actively working on this repo to overcome performance issues. Contributions are always welcome!

Sure, I'd be glad to help in anyway I can. If I found anything, I'll PR you.

Thanks again for your great work

[Francescu]

Do you think SCNView snapshot method also perform a new rendering? As it use the current state of the renderer, it should not be as resource consuming as snapshot(atTime:)?