Honor camera layers on light.shadow.camera for selective shadow

[Samsy]

Description

In order to build a selective real time shadow along with a baked shadow system the light.shadow.camera should honor the layers to be tested against an object to render the shadow

I did add support on a custom build of the WebGLRenderer and this was very handy

Here an example disabling all the layers for the shadow camera but still renders :

Live Link

Solution

Before rendering the shadow, the mask is copied from the main render camera, wondering if this copy is mandatory, and if we could go with manually setting the layers instead for more controls, or maybe provide a solution to play with the shadow.camera layer without being overwritten :

three.js/src/nodes/lighting/ShadowNode.js

Line 659 in 672d77a

shadow.camera.layers.mask = camera.layers.mask;

[Samsy]

any thoughts @Mugen87 ?

[RenaudRohlinger]

Maybe if the shadowMap layer is different than default then we use shadow.camera.layer instead of camera.layer?

[Samsy]

You mean adding a layer property to light.shadow, and test this one to check if it isnt default it would then be used correct ?

[Samsy]

hey @Mugen87
Have you had to chance to look at this one ?

Thanks a lot

[Mugen87]

Is there currently no alternative to achieve selective shadows? I never had the use case for something like this hence I'm asking...

We could introduce a boolean property in LightShadow. When set to true, the layer configuration from the shadow camera is used instead of copying the layers from the main camera.

[Samsy]

I don't think there is any other way to render selective shadow

[yelouafi]

webgl still got the issue btw
so there's an inconsistency between the 2 renderers

[yelouafi]

FI ended up with this hacky workaround for webgl renderer

function monkeyPatchRenderer(renderer) {

    const originalRenderBufferDirect = renderer.renderBufferDirect;

    function _fixedRenderBufferDirect( camera, scene, geometry, material, object, group )  {
        if(camera.userData._IS_SHADOW_CAMERA && !object.layers.test( camera.layers )) {
            return;
        }
        originalRenderBufferDirect.call(this, camera, scene, geometry, material, object, group);
    }

    renderer.renderBufferDirect = _fixedRenderBufferDirect;
}

this essentialy skips rendering on

• https://github.com/mrdoob/three.js/blob/r182/src/renderers/webgl/WebGLShadowMap.js#L536
• https://github.com/mrdoob/three.js/blob/r182/src/renderers/webgl/WebGLShadowMap.js#L550