[REQUEST] 3D LUT Shader

TreyM · August 3, 2016 at 12:24 PM

I have been using ReShade and ENB to color grade my video games for a while now, and was wondering if this 3D LUT code (or any 3D LUT code for that matter) could be converted to an MMF 2.5 shader?

Please login to see this link.

Quote

UE4, Unity compatible 256x16 look up table(LUT)

Base LUT Image: Please login to see this picture.

Code

float2 CLut_pSize = float2(0.00390625, 0.0625);// 1 / float2(256, 16);
color.rgb  = saturate(color.rgb);
color.b   *= 15;
float4 CLut_UV = 0;
CLut_UV.w  = floor(color.b);
CLut_UV.xy = color.rg * 15 * CLut_pSize + 0.5 * CLut_pSize ;
CLut_UV.x += CLut_UV.w * CLut_pSize.y;
color.rgb  = lerp( tex2Dlod(_s7, CLut_UV.xyzz).rgb, tex2Dlod(_s7, CLut_UV.xyzz + float4(CLut_pSize.y, 0, 0, 0)).rgb, color.b - CLut_UV.w);

Further example for higher res LUT image:

Code

//dx10/11
    float2 CLut_pSize =  1 / float2(4096, 64);
    color.rgb  = saturate(color.rgb) * 63;
    float4 CLut_UV;
    CLut_UV.w  = floor(color.b);
    CLut_UV.xy = (color.rg + 0.5) * CLut_pSize;
    CLut_UV.x += CLut_UV.w * CLut_pSize.y;
    CLut_UV.z  = CLut_UV.x + CLut_pSize.y;
    color.rgb  = lerp(LUTtex.SampleLevel(Sampler1, CLut_UV.xy, 0).rgb, LUTtex.SampleLevel(Sampler1, CLut_UV.zy, 0).rgb, color.b - CLut_UV.w);

If so, could it be modified to have multiple LUTs to define, and fade between them at runtime? If not with the multi-lut single texture shown below, then perhaps with multiple individual textures.

Example of multiple LUTs in a single texture:

Quote

Please login to see this picture.

UI:

Code

int   lut_D <   string UIName="lut_D";   float UIMin=0;   float UIMax=9;> = {0};
int   lut_N <   string UIName="lut_N";   float UIMin=0;   float UIMax=9;> = {0};
int   lut_I <   string UIName="lut_I";   float UIMin=0;   float UIMax=9;> = {0};

Resources:

Code

Texture2D          LUTtex_D< string UIName = "3DLut0";  string ResourceName = "LUTtex_D.png"; >; //day
Texture2D          LUTtex_N< string UIName = "3DLut0";  string ResourceName = "LUTtex_N.png"; >; //night
Texture2D          LUTtex_I< string UIName = "3DLut0";  string ResourceName = "LUTtex_I.png"; >; // interior

Code

if(enablelut)
{
    float2 CLut_pSize = {0.00390625, 0.0625};// 1 / float2(256, 16)
    color.rgb         = saturate(color.rgb) * 15;
    
    float3 CLut_UV;    
    CLut_UV.z = floor(color.z);
    color.z  -= CLut_UV.z;
    color.xy  = (color.xy + 0.5) * CLut_pSize;
    color.x  += CLut_UV.z * CLut_pSize.y;
    color.y  *= 0.1; // 1 / 10


    CLut_UV.x = color.x;
    CLut_UV.z = CLut_UV.x + CLut_pSize.y;
    CLut_UV.y = color.y + lut_D * 0.1;
    float3 lutcolor_D = lerp(LUTtex_D.SampleLevel(Sampler1, CLut_UV.xy, 0).rgb, LUTtex_D.SampleLevel(Sampler1, CLut_UV.zy, 0).rgb, color.z);
    CLut_UV.y = color.y + lut_N * 0.1;
    float3 lutcolor_N = lerp(LUTtex_N.SampleLevel(Sampler1, CLut_UV.xy, 0).rgb, LUTtex_N.SampleLevel(Sampler1, CLut_UV.zy, 0).rgb, color.z);
    CLut_UV.y = color.y + lut_I * 0.1;
    float3 lutcolor_I = lerp(LUTtex_I.SampleLevel(Sampler1, CLut_UV.xy, 0).rgb, LUTtex_I.SampleLevel(Sampler1, CLut_UV.zy, 0).rgb, color.z);


    color.rgb         = lerp(lerp( lutcolor_N, lutcolor_D, ENightDayFactor), lutcolor_I, EInteriorFactor);
}

Display More

Another example from ReShade:

Code

texture ColorLUTDstTex	< string source = "ReShade/CustomFX/Textures/" TuningColorLUTDstTexture; > {Width = TuningColorLUTTileAmountX; Height = TuningColorLUTTileAmountY; Format = RGBA8;};
sampler	ColorLUTDstColor 	{ Texture = ColorLUTDstTex; };


#define TuningColorLUTNorm float2(1.0/float(TuningColorLUTTileAmountX),1.0/float(TuningColorLUTTileAmountY))


float4 PS_TuningPalette(float4 vpos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
	float4 original = tex2D(RFX_backbufferColor, texcoord.xy);


	float4 ColorLUTDst = float4((original.rg*float(TuningColorLUTTileAmountY-1)+0.5f)*TuningColorLUTNorm,original.b*float(TuningColorLUTTileAmountY-1),original.w);
	ColorLUTDst.x += trunc(ColorLUTDst.z)*TuningColorLUTNorm.y;
	ColorLUTDst = lerp(tex2D(ColorLUTDstColor, ColorLUTDst.xy),tex2D(ColorLUTDstColor, float2(ColorLUTDst.x+TuningColorLUTNorm.y,ColorLUTDst.y)),frac(ColorLUTDst.z));
	original = lerp(original,ColorLUTDst,TuningColorLUTIntensity);
	return original;

Display More

TreyM · August 8, 2016 at 10:19 AM

No one?

MuddyMole · August 9, 2016 at 3:10 PM

MMF2 (and CF2.5) supports pixel shader models up to 2.0a, while the code you provided uses functions only supported in 4.0 and later. That's not really the issue here though, as the shader could easily be rewritten without them.

The real problem is that MMF2 only uses (bi?)linear texture filtering on object textures, and not on textures which are passed as image parameters (eg. your LUT image), for which it uses nearest-point sampling - and for this, you really need linear texture filtering to be able to interpolate between the colors in your LUT image (otherwise you'd end up with an image reduced to only 16x16x16 = 4096 colors, which would look awful).

Explanation of the difference:
Please login to see this link.

The only way around it would be to sample from something like 8 different points, and interpolate between all of them, which would be very, very complicated (and not terribly efficient).

TreyM · August 11, 2016 at 5:33 AM

Thank you for responding MuddyMole.

I personally prefer using larger resolution LUTs anyway. I use a 4096 X 64 lut that I created from the texture below:

Please login to see this picture.

To make the 4096 X 64 lut from that, I just pulled the rows in photoshop into a single line, top to bottom order, reordered left to right. It basically is a higher resolution version of the 256 x 16 LUT with 64 blocks instead of 16.

AFAIK, the 512x512 (or 4096x64) texture contains every possible color in the sRGB colorspace, so there would be no need to interpolate, or am I wrong and totally misunderstanding?

MuddyMole · August 11, 2016 at 7:15 PM

You'd need a texture 256 x 65536 (or 4096 x 4096), which is bigger than DX9 (and many graphics cards) support, and there'd likely be a performance hit from using such a huge texture anyway.

Looki · August 11, 2016 at 8:15 PM

I haven't read this entire thread, but MuddyMole, it is possible to override the sampler state of a texture quite easily!
Here's what I've used a lot (not so much in shaders I've released - but I had to make a LOT of specialized shaders for Heart Forth, Alicia ;D ):
sampler2D img = sampler_state {
MinFilter = Linear;
MagFilter = Linear;
AddressU = Border;
AddressV = Border;
BorderColor = float4(0, 0, 0, 0);
};

Which forces interpolation for all texture lookups and returns a transparent pixel for all texture lookups outside 0..1

TreyM · August 11, 2016 at 8:27 PM

I really hope this is possible, then. The ability to change the look of the game on the fly with advanced grading is very enticing.

MuddyMole · August 12, 2016 at 8:49 AM

Looki: thank you - that's very interesting, and potentially useful. I didn't know that was possible.
I'll look into it later (possibly not today, as I've got a lot on).

MuddyMole · August 14, 2016 at 12:20 PM

So I had a brief go at it, and it should be do-able, but it's tricky and I can't be bothered to keep trying.
The image on the right is the original; the image on the left is after applying a "neutral" LUT, so it should look about the same.

Please login to see this picture.

It's not far off, but there's a problem: look at the red, yellow (on the right), light blue and dark green pencils - there's either too much or not enough red.
This is due to issues with sampling at the boundary between two 16x16 squares, where the red component instantly jumps from 0 to 255 - possibly the result of slight rounding and calculation errors, caused by the limited precision of floats; possibly just a mistake in my code.
It could probably be tweaked to work, but I've come across this kind of issue before, and they can be a real pain to solve, so I'm not going to waste my time on it.

EDIT: Sorry, ignore that... As I was typing, it gave me an idea, and it's actually fixed the problem

Please login to see this picture.

A quick sepia effect:
Please login to see this picture.

TreyM · August 14, 2016 at 1:12 PM

OMG Fantastic MuddyMole! Getting this working is a BIG DEAL! At least to me anyway. I cannot thank you enough for working on this.

MuddyMole · August 14, 2016 at 11:41 PM

Usage Notes:
This shader is to be used as an overlay, adjusting the color of everything behind it.
It requires a 17x17x17 (289x17) LUT image to be passed as the image parameter "lut".

Download:
Please login to see this link.

.XML File:

Code

<effect>
<name>Color Grading</name>
<description>Uses a 3d lookup table (LUT) to modify image colors.</description>
<author>Sketchy / MuddyMole</author>
<parameter>
    <name>LUT Image</name>
    <code>lut</code>
    <type>IMAGE</type>
    <property>IMAGE</property>
    <value></value>
</parameter>
<BackgroundTexture>1</BackgroundTexture>
</effect>

Display More

.FX File:

Code

sampler2D bkd : register(s1);
sampler2D lut : register(s2) = sampler_state {
    MinFilter = Linear;
    MagFilter = Linear;
};


float4 ps_main(in float2 In : TEXCOORD0) : COLOR0 {
    float4 imgColor = tex2D( bkd, In );
    float red = ( imgColor.r * 16 + 0.5 ) / 289;
    float green = ( imgColor.g * 16 + 0.5 ) / 17;
    float blueA = floor( imgColor.b  * 16 ) / 17;
    float blueB = ceil( imgColor.b  * 16 ) / 17;
    float4 colorA = tex2D( lut, float2( blueA + red, green ));
    float4 colorB = tex2D( lut, float2( blueB + red, green ));
    float lerpAB = ( imgColor.b - ( blueA + red )) / (( blueB + red ) - ( blueA + red ));
    float4 colorOut = lerp( colorA, colorB, lerpAB );
    return colorOut;
}


technique tech_main {
    pass P0 {
        PixelShader = compile ps_2_0 ps_main();
    }
}

Display More

Default 17x17x17 LUT:
Please login to see this picture.

Examples:
Please login to see this picture. (negative image)
Please login to see this picture. (posterize)
Please login to see this picture. (sepia)
Please login to see this picture. (Schindler's List)

TreyM · August 14, 2016 at 11:45 PM

Fantastic sir! If you have PayPal, please send me your info via PM so I can donate to you.

TreyM · August 15, 2016 at 1:02 AM

One last thing. This is not a complaint. I really appreciate what you have done.

Is there any way to make the blend coefficient work? It sort of works as it is, with 255 being no effect, and dropping down to 254 applies the effect in full. What I'm asking is basically, is there any way to control the strength of the effect? If not, I totally understand. I am very happy with the ability to do color grading in CTF 2.5!

TreyM · August 15, 2016 at 7:03 AM

Thanks again for making this, MuddyMole.
Did some digging and modified your code to work with other LUT sizes:

Code for 4096px X 64px (64,64,64) LUT:

Code

sampler2D bkd : register(s1);
sampler2D lut : register(s2) = sampler_state {
    MinFilter = Linear;
    MagFilter = Linear;
};


float4 ps_main(in float2 In : TEXCOORD0) : COLOR0 {
    float4 imgColor = tex2D( bkd, In );
    float red = ( imgColor.r * 63 + 0.5 ) / 4096;
    float green = ( imgColor.g * 63 + 0.5 ) / 64;
    float blueA = floor( imgColor.b  * 63 ) / 64;
    float blueB = ceil( imgColor.b  * 63 ) / 64;
    float4 colorA = tex2D( lut, float2( blueA + red, green ));
    float4 colorB = tex2D( lut, float2( blueB + red, green ));
    float lerpAB = ( imgColor.b - ( blueA + red )) / (( blueB + red ) - ( blueA + red ));
    float4 colorOut = lerp( colorA, colorB, lerpAB );
    return colorOut;
}


technique tech_main {
    pass P0 {
        PixelShader = compile ps_2_0 ps_main();
    }
}

Display More

Please login to see this picture.
^ Right click, save as

As far as the rest of the code goes, I'm still a bit lost lol.

I did the pass-through test as you did and did not observe errors:
Please login to see this picture.

And here is a test grade using a lut I had laying around:
Please login to see this picture.

I also tried my hand at adding an intensity value to the shader and XML, but failed miserably.

-------------------------------------

For reference for other users, here is the code for a 1024px X 32px (32,32,32) LUT:

Code

sampler2D bkd : register(s1);
sampler2D lut : register(s2) = sampler_state {
    MinFilter = Linear;
    MagFilter = Linear;
};


float4 ps_main(in float2 In : TEXCOORD0) : COLOR0 {
    float4 imgColor = tex2D( bkd, In );
    float red = ( imgColor.r * 31 + 0.5 ) / 1024;
    float green = ( imgColor.g * 31 + 0.5 ) / 32;
    float blueA = floor( imgColor.b  * 31 ) / 32;
    float blueB = ceil( imgColor.b  * 31 ) / 32;
    float4 colorA = tex2D( lut, float2( blueA + red, green ));
    float4 colorB = tex2D( lut, float2( blueB + red, green ));
    float lerpAB = ( imgColor.b - ( blueA + red )) / (( blueB + red ) - ( blueA + red ));
    float4 colorOut = lerp( colorA, colorB, lerpAB );
    return colorOut;
}


technique tech_main {
    pass P0 {
        PixelShader = compile ps_2_0 ps_main();
    }
}

Display More

Please login to see this picture.
^ Right click, save as

-------------------------------------

And here is the code for a 256px X 16px (16,16,16) LUT:

Code

sampler2D bkd : register(s1);
sampler2D lut : register(s2) = sampler_state {
    MinFilter = Linear;
    MagFilter = Linear;
};


float4 ps_main(in float2 In : TEXCOORD0) : COLOR0 {
    float4 imgColor = tex2D( bkd, In );
    float red = ( imgColor.r * 15 + 0.5 ) / 256;
    float green = ( imgColor.g * 15 + 0.5 ) / 16;
    float blueA = floor( imgColor.b  * 15 ) / 16;
    float blueB = ceil( imgColor.b  * 15 ) / 16;
    float4 colorA = tex2D( lut, float2( blueA + red, green ));
    float4 colorB = tex2D( lut, float2( blueB + red, green ));
    float lerpAB = ( imgColor.b - ( blueA + red )) / (( blueB + red ) - ( blueA + red ));
    float4 colorOut = lerp( colorA, colorB, lerpAB );
    return colorOut;
}


technique tech_main {
    pass P0 {
        PixelShader = compile ps_2_0 ps_main();
    }
}

Display More

Please login to see this picture.
^ Right click, save as

The benefit of using a larger LUT size is greater accuracy across complex color changes. For most users, the 256 X 16 will be good enough, as will MuddyMole's 289 X 17 LUT (which should be slightly more accurate.)
As far as I know, the 4096 X 64 is a near-perfect translation, while the smaller sizes interpolate the colors to a far heavier degree.

As far as performance differences, I have no idea. I don't have a project large enough to test it, and even if I did, my computer is really overkill for MMF anyway so I doubt I'd see much of a performance drop.

MuddyMole · August 15, 2016 at 1:44 PM

Yeah, it's not really the finished article - it could do with a few additions...
- square LUT images (eg. 8x8 instead of 64x1), to improve performance and compatibility.
- easily selectable LUT size, using a parameter in MMF2.
- the option to fade between 2 LUT images.

I'm actually quite busy though, and since I'm not promising to add any of those features any time soon, I thought I'd just post what I'd done.

The shader is actually extremely simple in principle - the tricky part is correctly offsetting all the coordinates to make sure that you sample from exactly the right spots (which is what "(n * 16 + 0.5) / 17" does).

TreyM · August 15, 2016 at 3:59 PM

Do the code modifications I made make the image coordinates not as accurate then? I really appreciate the time you took to make it in the first place.

MuddyMole · August 17, 2016 at 7:07 PM

No, your changes are fine - it's just much easier to change one parameter in MMF2 than to modify lots of values in the shader file itself.
I was thinking before that graphics cards always internally use square textures with dimensions that are a power of two (thought I read that somewhere?), so the square LUTs would be more efficient, but actually it seems this is not the case, so I won't be modifying the shader to use square LUTs.

This is the latest (final) version:
Please login to see this link.

happygreenfrog · August 17, 2016 at 7:17 PM

Graphics cards don't always use SQUARE textures with dimensions that are a power of 2. However, I'm pretty sure the dimensions are, in fact, best as a power of 2 (I've never heard the square bit before, quite frankly).

Just thought I'd point that out.

On a more related note, this seems like it could be a useful shader! Nice work!

MuddyMole · August 17, 2016 at 9:41 PM

Yeah, that seems to be the case, although the performance difference appears to be quite minimal (esp. by the time it's just one object anyway, rather than hundreds of polygons). There are also apparently still some other restrictions on non-power-of-two textures that don't apply to MMF2 shaders.

Anyway, yes, I think the shader could be very useful for day/night/weather effects, but I'm not too sure what other applications it would have...

TreyM · August 17, 2016 at 10:05 PM

Thanks for the update MuddyMole! You're a beast.

Log in or register!

Looking for Customer Support?

[REQUEST] 3D LUT Shader

Participate now!

Useful Links, Resources & Information

Fusion 2.5 Resources

Crowdfunding Focus