Gamasutra: The Art & Business of Making Gamesspacer
View All     RSS
September 22, 2017
arrowPress Releases






If you enjoy reading this site, you might also want to check out these UBM Tech sites:


 

State-of-the-art HLSL to GLSL Converter

by Egor Yusov on 07/17/17 10:22:00 am

The following blog post, unless otherwise noted, was written by a member of Gamasutra’s community.
The thoughts and opinions expressed are those of the writer and not Gamasutra or its parent company.

 

Introduction

There are many situations when DirectX shaders authored in HLSL need to be converted to GLSL. Typical examples include porting existing application to OpenGL and cross-platform development. HLSL and GLSL share a lot in common, but sometimes differ substantially. Maintaining two versions of shaders is time-consuming and error-prone, and is thus not an option for most of the applications. One typical solution to the problem is using intermediate representation that translates to both languages, but it does not work when existing shaders are written in HLSL. Authoring shaders in HLSL also allows using HLSL shader development tools.

There exist several converters, but they provide limited functionality. For instance, this and this projects only support D3D9-style shaders. This online translator only supports vertex and fragment shaders.

Diligent Engine uses state-of-the art shader sonverter that allows transforming HLSL source into GLSL. The converter supports HLSL 5.0 and all shader types (vertex, pixel, geometry, hull, domain, and compute). 

The converter can be downloaded from this page.

Conversion Details

Basics

The converter supports HLSL5.0, all shader types (vertex, geometry, pixel, domain, hull, and compute) as well as most of the language constructs. There are however few special requirements that must be met in order for the HLSL source to be successfully converted to GLSL:

  • Inputs to a vertex shader may have ATTRIBn semantic, where n defines the location of the corresponding GLSL input variable (layout(location = n)). For any other input semantic, the converter automatically assigns input location
  • Inputs of a subsequent shader stage must be declared in exact same order as outputs of the previous shader stage. Return value of a function counts as its first output.
    • The converter parses all input and output arguments (including structure members) in the order of declaration and automatically assigns locations to every argument. To make sure that input and output locations match, the arguments must be declared in exact same order. For the same reason, if an argument is not used by the shader, it still needs to be declared to preserve original ordering

The code snippet below gives examples of supported shader declarations:

struct VSInput
{
    // It is recommended (though not required) to assign 
    //  ATTRIBn semantics to inputs from IA
    in float3 f3PosWS : ATTRIB0;
    in float2 f2UV  : ATTRIB1;
    uint VertexID : SV_VertexID;
};

struct VSOutput
{
    float2 UV : TEX_COORDINATES;
    float3 Normal : NORMAL;
    float4 f4PosPS : SV_Position;
};

VSOutput VertexShader ( in VSInput VSIn,
                        in float3 f3UV  : ATTRIB2,
                        uint InstID : SV_InstanceID,
                        out float3 f3Tangent : TANGENT )
{
    VSOutput VSOut;
    // Body elided
    f3Tangent = ...
    return VSOut;
}

float4 PixelShader ( // Pixel shader inputs must be declared in exact same order 
                     // as outputs of the vertex shader
                     VSOutput PSIn,
                     in float3 f3Tangent : TaNgEnT // Semantics are case-insensitive
                     out float3 Color2 : SV_Target2 ) :  SV_Target
{
    // Body elided
}

 

  • When tessellation is enabled in OpenGL, partitioning as well as output patch topology are defined by the tessellation evaluation shader (domain shader) rather than by the tessellation control shader (hull shader). As a result, the converter cannot generate GLSL code without using special hints. To indicate missing attributes, the following specially formatted comment should be added right on top of the domain shader body:
    /* partitioning = {integer|fractional_even|fractional_odd}, 
       outputtopology = {triangle_cw|triangle_ccw} */
    For example, the following is a valid domain shader declaration:
    [domain("quad")]
    /* partitioning = fractional_even, outputtopology = triangle_cw */
    DSOutput main( HS_CONSTANT_DATA_OUTPUT input, 
                   float2 QuadUV : SV_DomainLocation, 
                   OutputPatch<HSOutput, 2> QuadPatch)
    {
        // Body elided
    }
  • Geometry, Domain and Hull shaders must be defined in separate files
  • GLSL allows samplers to be declared as global variables or function arguments only. It does not allow local variables of sampler type.

Textures and samplers

The following rules are used to convert HLSL texture declaration into GLSL sampler:

  • HLSL texture dimension defines GLSL sampler dimension:
    • Texture2D   →   sampler2D
    • TextureCube →   samplerCube
  • HLSL texture component type defines GLSL sampler type. If no type is specified, float4  is assumed:
    • Texture2D<float>     →   sampler2D
    • Texture3D<uint4>     →   usampler3D
    • Texture2DArray<int2> →   isampler2DArray
    • Texture2D            →   sampler2D
  • To distinguish if sampler should be shadow or not, the converter tries to find <Texture Name>_sampler among samplers (global variables and function arguments). If the sampler type is comparison, the texture is converted to shadow sampler. If sampler state is either not comparison or not found, regular sampler is used. Examples:
Texture2D g_ShadowMap;
SamplerComparisonState g_ShadowMap_sampler;

Texture2D g_Tex2D;
SamplerState g_Tex2D_sampler;

Texture3D g_Tex3D;

↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓    ↓

sampler2DShadow g_ShadowMap;
sampler2D g_Tex2D;
sampler3D g_Tex3D;
  • GLSL requires format to be specified for all images (rw textures) allowing writes. HLSL converter allows GLSL image format specification inside the special comment block:
    RWTexture2D<float /* format=r32f */ > Tex2D;

 

Please visit this page for the full list of supported features.

Important notes/known issues

  • GLSL compiler does not handle float3 structure members correctly. It is strongly suggested avoid using this type in structure definitions
  • At least NVidia GLSL compiler does not apply layout(row_major) to structure members. By default, all matrices in both HLSL and GLSL are column major
  • GLSL compiler does not properly handle structs passed as function arguments!

DO NOT pass structs to functions, use only built-in types!

  • GLSL does not support most of the implicit type conversions. The following are some examples of the required modifications to HLSL code:
    float4 vec = 0; ->  float4 vec = float4(0.0, 0.0, 0.0, 0.0);
    float x = 0;    ->  float x = 0.0;
    uint x = 0;     ->  uint x = 0u;
  • GLES is immensely strict about type conversions. For instance, this code will produce compiler error: float4(0, 0, 0, 0). It must be written as float4(0.0, 0.0, 0.0, 0.0)
  • GLSL does not support relational and boolean operations on vector types:
    float2 p = float2(1.0, 2.0), q = float2(3.0, 4.0);
    bool2 b = p < q; // Error
    all(p < q); // Error
  • To facilitate relational and Boolean operations on vector types, the following functions are predefined:
    • Less
    • LessEqual
      • Greater
    • GreaterEqual
      • Equal
    • NotEqual
    • Not
    • And
    • Or
    • BoolToFloat
  • Examples:
    bool2 b = x < y;  -> bool2 b = Less(x, y);
    all(p>=q)  ->  all( GreaterEqual(p,q) )
  • When accessing elements of an HLSL matrix, the first index is always a row:
    mat[row][column]
  • In GLSL, the first index is always a column:
    mat[column][row]
  • MATRIX_ELEMENT(mat, row, col)  macros is provided to facilitate matrix element retrieval
  • The following functions do not have counterparts in GLSL and should be avoided:
    Texture2DArray.SampleCmpLevelZero()
    TextureCube.SampleCmpLevelZero()
    TextureCubeArray.SampleCmpLevelZero()

Limitations

Converter does not perform macros expansion, so usage of preprocessor directives is limited to text block that do not need to be converted. The following are some usage examples that are not supported:

  • Using macros in declarations of shader entry points:
    VSOut TestVS  (
    #ifdef SOME_MACRO
                   in VSInput0 VSInput
    #else
                   in VSInput1 VSInput
    #endif
                   )
    The following is not allowed as well:
    #ifdef SOME_MACRO
        VSOut TestVS  (in VSInput0 VSInput)
    #else
        VSOut TestVS  (in VSInput1 VSInput)
    #endif
    In cases like that it is necessary to create two separate shader entry points and give them distinctive names. Likewise, macros cannot be used in definitions of structures that are used to pass data between shader stages:
    struct VSInput
    {
        in float3 f3PosWS : ATTRIB0;
    #ifdef SOME_MACRO
        in float2 f2UV  : ATTRIB1;
    #else
        in float4 f4UV  : ATTRIB1;
    #endif
        uint VertexID : SV_VertexID;
    };
    Similarly to shader entry points, in the scenario above, the two structures need to be defined with distinctive names. Shader macros are allowed in structures that are not used to pass data between shader stages.
  • Defining language keywords with macros is not allowed:
    #define TEXTURE2D Texture2D
    TEXTURE2D MacroTex2D;

Macros can be used within function bodies:

VSOut VSTes(...)
{
#ifdef SOME_MACRO
    // OK
#else
    // OK
#endif
}

Standalone Converter

Standalone HLSL2GLSLConverter allows off-line conversion of shaders authored in HLSL to GLSL. The command line options for the app are given below:

Argument Description
-h Print help message
-i <filename> Input file path (relative to the search directories)
-d <dirname> Search directory to look for input file as well as all #include files.
Every search directory should be specified using -d argument
-o <filename> Output file to write converted GLSL source to
-e <funcname> Shader entry point
-c Compile converted GLSL shader
-t <type> Shader type. Allowed values:
  • vs - vertex shader
  • ps - pixel shader
  • gs - geometry shader
  • ds - domain shader
  • hs - domain shader
  • cs - domain shader
-noglsldef Do not include glsl definitions into the converted source

Command line example:

HLSL2GLSLConverter -i ConverterTest.fx -d .\testshaders -d .\ -t vs -e TestVS -c -o .\testshaders\ConvertedShader.txt

By default, the converter includes auxiliary GLSL definitions into the converter source. This can be disabled with -noglsldef command line option, but the definitions file GLSLDefinitions.h is required for the shader file to be compiled and must be included manually.

When compiling GLSL shader, Diligent Engine adds the following lines on top of the converted GLSL source.

For desktop GL:

#version 430 core
#define DESKTOP_GL 1
layout(std140) uniform;

For GLES:

#version 310 es

#ifndef GL_ES
#  define GL_ES 1
#endif

precision highp float;
precision highp int;
//"precision highp uint;  // This line causes shader compilation error on NVidia!

precision highp sampler2D;
precision highp sampler3D;
precision highp samplerCube;
precision highp samplerCubeArray;
precision highp samplerCubeShadow;
precision highp samplerCubeArrayShadow;
precision highp sampler2DShadow;
precision highp sampler2DArray;
precision highp sampler2DArrayShadow;
precision highp sampler2DMS;       // ES3.1

precision highp isampler2D;
precision highp isampler3D;
precision highp isamplerCube;
precision highp isamplerCubeArray;
precision highp isampler2DArray;
precision highp isampler2DMS;      // ES3.1

precision highp usampler2D;
precision highp usampler3D;
precision highp usamplerCube;
precision highp usamplerCubeArray;
precision highp usampler2DArray;
precision highp usampler2DMS;      // ES3.1

precision highp image2D;
precision highp image3D;
precision highp imageCube;
precision highp image2DArray;

precision highp iimage2D;
precision highp iimage3D;
precision highp iimageCube;
precision highp iimage2DArray;

precision highp uimage2D;
precision highp uimage3D;
precision highp uimageCube;
precision highp uimage2DArray;

layout(std140) uniform;


Also on Android, the engine defines the following extension when compiling geometry shader:

#extension GL_EXT_geometry_shader : enable

and the following extension when compiling tessellation control (hull) or tessellation evaluation (domain) shader:

#extension GL_EXT_tessellation_shader : enable

Besides that, the engine also defines one of the following macros depending on the type of the shader being compiled: VERTEX_SHADER, FRAGMENT_SHADER, GEOMETRY_SHADER, TESS_CONTROL_SHADER, TESS_EVALUATION_SHADER, COMPUTE_SHADER. If converted source is not handled by Diligent Engine, one of the macros above must always be defined before including GLSLDefinitions.h.

Downloading Converter

Download HLSL2GLSLConverter.

The archive contains two versions of converter app built for win32 and win64 platforms. testshaders folder contains some shader examples and .bat files conaining commands to perform conversion.


Related Jobs

Naughty Dog
Naughty Dog — Santa Monica, California, United States
[09.22.17]

Graphics Programmer (Game Team)
Insomniac Games
Insomniac Games — Burbank, California, United States
[09.22.17]

Mid to Sr Gameplay Programmer
Insomniac Games
Insomniac Games — Durham, North Carolina, United States
[09.22.17]

Mid to Sr Gameplay Programmer
Monomi Park
Monomi Park — San Mateo, California, United States
[09.22.17]

Game Engineer





Loading Comments

loader image