Vulkan Enum Stringifier Project - Quick Postmortem

Prelude

#

Operating with Vulkan can be cumbersome. There’s a fair number of moving parts, between shaders, renderpasses, framebuffers, pipelines, and more. For fairly simple or toy projects, defining each individual part and enumerating the settings and interactions between each of them isn’t so bad. Still, with some concerted effort, or some specialized classes or other organizational pattern, I’m sure it’s very possible to keep much of it organized and kept within code.

My first efforts were defining different shaders to be loaded from files, with the shader stages (Vertex, geometry, tesselation, fragment, etc) being turned to/from strings within a YAML file not being too bad when hand-rolled:

shader:
  name: ShadowNoBoned
  file: data/tests/shaders/deferred_shadows/shadow.vert.spv
  type: vertex

auto const type = typeNode.as<std::string>();
if (type == "vertex") {
    stage = VK_SHADER_STAGE_VERTEX_BIT;
} else if (type == "geometry") {
    stage = VK_SHADER_STAGE_GEOMETRY_BIT;
} else if (type == "fragment") {
    stage = VK_SHADER_STAGE_FRAGMENT_BIT;
} else if (type == "compute") {
    stage = VK_SHADER_STAGE_COMPUTE_BIT;
} else {
    // ...
}

However, as I tried to expand this beyond into other components, this blew up immediately. As of Vulkan header v133, there are 167 enum/flag types available, with countless options. Even worse, those values can eventually lose their vendor tags as they are promoted into the mainline spec between versions, such as VkResolveModeKHRFlagBits::VK_RESOLVE_MODE_MIN_BIT_KHR => VkResolveModeFlagBits::VK_RESOLVE_MODE_MIN_BIT. This means that we’re also dealing with a moving target, meaning doing all of this by hand is right out.

So, obviously some automation is in order.

Original Goal

#

Luckily for us, with Vulkan being a nice industry standard, specifications are produced, and one can find the original vk.xml spec (at least on 1.1.72+) that generates the vulkan_core.h headers distributed for development use. So with this in hand, I decided to auto-generate source/header pairs that could be used to easily stringify Vulkan enums/flags.

This would both generate and read strings, with type type prefixes removed (VkImageLayout enums all have the VK_IMAGE_LAYOUT_ prefix), and stripped of vendor tags (AMD, NV, KHR, etc).

Issues and the Solution Evolution

#

The First Draft’s Terrible Assumptions

#

The original first attempt, was a system that simply iterated through the XML spec, and transcribed all the given items into the new file, where each type got its own specialized struct for name/values, and thus also would have it’s own function to return with:

struct VkImageLayoutEnumValueSet {
    const char *name;
    VkImageLayout value;
};
constexpr VkImageLayoutEnumValueSet VkImageLayoutSets[] = {
    {"UNDEFINED", VK_IMAGE_LAYOUT_UNDEFINED},
    {"GENERAL", VK_IMAGE_LAYOUT_GENERAL},
    ...
};
std::string stringifyImageLayout(VkImageLayout);
VkImageLayout destringifyImageLayout(std::string);

This didn’t make it far, since this was locked-in with the Vulkan header that the files were generated for, essentially ruining the ability to use on more than one platform with varying versions of the Vulkan SDK.

This also had far too many functions to effectively manage or deal with, AND ran into issues where items lost vendor tags between versions.

Vulkan Spec Anomalies: Aliases

#

There are three types defined for an enum in the spec, and enum, bitset, or an alias.

Aliases, like the others are obviously aliases of other values. However a select few are of some interest, being from human error which made it through the process and are, presumably, with us for a long time if not forever:

<enums name="VkStencilFaceFlagBits" type="bitmask">
    <enum value="0x00000003" name="VK_STENCIL_FACE_FRONT_AND_BACK"/>
    <enum name="VK_STENCIL_FRONT_AND_BACK"
        alias="VK_STENCIL_FACE_FRONT_AND_BACK"
        comment="Alias for backwards compatibility"/>
</enums>

<enums name="VkPerformanceCounterScopeKHR" type="enum">
    <enum value="0"
        name="VK_PERFORMANCE_COUNTER_SCOPE_COMMAND_BUFFER_KHR"/>
    <enum value="1"
        name="VK_PERFORMANCE_COUNTER_SCOPE_RENDER_PASS_KHR"/>
    <enum value="2"
        name="VK_PERFORMANCE_COUNTER_SCOPE_COMMAND_KHR"/>
    <enum
        name="VK_QUERY_SCOPE_COMMAND_BUFFER_KHR"
        alias="VK_PERFORMANCE_COUNTER_SCOPE_COMMAND_BUFFER_KHR"/>
    <enum
        name="VK_QUERY_SCOPE_RENDER_PASS_KHR"
        alias="VK_PERFORMANCE_COUNTER_SCOPE_RENDER_PASS_KHR"/>
    <enum
        name="VK_QUERY_SCOPE_COMMAND_KHR"
        alias="VK_PERFORMANCE_COUNTER_SCOPE_COMMAND_KHR"/>
</enums>

<enums name="VkDebugReportObjectTypeEXT" type="enum">
    <enum value="33"
        name="VK_DEBUG_REPORT_OBJECT_TYPE_VALIDATION_CACHE_EXT_EXT"/>
    <enum
        name="VK_DEBUG_REPORT_OBJECT_TYPE_VALIDATION_CACHE_EXT"
        alias="VK_DEBUG_REPORT_OBJECT_TYPE_VALIDATION_CACHE_EXT_EXT"
        comment="Backwards-compatible alias containing a typo"/>
</enums>

I originally assumed that only enums/bitsets existed, and was putting these items in straight to the file. When hitting upon aliases, I was forced to ‘smarted up’ from just transcribing the items, to storing each enum set in a list of strings to iterate through for the rare times an alias did show up.

Of course, like any good edge case, aliases are only used in about 3 of the 167 current enum sets. Go figure.

Vulkan Spec Anomolies: Same but Different Enums

#

Along with stripping vendor tags from actual enum values, there was also the intention to do the same for the actual enum type names as well, since I assumed that such items, if mainlined, would be the same, except for the vendor tag being removed. Of course such nice things we do not deserve, as I quickly found this little oddity:

<enums name="VkExternalMemoryHandleTypeFlagBitsNV" type="bitmask">
    <enum bitpos="0"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT_NV"/>
    <enum bitpos="1"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_KMT_BIT_NV"/>
    <enum bitpos="2"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_IMAGE_BIT_NV"/>
    <enum bitpos="3"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_IMAGE_KMT_BIT_NV"/>
</enums>
<enums name="VkExternalMemoryHandleTypeFlagBits" type="bitmask">
    <enum bitpos="0"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_FD_BIT"/>
    <enum bitpos="1"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_BIT"/>
    <enum bitpos="2"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_OPAQUE_WIN32_KMT_BIT"/>
    <enum bitpos="3"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_BIT"/>
    <enum bitpos="4"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D11_TEXTURE_KMT_BIT"/>
    <enum bitpos="5"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_HEAP_BIT"/>
    <enum bitpos="6"
        name="VK_EXTERNAL_MEMORY_HANDLE_TYPE_D3D12_RESOURCE_BIT"/>
</enums>

or, when processed:

constexpr EnumValueSet VkExternalMemoryHandleTypeFlagBitsNVSets[] = {
    {"OPAQUE_WIN32_BIT", 0x00000001},
    {"OPAQUE_WIN32_KMT_BIT", 0x00000002},
    {"D3D11_IMAGE_BIT", 0x00000004},
    {"D3D11_IMAGE_KMT_BIT", 0x00000008},
};
constexpr EnumValueSet VkExternalMemoryHandleTypeFlagBitsSets[] = {
    {"OPAQUE_FD_BIT", 0x00000001},
    {"OPAQUE_WIN32_BIT", 0x00000002},
    {"OPAQUE_WIN32_KMT_BIT", 0x00000004},
    {"D3D11_TEXTURE_BIT", 0x00000008},
    {"D3D11_TEXTURE_KMT_BIT", 0x00000010},
    {"D3D12_HEAP_BIT", 0x00000020},
    {"D3D12_RESOURCE_BIT", 0x00000040},
};

Eeyup. Two enum sets, with the only typename difference being the NV vendor tag, but which some values are shifted over. For the NV set, OPAQUE_WIN32_BIT is 0x00000001 vs 0x00000002 for the regular flag set.

This singular set wrecked my idea for stripping vendor tags from names as well. Since if it can happen once, it could happen again in the future, a risk that can’t be taken.

Returning Failure

#

A problem I have when I implement other libraries is I can often skip checking error coes during the earlier stage of a project during prototyping, and often don’t go back and fix this glaring flaw.

I wanted to prevent this by forcing error cases upfront when interacting here. The original function signatures did not leave much room for this:

uint32_t parseEnum(std::string_view enumType, std::string value);
uint32_t parseBitmask(std::string_view enumType, std::string value);
std::string stringifyEnum(std::string_view enumType, uint32_t enumValue);
std::string stringifyBitmask(std::string_view enumType, uint32_t enumValue);

When parsing there are many valid zero-cases, and the same applies for stringified flag types. Some enums also just don’t have values yet.

In most libraries, beyond the use of exceptions, it’s usually the return value for error codes, or as a reference parameter.

However, C++17 allows for the use of std::optional<>, which I decided upon here. I was already locked-in to C++17 from the use of string_view, so I decided upon the simple pattern of:

If the enum/flag type/values are even found, then the optional will be filled with an empty value at the minimum. If it is not found, then the optional is returned with std::nullopt.

This neatly encompasses both returning whether the type/value was even found in the first place, and if so, then return concrete data, even if the data is nothing.

Final Capabilities

#

Like any real software project, I started with few nebulous goals, and as I became more aware of the problem space, only then did I solidify what I would be capable of reasonably doing. Eventually coming up with the following.

Be mostly back/forwards compatible

#

Many new enum/flag values are added or modified each iteration and may not be available on platforms with older Vulkan available.

For finding the matching type, it first tries with the vendor tag, and if that fails, strips out the vendor tag and tries again, as later spec versions the type may have been mainlined.

Finding values though, the vendor tag is always stripped out, as the main determining factor is the actual enum type first.

Not require Vulkan headers for generation/consumption

#

This means not dealing with the actually named types from Vulkan meaning having to consume/present values using basic uint32_t type.

Flag/bitset types

#

Allow for bitset types to be OR’d together. ie VK_DEBUG_REPORT_ERROR_BIT | VK_DEBUG_REPORT_DEBUG_BIT

Conversion to strings

#

• Removing the common prefix for the values, for example the VkImageLayout type, all of the values start with the VK_IMAGE_LAYOUT_ prefix. This would be tiresome and needlessly verbose for human-readable formats, so remove it’s requirement on stringified output.
• For bitset types, try to use combination/larger bitset values when possible, ie VK_CULL_MODE_FRONT_AND_BACK vs VK_CULL_MODE_FRONT_BIT | VK_CULL_MODE_BACK_BIT.

Conversion from strings

#

• Allow the prefix to be optional.
• Allowing for non-capitalized letters.
• Allow white space on the sides, being trimmed off.
• Allow spaces instead of underscores.

Source Code

#

Dev Log Video Playlist

#

A slow burn set of videos of me developing this in realtime can be found here.