I’d like to think that this is about information and education, and that if people knew how these things actually worked they’d be less surprised by it and in turn less inclined to trust and use it in all applications. So I will try to explain what “AI” is, how it works, and why you will never be able to trust its output.
Specifically in this case I am talking about Large Language Models (LLMs) which are the fashionable thing in AI these days.
At the highest level a LLM (Large Language Model) is a program which generates a sequence of words which are most likely to follow the previous sequence of words. The way it knows what word comes after the previous words is by scanning as many texts as possible and recording the connections between words. This is the “Model” of a Large Language Model. It has no knowledge of what the words actually mean, nor does it know if a sequence of words is factual, fictional, sarcasm, or just plain lies. All it knows is just the probabilities of which words could follow a given sequence of words.
A very simple example of this is presented in the ACCU 2025 talk titled “A Very Small Language Model” which is about building a very basic language model which can generate realistic looking text. A LLM is just a slightly more complicated version of this working with a massively larger set of word connections.
The massive Model is built by scanning (legally or otherwise) all the available written texts in existence. Naturally this includes texts which are accurate and factual, but also includes texts which are fictional, politically motivated, discredited, fraudulent, or just plain wrong. More importantly it lumps all of these texts together, meaning the connections between words in factual sentences are treated the same as in sentences containing falsehoods.
Scanning all written works in existence has caused authors and publishers to file lawsuits against these companies for copyright infringement, so far with mixed success. It has also caused online publishers to either close their archives or make licencing deals with these companies, and when that content has been user generated those sites have faced backlash from their users.
Now when you ask the LLM a question what it is actually doing is calculating which word is most likely to follow the words that have been fed into it. This not only includes your question, but also the previous chat history and control text which has been included by the company. It does this to generate one word, which it then adds to the input text and feeds the whole thing back into itself to generate the next word. This is shown in the way a LLM outputs text one word at a time, with the speed mostly being determined by the available processing power.
These calculations are also probabilistic meaning they include randomness in the result, as evidenced by a LLM generating different outputs for an identical input. This means that a LLM will not always pick the most likely word each time, but a word which is statistically likely to appear, which will then alter which words are likely to appear after.
A more mathematical explanation behind how a LLM works is provided in this delightful video, where it also contains links to videos explaining more of the theoretical foundations of the technology behind LLMs.
Given that a LLM is just a program which probabilistically generates a sequence of words that are statistically most likely to follow the input text that it was given, we should realise that it:
It just generates plausible looking text based on its input.
Therefore:
To reiterate, all a LLM does is generate a random sequence of statistically probable words based on an input. There is no knowledge, no thought, no creative decisions, and no internal consistency behind those words. Which means they can easily contain both true and false statements, or even a mixture of both in some bizarre combination. This makes the output unreliable as it cannot be trusted.
Unfortunately the AI industry has come to refer to these false statements as mere “hallucinations”, and claim that it is just a small problem that can be solved with more resources and time, rather than it being a fundamental issue with the technology itself.
This view is present in this article, where they at least admit that inaccurate and false statements are a fundamental problem of how LLMs work, but also say that the problems can be somewhat mitigated, and that we should just adapt to getting back unreliable information.
This seems completely counter-intuitive from a computer system which we have grown accustomed to being deterministic in its behaviour and output.
I really don’t like the use of the word “hallucinations” because it seems benign, like the LLM is having a temporary memory or mental issue, rather than calling it what it is, a “fabrication”, in both meanings of the word.
Another big problem is that LLMs can generate a lot of plausible looking text at a speed that is faster than humans can write. Given that the text is also unreliable and with questionable accuracy, it means that the costs of reading through and fact checking it has been externalised from the writer1 of the text to the people reading it. It is quicker and cheaper to generate the text, but it costs everyone else more to be able to process the text, even if they discard it.
There are notable examples of this happening in all manner of fields, some of which are very important to life and liberty.
So instead of a writer taking more time to write something good and succinct for people to read in a short time, LLMs vomit out a lot of text which takes other people’s time to read and process, even if they discard it quickly.
These days it is not just text that generative AI models like LLMs spew out, there’s images, audio, and video that also gets generated and is making it harder to distinguish fact from fiction.
Just realise that LLMs are just text generation programs, with no actual knowledge inside of them, and no capability to do actual data processing. With this in mind you can see what tasks a LLM might be useful for - emitting plausible looking text, and what tasks it is wholly unsuited for - emitting factual text, performing calculations, answering questions.
For some it might (seem to) be a useful tool which helps them do things and write what they need to write, but realise that you always need to check and verify its output with actual non LLM sources, as you the writer1 are ultimately liable for what is in the text that the LLM generates.
“A computer can never be held accountable, therefore a computer must never make a management decision”. - IBM Training Manual, 1979
As for me I see no present value in using LLMs for generating content, as we have existing proven technologies for many of its use claimed cases, I have no interest in spending time to fact check its output2, and I prefer to write code myself. I may experiment with it now and again, and if there comes a time in the future when LLMs have an actual useful use case I can always try it then.
I had originally intended to only present a longer version of my talk from CppCon about Optimising Data Building in Game Development, but I was unsure if it would get accepted for the conference. So I quickly thought about and wrote a shorter presentation about various mistakes that I’ve encountered when handling data while making games. Fortunately (or unfortunately) both of my presentations were accepted for the conference.
I’m posting this now, quite a long time after the event, as both videos for my talks are now up on YouTube and available for all to see, please check them out.
The conference itself felt a lot more community focused, smaller, and a lot less intimidating than CppCon, and covering a wider array of topics than just pure C++ programming. This was actually kind of refreshing because it meant that not every talk was a deep discussion of the technical details of C++, and you got to meet a whole range of different friendly people.
One of the unfortunate things for people reading this is that not all the talks were recorded - at the presenters request, one of which was a talk about programming a server for Ultima Online.
Out of the talks that are online I got the most value from seeing:
To anyone at the conference that is reading this, it was a pleasure meeting you and we’ll hopefully get the chance to again sometime.
]]>At its core the technique consists of the following items:
consteval function which creates and returns an array of function pointers.constinit variable at a global/file scope.The generated assembly for this technique ends up being a simple jump table as data and a handful of instructions used to index into it and then jump to the specified address1.
To best illustrate this technique I’m going to use a simple example that you may find in an interpreter, with an enum class that represents a type and a tagged union value struct which holds the payload, like so:
enum class Type : uint8_t
{
Void, Bool, Signed, Unsigned, Float, String, // ...
COUNT
};
struct RuntimeValue
{
Type type;
union { /* ... */ } value;
};
With these types defined I can use them to write the code for the rest of this simple example, where we only need to handle a single argument.
Note that I’m using the overloaded unary operator+ technique described in another article to make the code more concise. Likewise using unary operator+ on a lambda with no bound variables turns it into a function pointer.
// (1)
using UnaryTypeFunction = void (*)(const RuntimeValue&);
using UnaryTypeTable = std::array<UnaryTypeFunction, +Type::COUNT>;
// (2)
consteval UnaryTypeTable MakeUnaryActionTable()
{
UnaryTypeTable result{};
result[+Type::Bool] = +[](const RuntimeValue&) { /* Boolean stuff */ };
auto integer_function = +[](const RuntimeValue&) { /* Integer stuff */ };
result[+Type::Signed] = integer_function;
result[+Type::Unsigned] = integer_function;
result[+Type::Floating] = +[](const RuntimeValue&) { /* Floating Point stuff */ };
return result;
}
// (3)
static constinit auto unary_type_action_table = MakeUnaryActionTable();
// (4)
void DoUnaryTypeAction(const RuntimeValue& value)
{
auto function = unary_type_action_table[+value.type];
if (function != nullptr)
{
(*function)(value);
}
// Emit error...
}
The example above is split up into four sections, all of which can be implemented in the source (.cpp) file, and the first three of which can be put into a private or internal namespace to hide the implementation details and make the code somewhat cleaner.
Helper type definitions which simplify using the function pointer and function pointer table definitions in the code. In this example these are UnaryTypeFunction and UnaryTypeTable. These are not strictly necessary but they do make the code clearer and more concise.
A make table function MakeUnaryActionTable which wraps up creation of the table.
There are a few things to note with this function:
consteval is key, as it guarantees that the function will be evaluated at compile time or a compiler error will be emitted.result variable will assign nullptr to all entries. Therefore there’s no uninitialised memory in the table, and no need to explicitly specify every entry, only the entries that we want to do something.integer_function variable in the example above. This also means that we can call other consteval functions and pass in function pointers to them and also modify the tables that they return.Calling the MakeUnaryActionTable and storing its result in a constinit variable, ensuring that if we cannot assign it at compile time then the compiler will emit an error.
You can also use an immediately invoked consteval lambda to create the table if that is your preference, but I much prefer to use an explicit function instead, as you can give it a proper name and it is more consistent when you want to call other helper consteval functions.
The DoUnaryTypeAction function which actually performs the action at runtime. This is the public facing API of this technique, with the rest being implementation details. This function can also be a member function of a class even if the other components are not.
This technique actually shines when you have to execute code based on multiple input arguments, which would normally result in a complicated tree of conditional if/else and switch statements, but instead you encode the functions in a multidimensional lookup table, and then do a single lookup to figure out which code to run.
So in order to handle multiple arguments the previous example expands to this:
using BinaryTypeFunction = RuntimeValue (*)(const RuntimeValue&, const RuntimeValue&);
using BinaryTypeTable = std::array<std::array<BinaryTypeFunction, +Type::COUNT>, +Type::COUNT>;
consteval BinaryTypeTable MakeBinaryActionTable()
{
BinaryTypeTable result{};
result[+Type::Bool][+Type::Bool] = +[](const RuntimeValue&, const RuntimeValue&) { /* Boolean stuff */ };
auto integer_function = +[](const RuntimeValue&, const RuntimeValue&) { /* Integer stuff */ };
result[+Type::Signed][+Type::Signed] = integer_function;
result[+Type::Signed][+Type::Unsigned] = integer_function;
result[+Type::Unsigned][+Type::Signed] = integer_function;
result[+Type::Unsigned][+Type::Unsigned] = integer_function;
auto integer_float_function = +[](const RuntimeValue&, const RuntimeValue&) { /* Integer and Floating Point stuff */ };
result[+Type::Signed][+Type::Floating] = integer_float_function;
result[+Type::Unsigned][+Type::Floating] = integer_float_function;
result[+Type::Floating][+Type::Signed] = integer_float_function;
result[+Type::Floating][+Type::Unsigned] = integer_float_function;
result[+Type::Floating][+Type::Floating] = +[](const RuntimeValue&, const RuntimeValue&) { /* Floating Point stuff */ };
return result;
}
static constinit auto binary_type_action_table = MakeBinaryActionTable();
RuntimeValue DoBinaryTypeAction(const RuntimeValue& left, const RuntimeValue& right)
{
auto function = binary_type_action_table[+left.type][+right.type];
if (function != nullptr)
{
return (*function)(left, right);
}
// Emit error...
return RuntimeValue{};
}
As you can see the code hasn’t changed much at all but now we can select a function based on two input parameters instead of one.
The main down side of this is that the lookup table can grow quite large in size, as each entry is sizeof(void*) (which is usually 8 bytes on modern platforms), multiplied by Type::COUNT raised to the number of parameters/dimensions in the table. So in the example above it would have 36 entries and be 288 bytes in size.
Another use case where this technique can be applied is in writing a lexer or other similar type of parser which deals with characters. The main benefit in this case is being able to write clear concise code which handles the different starting characters without needing to specify the full table in code or have every valid character in a gigantic switch statement.
An example of the make table function for a simple lexer is:
using CharFunction = bool (*)(const char*&);
using CharLookupTable = std::array<CharFunction, 256>;
consteval CharLookupTable MakeLookupTable()
{
CharLookupTable table{};
// (1)
auto error_func = +[](const char*&) { /* Handle Error */ };
for (int i = 0; i < 256; ++i)
{
table[i] = error_func;
}
// (2)
auto handle_word = +[](const char*&) { /* Handle word */ };
table['_'] = handle_word;
for (char c = 'a'; c <= 'z'; ++c)
{
table[c] = handle_word;
}
for (char c = 'A'; c <= 'Z'; ++c)
{
table[c] = handle_word;
}
auto handle_number = +[](const char*&) { /* Handle number */ };
for (char c = '0'; c <= '9'; ++c)
{
table[c] = handle_number;
}
auto handle_symbol = +[](const char*&) { /* Handle symbol */ };
table['+'] = handle_symbol;
table['-'] = handle_symbol;
// etc
return table;
}
Which can be used in code like so:
void Process(const char*& current)
{
constexpr auto lookup_table = MakeLookupTable();
while (SkipWhitespaceAndComments(current))
{
auto result = lookup_table[*current](current);
if (!result) break;
}
}
The main difference in this example is in the way invalid or error entries are encoded in the table.
In this case every table entry is filled with a pointer to an error handling function that handles an unexpected or illegal character (labelled as (1) in the example). Then the entries which handle specific cases are set up (labelled as (2) in the example), such as the alphabet characters handling the start of a word or the digits handling the start of a number.
Because this function is evaluated at compile time we don’t need to worry about how many times we assign the entries in the table, as the table will be stored as data in the compiled code.
This also results in a more efficient way of handling errors as there doesn’t need to be an explicit check for a null pointer being stored in the table, and instead the function can just be executed directly.
Note that in this case an unsigned 8 bit number is used to index into the array of 256 entries, meaning that there’s no way to index out of bounds and therefore no need to check that. If using a smaller array size, especially one that is not a power of two, that a bounds check will be needed.
I’ve used both methods of error handling in my code, and which method to use depends on a variety of factors. When using a table with null pointers then the error checking is done in the function doing the table lookup, and therefore you might have a better error handling context. In the case above you handle the errors in the error handling functions, so either the error handling need to be simpler (or global), or you need to pass in the context to every function in the table.
As far as performance goes you can only tell by implementing both and measuring to see which method is faster, and with this approach it is easy to switch from one error handling strategy to another by just changing the default table entries.
One last detail in using this technique is how to use it to call member functions rather than just calling free functions. There are two main methods to accomplish this, either by storing pointers to member functions, or by binding lambdas which get passed the object in as a parameter and then call the required function.
The main issue with the first method is that the size of the pointer-to-member-function may be larger than a regular pointer-to-function, and therefore drastically increase the size of the lookup array. In the simplest case with a basic style class the sizes will be the same so that’s not an issue, but in more complicated classes with inheritance and virtual functions the size will end up being bigger. It could also be an issue that the pointer-to-member-function syntax in C++ is not used that often, so it will be unknown to some people.
Therefore my preferred method is to bind simple lambdas that take a reference to the class as an additional parameter and then just call the desired function on the object in the lambda. It can be best illustrated with the example below:
class Lexer
{
public:
void HandleError();
void HandleWord();
void HandleNumber();
void HandleSymbol();
};
consteval CharLookupTable MakeLookupTable()
{
CharLookupTable table{};
auto error_func = +[](Lexer& lexer) { lexer.HandleError(); };
// ...
auto handle_word = +[](Lexer& lexer) { lexer.HandleWord(); };
// ...
auto handle_number = +[](Lexer& lexer) { lexer.HandleNumber(); };
// ...
auto handle_symbol = +[](Lexer& lexer) { lexer.HandleSymbol(); };
return table;
}
With this method you end up with somewhat simpler and cleaner looking code, and guaranteed smaller size of table than by using pointer-to-member functions. Though one issue is that you need to make the member functions public as they need to be called from functions outside the class.
This happens to be very similar (if not identical) to assembly code generated by compilers for a switch statement. ↩
enum class into its underlying type, which I will describe in this article.
In modern C++ we have the enum class construct (or enum struct) which should be used as it fixes some issues with C style enums1. One feature of this is that we have to explicitly cast to get an integer value, and when you’re needing the underlying value in many places then it becomes a burden.
The correct way to do this is to do a static cast to the underlying type of the enumeration. In C++ this is properly done by using the expression static_cast<std::underlying_type_t<EnumType>>, but as you can see it is pretty verbose. If it only needs to be used in a handful of places then using this construct will be fine, but if it needs to be used multiple times in the same piece of code then it will be a distraction from the actual intent of the code.
Some might want to use C-style casts, and while it does reduce the total amount of characters in the code, it brings with it many issues and removes the type safety that using an enum class provides. So in this case you might as well just use a plain enum instead, and if you’re fine with this you can stop reading here.
Some others might just explicitly put the type into the static_cast<> in order to remove the rather verbose std::underlying_type_t<> part of the expression. While this is a much better choice than a C-style cast, it will still require a lot of code changes if the underlying type of the enum ever changes, and there is a potential for silent problems if someone forgets to update a cast to the new type.
Now this is where we can overload the unary operator+ for our specific enun class in order to create a convenient notation to convert the enum to its underlying type. This ends up being a pretty simple bit of code, just a function which wraps up the static cast to the underlying type, like so:
enum class Foo : uint16_t;
constexpr auto operator+(Foo value)
{
return static_cast<std::underlying_type_t<Foo>>(value);
}
Which can be used in code like this:
enum class Foo : uint8_t
{
A, B, C, // ...
COUNT,
};
constexpr auto operator+(Foo value)
{
return static_cast<std::underlying_type_t<Foo>>(value);
}
constexpr Value ValueFromFoo(Foo foo)
{
static constexpr Value values[+Foo::COUNT] = { /* ... */ };
return values[+foo];
}
Additionally in Visual Studio you can add the [[msvc::intrinsic]] attribute to the operator function it to make it more performant in debug builds2.
However this is not the best example of where a function like this is needed, as there are only two places where the unary operator+ is used. I will present a much more extensive example of where this can be used to great benefit in a future article.
Lastly a word of caution, only use this where it is absolutely necessary, preferably only defining it in C++ source files near where it is needed, and only for enum classes that actually need the functionality. My reasoning for this is that this is a single character function converts a strongly typed enumeration to an integral type which can easily and accidentally be used in arithmetic operations. Same as you wouldn’t automatically provide an operator++ and operator-- for an enum class unless you actually need to iterate over it in some code.
Issues include leaking enumeration names into the enclosing namespace and implicit conversion to integer types. ↩
The main talk will be an expanded version of my CppCon talk from last year titled “Optimising Data Building in Game Development”, going into more detail and hopefully with a more consistent presentation. The second talk will be a shorter presentation on mistakes that people have made in handling data during game development.
Originally I had only intended to do the one main presentation, but after submitting the proposal I felt that it would get rejected as I had already presented it. So I came up with some other ideas for shorter talks and submitted the best one, and now I am presenting both.
If you happen to be at the conference then I would be delighted if you would attend my talks, or even meet me in the hallway to say hi.
]]>Take note that these are just guidelines and one size will not fit all situations, so feel free to mix and match them as required. I’ve ordered them roughly from most readable and least packed to least readable and most packed.
The most important thing to remember is that the order of initialisation of the member variables happens in the order they are declared in the class definition, not in the order they are listed in the initializer list in the constructor. There is a C++ core guideline which says to keep the order of members in the class definition and in the initializer list the same.
The main effect of this in regards to the layout of data members is that you will want to initialise some data members before others, especially when they are not just simple assignments. For example computing the size from width and height parameters before allocating an array to store the data.
The first (and probably main) method of organisation is to group related members together so that they are logically close in the source code. An example of this is putting members related to storage of data in one group, and members related to efficiently finding the data in another group. I’d like to think that people do this naturally, grouping related members together surrounded with whitespace, but it may just be me.
One could argue that you don’t need to create these groupings because in principle a class should only do one thing, so all its members should be in one group. However in reality classes can do one thing but still contain many members and syb-systems used to accomplish that, or they are responsible for several things so it makes sense to group the members for those logically to aid readability.
One down side of this method is that you’re likely to get plenty of padding inside and in between groups of members, as smaller data members such as bool, int, etc are placed next to larger data structures which have bigger alignment requirements. Though depending on the class it might not matter as we will discuss below.
Related to the above method is to group members by usage together, so that they are physically close to each other in memory, and more likely to be on the same cache line in the processor. The main difference between the previous method and this is that you’re taking into consideration how the data is used and not just what part of the system it logically belongs to.
This can reduce readability of the class definition but in the same vein it can also increase performance in some specific circumstances.
The next major method of organisation is to arrange the members of a class in a way that minimises padding and wasted space within the class. This is especially useful when you’re creating many thousands or millions of instances of these classes, as each byte of wasted space becomes significant in aggregate. This also helps to efficiently pack these classes contiguously in memory, but it can come at a cost to readability.
An important detail to remember here is that a class’s size is a multiple of its alignment, and its alignment is the highest alignment of its members. This means that padding will be inserted after the last data member to make the class’s size a multiple of its alignment.
The simplest way to minimise padding is to put the elements with the highest alignment requirements at the beginning of the class, followed by members with successively smaller alignment requirements, with byte sized elements like bool or uint8_t at the end. Of course in doing this there may be gaps created in between the larger members, and in this case fill the gaps by moving any appropriately sized elements in between the larger ones. If all goes well then there should not be any padding, or only a few bytes of padding at the end of the class.
Think of this as filling a jar with various sized rocks, first put in the biggest rocks (big members), then fill the gaps in with smaller pebbles (smaller members, integers, etc), and finally pour in the sand to fill in the remaining space (bytes and booleans).
This works best with more plain-old-data style classes and structures that contain many smaller sized members that themselves have smaller alignment requirements and do not have any internal padding. If larger classes are used be aware that they might have their own internal padding which is created automatically due to their alignment requirements. A simple example of this is:
struct BufferView // Size: 16 bytes, Alignment: 8 bytes
{
void* data; // Size: 8 bytes, Alignment: 8 bytes
int size; // Size: 4 bytes, Alignment: 4 bytes
// Padding: 4 bytes
};
Making this a member of another class will introduce 4 bytes of padding each time it is used, even if it is followed by a 4-byte value which would otherwise fit within the padding.
The remedy for this situation is to use compiler-specific attributes and pragmas to specify the packing of elements within the classes. For MSVC this involves surrounding the class declaration(s) with a pragma pack declaration like so:
#pragma pack(push, 1) // Could also be 4 instead of 1
struct BufferView // Size: 12 bytes, Alignment: 1 byte
{
void* data; // Size: 8 bytes, Alignment: 8 bytes
int size; // Size: 4 bytes, Alignment: 4 bytes
};
#pragma pack(pop)
Likewise on GCC and Clang you can use the packed attribute to tell the compiler to pack the members tightly. Note that the packed attribute has to be applied to each class declaration separately in order to pack the elements as tightly as possible.
Visual Studio 2022 version 17.8.0 introduced a neat little feature to show the size of a type or value in a tooltip, it helps as you can quickly and easily see what effect moving a member has on the size of the class. There are also other plugins which help visualise class members and padding, though I do not use them.
In some cases just packing the elements is not enough, as the size of the members just exceeds a multiple of the alignment, thereby creating a relatively large amount of padding at the end of the class.
These techniques can be used to combat this, helping reduce the size of the data and making it fit more nicely within a multiple of the alignment. This becomes important when these classes are stored contiguously and processed by performance critical code, as more instances can be packed within the same amount of memory.
The first technique is to combine bool values into a bitfield, as traditionally each bool value takes a byte. This can be done by having an integer and manually using masks, or by declaring a C++ bitfield. One trick I learned is to create a bitfield using bool, like bool a : 1;, bool b : 1; etc, which has the advantage of being descriptive but also combining adjacent values together.
So instead of something like this:
struct Example
{
bool a;
bool b;
bool c;
};
static_assert(sizeof(Example) == 3);
It would instead be something like this:
struct Example
{
bool a : 1;
bool b : 1;
bool c : 1;
};
static_assert(sizeof(Example) == 1);
The next technique is to bitpack smaller integer values (and enumerations) together in a larger field. For example when you have 3 integer values that range only from 0 to 1000 you can store them as three 10 bit values inside a single uint32_t instead of one for each value. The left-over bits can be used for flags.
struct Example
{
uint32_t r : 10;
uint32_t g : 10;
uint32_t b : 10;
uint32_t a : 2;
};
static_assert(sizeof(Example) == sizeof(uint32_t));
A more advanced version of this would be to use some other encoding scheme to store multiple values in the same integer variable. This can be something simple like encoding in the same way that multidimensional array indexes are calculated. For example storing values a, b, c, as: v = (a + A * (b + B * c)), and then decoding it using / and %. There are other encoding schemes that could be used, but the more complex the encoding is the slower it will be to interact with the values.
An example of the simple multidimensional array index calculation:
uint64_t encode(uint32_t x, uint32_t y, uint32_t z)
{
return x + MAX_X * (y + MAX_Y * z);
}
void decode(uint64_t value, uint32_t& x, uint32_t& y, uint32_t & z)
{
z = value % MAX_Y;
value /= MAX_Y;
y = value % MAX_X;
value /= MAX_X;
x = value;
}
If dealing with floating point values then a good technique is to quantise them and store them in smaller integer types. The simplest version is to store normalised values (ranging from 0.0 to 1.0) in unsigned integer types like uint8_t or uint16_t, where 0.0 maps to 0 and 1.0 maps to 255 or 65535 respectively. Values ranging from -1.0 to 1.0 can be similarly stored in signed integer types, and as long as the range is limited the values can be quantised into a smaller type while retaining reasonable accuracy. The tricky part is that the range has to be defined in code and not in data for there to be a reduction in size.
For example to quantise a value between 0 and 1 into a smaller integer:
uint8_t encode(float value)
{
return static_cast<uint8_t>(value * 255.0f);
}
float decode(uint8_t value)
{
return value / 255.0f;
}
Another technique which is common in graphics programming is to drop elements of compound types when they can easily be recomputed. When dealing with normalised vectors and unit quaternions, in addition to quantising their values, one of them can be dropped entirely. One caveat to that is the sign of the dropped element must be stored somewhere or explicitly known externally, as squaring a number will make it positive.
A lot of this only matters if the class you’re writing will have many (millions of) instances created of it and therefore you need to organise your data for optimum efficiency. In all other cases you should make it easy to read and easy to understand.
To summarise:
I’m posting this now quite a few months after the event, as I’ve been busy with work and other projects. Also the video of my presentation has been officially released, with the other videos for the event also slowly trickling out on the CppCon YouTube Channel,1.
This was my first ever public presentation and I was a bit nervous, so I’m thankful to my friends and colleagues who helped me do practice runs of the presentation at home and at work before going. This was incredibly helpful since at the even I had some technical difficulties with my laptop refusing to cooperate with the AV equipment2, so I had to give the talk from memory with no speaker notes, just the slides on screen.
The event itself was kind of intense, I tried to attend every talk, especially the ones that weren’t being recorded (the open sessions in the morning, during the lunch break, and in the evening). Meaning most days started at 8am and finished at 10pm, though with breaks in between. Luckily for me the hotel was pretty cool, with an impressive view of the rocky mountains in Colorado, and warm pools to soak in.
If people are interested in C++ I would definitely recommend going to CppCon, I had a blast there and would love to go again.
]]>The talk is titled “Techniques to Optimise Multi-threaded Data Building During Game Development”, and I know it’s quite a mouthful but they said to be detailed.
This will be my first big public presentation and I am honoured to have been selected. So now I am deep into preparing and practicing the presentation, writing, coding, editing, and everything else that comes with it. I’ll see what I can release publicly after the conference but at least the slides will be available on the CppCon github, and if I can I’ll post something on my own Github.
If you happen to be at the conference then I would be delighted if you would attend my talk, and if not then please say hi!
]]>The first update was to add the [[unlikely]] attribute to the assert condition. This will tell the compiler to generate the assembly code under the assumption that the condition will not be true at runtime (but not with the assumption it will never be true).
This actually changes the generated assembly quite a bit in places, moving the handling of the assert to the end of the function and out of the immediate code to execute. While I haven’t measured any performance impact of this change, the assembly code looks tidier and because the regular function code doesn’t need a jump to reach it should be more efficient.
Before Unlikely
; 9 : ASSERT(name.length() <= 255);
cmp QWORD PTR [rcx+16], 255 ; 000000ffH
mov rbx, rcx
jbe SHORT $LN2@Example
lea rax, OFFSET FLAT:??_C@_08MPNMAILL@Test?4cpp@
mov DWORD PTR $T1[rsp], 9
mov QWORD PTR $T1[rsp+8], rax
lea rdx, OFFSET FLAT:??_C@_0BF@CFNDKGCM@name?4length?$CI?$CJ?5?$DM?$DN?5255@
lea rax, OFFSET FLAT:??_C@_0HF@HDFDIDJI@int?5__cdecl?5Example?$CIconst?5class@
mov DWORD PTR $T1[rsp+4], 2
lea rcx, QWORD PTR $T1[rsp]
mov QWORD PTR $T1[rsp+16], rax
call ?handle_assert@error@@YAXUsource_location@std@@PEBD@Z ; error::handle_assert
int 3
call ?terminate@std@@YAXXZ ; std::terminate
int 3
$LN2@Example:
; ... Normal function code here ...
After Unlikely
; 9 : ASSERT(name.length() <= 255);
cmp QWORD PTR [rcx+16], 255 ; 000000ffH
mov rbx, rcx
ja SHORT $LN20@Example
; ... Normal function code here ...
$LN20@Example:
; 9 : ASSERT(name.length() <= 255);
lea rax, OFFSET FLAT:??_C@_08MPNMAILL@Test?4cpp@
mov DWORD PTR $T1[rsp], 9
mov QWORD PTR $T1[rsp+8], rax
lea rdx, OFFSET FLAT:??_C@_0BF@CFNDKGCM@name?4length?$CI?$CJ?5?$DM?$DN?5255@
lea rax, OFFSET FLAT:??_C@_0HF@HDFDIDJI@int?5__cdecl?5Example?$CIconst?5class@
mov DWORD PTR $T1[rsp+4], 2
lea rcx, QWORD PTR $T1[rsp]
mov QWORD PTR $T1[rsp+16], rax
call ?handle_assert@error@@YAXUsource_location@std@@PEBD@Z ; error::handle_assert
int 3
call ?terminate@std@@YAXXZ ; std::terminate
int 3
The next investigation was trying various ways to integrate a check to see if a debugger was attached before triggering the debug break. The main goal behind this was that when running the program with a debugger attached it would trigger the breakpoint and allow the programmer to see the assert that was triggered, and when running outside of a debugger it would just terminate without triggering a breakpoint.
The simplest way of doing this was to wrap the __debugbreak() (or DebugBreak()) with a check like if (IsDebuggerPresent()) { ... }. Doing this added a function call, a test, and a jump to the assert code, which in most cases made the code significantly larger. It also required forward declaring or including debugapi.h, into what otherwise is a fairly low level header.
Another way of doing this was to move the IsDebuggerPresent() call to be inside the handle_assert function, and have that return a boolean indicating if the breakpoint should be triggered or not. This eliminated a function call instruction from the assert macro but it didn’t clean up the assembly all that much.
Overall I wasn’t happy with either of these solutions so I ended up looking for alternatives, but not ones which would require me to implement magical assembly or weird intrinsics. (For reference most alternatives involved manually implementing the IsDebuggerPresent() function by looking up the debugger present flag in the thread information block in Windows. As such I didn’t want the support burden to keep this up to date with newer versions of Windows.)
It was when I was investigating how to handle other program faults that I realised that asserts (and error handling in general) need to be handled differently in developer and retail versions of the program. During development you want to use breakpoints to catch problems early, either by running in a debugger or by being able to attach one as easily as possible. However in retail mode you cannot do that so you want to create a detailed error report with plenty of supporting information, and send that to yourself as a package in order to try and figure out what went wrong.
This means that a separate retail version of the assert macro and assert handler function will need to be created, though that can be done at a later time together with a more thorough error reporting system.
The last thing to add was a call to std::terminate() inside the macros to actually make the asserts fatal and exit the program.
One interesting thing discovered by doing this was that in some cases adding the terminate function to the macro caused the compiler to move the implementation of the assert contents to the end of the function, in a similar way as when adding the unlikely attribute. But it did not do this in every situation, therefore using the unlikely attribute is still a good idea.
]]>This is the simplest method to understand and implement, it’s the logic that you would write within an if statement just wrapped up in a convenient and descriptive function. In general you would write a function for each character classification that you need to distinguish.
There are two main ways of performing the tests, checking a range of characters such as '0' - '9', and testing individual characters such as '$'. The way I’ve written the examples below is designed to make it easy to read the ranges being tested in each function.
constexpr bool IsNumber(char c)
{
return '0' <= c && c <= '9';
}
constexpr bool IsAlpha(char c)
{
return ('A' <= c && c <= 'Z') || ('a' <= c && c <= 'z');
}
constexpr bool IsWhitespace(char c)
{
return c == ' ' || c == '\t' || c == '\r' || c == '\n';
}
These can also be broken down into more specific functions like IsLower and IsUpper, and combined to create character classifiers of a more broad type. By using the C++ constexpr keyword it pretty much guarantees (when compiled with any optimisation level enabled) that the function code will be inlined rather than cause a function call in assembly. So much so in fact that I had to remove the constexpr or make secondary non-constexpr functions when testing in order to see the assembly output for GCC and Clang. In some ways MSVC is nice in that it emits the assembly code for inlined and constexpr functions anyway.
constexpr bool IsNumber(char c)
{
return '0' <= c && c <= '9';
}
constexpr bool IsLower(char c)
{
return 'a' <= c && c <= 'z';
}
constexpr bool IsUpper(char c)
{
return 'A' <= c && c <= 'Z';
}
constexpr bool IsAlpha(char c)
{
return IsLower(c) || IsUpper(c);
}
constexpr bool IsAlphaNum(char c)
{
return IsAlpha(c) || IsNumber(c);
}
Note that this is just a short selection of functions and not an exhaustive set that one might need.
In examining the assembly generated by each compiler for the code in this article I made some interesting observations.
In general each compiler will attempt to optimise the code to the best of their ability, and some common optimisation techniques are:
There are also differences between compilers in what assembly instructions they generate, with which you can make some generalisations:
For functions which only test a single range, such as IsNumber, all compilers effectively generate code similar to:
IsNumber(char):
sub cl, 48
cmp cl, 9
setbe al
ret 0
Which in C++ is equivalent to:
bool IsNumber(char c)
{
return (c - 48) <= 9; // '0' == 48
}
This also ends up being the general pattern used for testing ranges, where the minimum of the range is subtracted from the value and then its tested against the length of the range. In this way only a single comparison is needed rather separately testing and branching for each of the minimum and maximum.
For functions which test multiple ranges the compilers generate different code at all optimisation levels. Using the IsAlphaNum function as our subject for comparison and compiling at the O2 optimisation level we can clearly see the differences.
MSVC generates assembly code which most accurately reflects the C++ language semantics in the original code. It tests each condition in an optimised form but then jumps to the end if true, mirroring the short-circuit evaluation of the original C++ source code.
bool IsAlphaNum(char) PROC
lea eax, DWORD PTR [rcx-65]
cmp al, 25
jbe SHORT $LN3@IsAlphaNum
lea eax, DWORD PTR [rcx-97]
cmp al, 25
jbe SHORT $LN3@IsAlphaNum
sub cl, 48
cmp cl, 9
jbe SHORT $LN3@IsAlphaNum
xor al, al
ret 0
$LN3@IsAlphaNum:
mov al, 1
ret 0
bool IsAlphaNum(char) ENDP
The MSVC implementation in the code above uses the
leainstruction to compute the initial subtraction of the minimum value before testing. For example the firstleacomputeseax = ecx - 65.
GCC actually does a similar thing, where it jumps to the end if the first IsAlpha condition is true, but it only has a single branch as the assembly it generates for IsAlpha has no branches.
IsAlphaNum(char):
mov eax, edi
mov edx, 1
and eax, -33
sub eax, 65
cmp al, 25
jbe .L6
sub edi, 48
cmp dil, 9
setbe dl
.L6:
mov eax, edx
ret
In the GCC implementation it uses
andto make make the character upper case and then performs the test on that. As-33is1101 1111in binary.
Clang on the other hand follows the intention of the function and generates assembly which produces the right result but does not strictly represent the language semantics of the C++ code as written. Specifically it does not perform any short-circuit evaluation of the logical code and just tests all conditions, combining the result at the end.
IsAlphaNum(char):
mov eax, edi
and al, -33
add al, -65
cmp al, 26
setb cl
add dil, -48
cmp dil, 10
setb al
or al, cl
ret
My intuition says that the branch-less code that Clang produces should run marginally faster than the other code with branches, as getting a branch misprediction costs many cycles, where as executing a handful more simple instructions would almost be free.
It is generally advisable when writing functions to test equality to put the test that partitions the search space the most, first. For example the test that can eliminate 90% of values should go before the test that eliminates only 50%.
To apply it to this case would mean putting the test c <= '9' (which eliminates 184 values) before c >= '0' (which eliminates 60 values), and likewise swapping the order of tests for the alphabet ranges.
However when investigating this with Compiler Explorer these changes generally had no effect on the generated assembly code, but in some cases made the assembly code worse.
IsNumber there was no difference in the generated assembly.IsAlpha the generated assembly was slightly larger and contained branching on all compilers.So in this case the idea to take away from this is to write simple straightforward code that is easy for both people and the compiler to understand.
The other type of tests involved directly compare against individual characters rather than ranges of characters. An example of this is the IsWhitespace function from the beginning of the article, though here is a more complete version which tests all of the white-space characters including the lesser known form feed ('\f', 12 dec, 0x0C hex) and vertical tab ('\v', 11 dec, 0x0B hex).
I was not actually aware of these characters myself until I started looking into the Clang and Carbon compiler source code and then cross-referencing with the ASCII table.
bool IsWhitespace(char c)
{
return c == ' ' || c == '\t' || c == '\v' || c == '\f' || c == '\r' || c == '\n';
}
MSVC compiles this into a couple of tests, one for the space character (32 dec, 0x20 hex), and one for the range of white-space characters (From 0x09 to 0x0D).
bool IsWhitespace(char) PROC
cmp cl, 32
je SHORT $LN3@IsWhitespa
sub cl, 9
cmp cl, 4
jbe SHORT $LN3@IsWhitespa
xor al, al
ret 0
$LN3@IsWhitespa:
mov al, 1
ret 0
Clang compiles this code into a range check and test against a computed bit mask, combining the result together using logical operations.
IsWhitespace(char):
cmp dil, 33
setb cl
movabs rax, 4294983168
bt rax, rdi
setb al
and al, cl
ret
GCC however compiles to something more interesting where it creates assembly code which both uses a lookup table to check most of the values and then explicitly checks for carriage-return (13 dec, 0x0D hex) and line-feed (10 dec, 0x0A hex).
IsWhitespace(char):
cmp dil, 32
ja .L12
movabs rax, 4294973952
bt rax, rdi
setc al
test al, al
je .L12
ret
.L12:
cmp dil, 13
sete al
cmp dil, 10
sete dl
or eax, edx
ret
My suspicion was that GCC tried to honour the ordering of the comparisons in the written C++ code, and it managed to collapse the first set of 4 comparisons into a bit field lookup, but it did not do so for the last two.
This was confirmed when I sorted the comparisons in the C++ function to match the ASCII values.
bool IsWhitespace(char c)
{
return c == '\t' || c == '\n' || c == '\v' || c == '\f' || c == '\r' || c == ' ';
}
As both GCC and Clang generated nearly identical and heavily optimised assembly. (This is the GCC version)
IsWhitespace(char):
lea eax, [rdi-9]
cmp al, 4
setbe al
cmp dil, 32
sete dl
or eax, edx
ret
With MSVC generating the same tests but branching to return the result and therefore adhering to the short-circuit evaluation of the C++ code.
IsWhitespace(char) PROC
lea eax, DWORD PTR [rcx-9]
cmp al, 4
jbe SHORT $LN5@IsWhitespa
cmp cl, 32
je SHORT $LN5@IsWhitespa
xor al, al
ret 0
$LN5@IsWhitespa:
mov al, 1
ret 0
A more complete example is the IsSymbol function below, which I’ve written to classify all symbols used in ASCII just as they appear on my keyboard (US layout).
bool IsSymbol(char c)
{
return c == '~' || c == '`' || c == '!' || c == '@'
|| c == '#' || c == '$' || c == '%' || c == '^'
|| c == '&' || c == '*' || c == '(' || c == ')'
|| c == '_' || c == '-' || c == '+' || c == '='
|| c == '[' || c == ']' || c == '{' || c == '}'
|| c == '|' || c == '\\' || c == ';' || c == ':'
|| c == '\'' || c == '"' || c == ',' || c == '.'
|| c == '<' || c == '>' || c == '/' || c == '?'
;
}
In this initial version MSVC ends up with the smallest and possibly cleanest assembly code. It performs a quick test against a short range at the end of the ASCII sequence (for '{', '|', '}', and '~'), then checks the value is within the desired range before doing a bit mask test for the remaining characters.
bool IsSymbol(char) PROC
lea eax, DWORD PTR [rcx-123]
cmp al, 3
jbe SHORT $LN3@IsSymbol
sub cl, 33
cmp cl, 63
ja SHORT $LN5@IsSymbol
mov rax, -288230371890266113
bt rax, rcx
jb SHORT $LN3@IsSymbol
$LN5@IsSymbol:
xor al, al
ret 0
$LN3@IsSymbol:
mov al, 1
ret 0
bool IsSymbol(char) ENDP
Clang also generates simple code, although it uses a 93 element jump table instead of testing against a bit mask. While this may be compact code this table takes up 372 bytes of space, which will take up instruction cache space and could affect performance.
IsSymbol(char):
add edi, -33
cmp edi, 93
ja .LBB1_2
mov al, 1
lea rcx, [rip + .LJTI1_0]
movsxd rdx, dword ptr [rcx + 4*rdi]
add rdx, rcx
jmp rdx
.LBB1_3:
ret
.LBB1_2:
xor eax, eax
ret
.LJTI1_0:
.long .LBB1_3-.LJTI1_0 # When false
.long .LBB1_2-.LJTI1_0 # When true
# ...
Much like in the previous unordered IsWhitespace function it turns out that GCC doesn’t like the IsSymbol C++ code as written and produces quite a long and branchy sequence of assembly.
The actual assembly code is a bit of a mess, doing a lot of individual tests, some range tests, and a bit mask test. Though the bit mask only has 3 bits set, meaning that it’s only testing for 3 characters even though it could test nearly the entire range of symbols using it (as the MSVC assembly code does).
Just for fun I reconstructed the C/C++ code from the assembly and it looks something like this: After doing this the best that I can tell is that GCC is grouping contiguous sequences together where it can find them but otherwise attempts to perform the tests in order. So if the characters tested were more randomly organised then the code would end up looking quite different (in GCC).Expand GCC assembly code
IsSymbol(char):
cmp dil, 33
je .L12
lea eax, [rdi-64]
cmp al, 62
ja .L18
movabs rdx, 4611686022722355201
bt rdx, rax
setc al
test al, al
je .L19
.L5:
ret
.L19:
cmp dil, 95
jg .L10
mov eax, 1
cmp dil, 90
jg .L5
.L8:
and edi, -17
cmp dil, 47
sete al
ret
.L18:
cmp dil, 95
jg .L8
cmp dil, 46
jg .L11
mov eax, 1
cmp dil, 33
jg .L5
jmp .L8
.L10:
lea edx, [rdi-123]
mov eax, 1
cmp dl, 2
ja .L8
ret
.L11:
lea eax, [rdi-58]
cmp al, 4
ja .L8
.L12:
mov eax, 1
ret
bool IsSymbol(char c)
{
if (c == 33) // '!'
{
return true;
}
if (c - 64 <= 62)
{
if (((1 << (c - 64)) & 4611686022722355201) != 0) // '@', '`', '~'
{
return true;
}
if (c > 95)
{
if (c - 123 <= 2) // '{', '|', '}'
{
return true;
}
}
else
{
if (c > 90) // '[', '\\', ']', '^', '_'
{
return true;
}
}
}
else
{
if (c <= 95)
{
if (c > 46)
{
if (c - 58 <= 4) // ':', ';', '<', '=', '>', '?'
{
return true;
}
}
else
{
if (c > 33) // '"', '#', '$', '%', '&', '\'', '(', ')', '*', '+', ',', '-', '.'
{
return true;
}
}
}
}
return (c & 0xEF) == 47; // '/'
}
Given that we’re testing so many different characters it may seem more natural to use a switch statement for this instead, so converting the previous implementation gives us this:
bool IsSymbol_switch(char c)
{
switch (c)
{
case '~': case '`': case '!': case '@':
case '#': case '$': case '%': case '^':
case '&': case '*': case '(': case ')':
case '_': case '-': case '+': case '=':
case '[': case ']': case '{': case '}':
case '|': case '\\': case ';': case ':':
case '\'': case '"': case ',': case '.':
case '<': case '>': case '/': case '?':
return true;
default:
return false;
}
}
This greatly simplifies the assembly code that GCC emits, turning it into a far simpler set of comparisons and branches. My intuition is that in a switch statement the compiler is more free to re-order case statements giving it more opportunity to optimise.
IsSymbol_switch(char):
cmp dil, 96
jg .L28
cmp dil, 90
jg .L31
cmp dil, 47
jg .L30
cmp dil, 32
setg al
ret
.L28:
sub edi, 123
cmp dil, 3
setbe al
ret
.L30:
sub edi, 58
cmp dil, 6
setbe al
ret
.L31:
mov eax, 1
ret
While MSVC now generates code that uses an indirect jump table. It stores the jump addresses in a smaller table, and then has the main 93 element table store an index to the smaller table. This means that there are two table lookups per character, but the tables are about a quarter the size of the Clang version.
bool IsSymbol_switch(char) PROC
movsx eax, cl
add eax, -33
cmp eax, 93
ja SHORT $LN36@IsSymbol_s
lea rdx, OFFSET FLAT:__ImageBase
cdqe
movzx eax, BYTE PTR $LN38@IsSymbol_s[rdx+rax]
mov ecx, DWORD PTR $LN39@IsSymbol_s[rdx+rax*4]
add rcx, rdx
jmp rcx
$LN4@IsSymbol_s:
mov al, 1
ret 0
$LN36@IsSymbol_s:
xor al, al
ret 0
npad 2
$LN39@IsSymbol_s:
DD $LN4@IsSymbol_s
DD $LN36@IsSymbol_s
$LN38@IsSymbol_s:
DB 0 # When true
DB 1 # When false
# ...
Interestingly enough the Clang version remains the same between the two C++ implementations, suggesting that somewhere along the way it converts both versions of the code to a common sequence (I suspect this is likely in the intermediate representation that LLVM uses).
Now just as reordering comparisons in the shorter IsWhitespace case helped generate better assembly code, I wanted to see what effect ordering the characters would have in the more complete IsSymbol case. So sorting all of the symbols and updating the C++ code gave me this:
bool IsSymbol_ordered(char c)
{
return c == '!' || c == '"' || c == '#' || c == '$'
|| c == '%' || c == '&' || c == '\'' || c == '('
|| c == ')' || c == '*' || c == '+' || c == ','
|| c == '-' || c == '.' || c == '/'
|| c == ';' || c == ':' || c == '<' || c == '='
|| c == '>' || c == '?' || c == '@'
|| c == '[' || c == '\\' || c == ']' || c == '^'
|| c == '_' || c == '`'
|| c == '{' || c == '|' || c == '}' || c == '~'
;
}
For MSVC it ended up with almost identical code as the original, with some of the tests swapped around but otherwise performing the same tests.
bool IsSymbol_ordered(char) PROC
lea eax, DWORD PTR [rcx-33]
cmp al, 63
ja SHORT $LN7@TestSymbol
mov rdx, -288230371890266113
bt rdx, rax
jb SHORT $LN5@TestSymbol
$LN7@TestSymbol:
sub cl, 123
cmp cl, 3
jbe SHORT $LN5@TestSymbol
xor al, al
ret 0
$LN5@TestSymbol:
mov al, 1
ret 0
bool IsSymbol_ordered(char) ENDP
GCC ended up with the same comparison functions but just organised differently, and it even generated an identical bit mask to test against.
IsSymbol_ordered(char):
lea eax, [rdi-33]
cmp al, 63
jbe .L26
.L21:
sub edi, 123
cmp dil, 3
setbe dl
mov eax, edx
ret
.L26:
movabs rcx, -288230371890266113
mov edx, 1
bt rcx, rax
jnc .L21
mov eax, edx
ret
With Clang being the odd one out generating two more range comparisons before performing a similar but different bit mask test.
IsSymbol_ordered(char):
lea ecx, [rdi - 33]
mov al, 1
cmp cl, 15
jb .LBB2_5
movsx ecx, dil
lea edx, [rcx - 91]
cmp edx, 35
ja .LBB2_2
movabs rsi, 64424509503
bt rsi, rdx
jb .LBB2_5
.LBB2_2:
add ecx, -58
cmp ecx, 7
jae .LBB2_3
.LBB2_5:
ret
.LBB2_3:
xor eax, eax
ret
So we see that there is a vast difference between the assembly code generated when the characters being compared are sorted versus when they are unsorted. The best reason I can think of that this is the case is because of the short circuit behaviour of C++’s logical operators (|| and &&). These effectively imply an ordering to the tests where in this case we don’t really need that.
Now the final thing to check is to see what happens when we reorder the case statements within the switch. The code should look something like:
bool IsSymbol_switch_ordered(char c)
{
switch (c)
{
case '!': case '"': case '#': case '$':
case '%': case '&': case '\'': case '(':
case ')': case '*': case '+': case ',':
case '-': case '.': case '/': case ':':
case ';': case '<': case '=': case '>':
case '?': case '@': case '[': case '\\':
case ']': case '^': case '_': case '`':
case '{': case '|': case '}': case '~':
return true;
default:
return false;
}
}
And the answer is absolutely nothing!
The generated assembly code for this function with the sorted case statements is exactly the same as the unordered switch function shown before for every compiler tested. This confirms my intuition from before that the compiler is free to reorder the case statements in this situation.
There’s a few things that we can learn from this investigation, even before the other articles are written. Though the performance evaluation will have to wait until the benchmarks are done.
All compilers can create small and efficient bits of code from these simple character classification functions. They can combine multiple tests written in C++ into a single range or bit test down at the assembly level.
So in general terms do not worry about the performance of such functions but instead write simple and clear code, as that also makes it easier for the compiler to optimise.
If you’re in doubt or curious then check with Compiler Explorer.
If the order of the comparisons is unimportant then sorting the values being compared will give better code generation than having them in a random order. This will more easily allow the compiler to implement the comparisons as range or bit mask tests rather than single character tests.
This also applies to testing ranges as keeping the begin and end ranges ordered helps both humans read the code and the compiler optimise the code.
If code size is not an issue, or you want to avoid sorting the comparisons each time, then using a switch statement might work better for you.
Of course if you know the distribution of the values in your data then place the most common tests first, because the compiler should retain that order and the code should run faster in practice. Of course you should test and measure to get a proper understanding.