{"id":257,"date":"2024-07-23T18:36:00","date_gmt":"2024-07-23T18:36:00","guid":{"rendered":"https:\/\/permutationcity.co.uk\/bp\/?p=257"},"modified":"2024-07-27T15:39:13","modified_gmt":"2024-07-27T15:39:13","slug":"iterative-development","status":"publish","type":"post","link":"https:\/\/permutationcity.co.uk\/bp\/2024\/07\/23\/iterative-development\/","title":{"rendered":"Iterative development"},"content":{"rendered":"\n

It can be useful to develop in a series of small steps. At each point trying to write the code that you need but not too much more. Building a huge library that you think you’ll need might be a waste of time<\/a> but building something small that lets you simplify you current code is worthwhile.<\/p>\n\n\n\n

I’ve often do this at work but that code isn’t available to me here.\nSo a direct demonstration isn’t possible.\nInstead I’ve have done\nsome prototyping<\/a>\nand I’ll use that to tell the story.\nI’m going to use the example of a bit manipulation library because I’ve done one before.\nLets imagine we’re working on some sort of packet inspector on a network.\nIn particular I’m going to reference the\nTCP segment structure<\/a>\nin the code.<\/p>\n\n\n\n

Direct manipulation<\/h2>\n\n\n\n

The first way to access bit flags is directly with basic bit manipulation and some masks.<\/p>\n\n\n\n

bool IsECE(const std::array<std::byte, 56>& segmentHeader) {\n    const auto flags = segmentHeader[13];\n    return ((flags >> 1) & 1u) && ((flags >> 6) & 1u);\n}<\/pre><\/div>\n\n\n\n
This works but for me this is just not enough.\nIf someone comes across this later it will mean nothing.\nMy minimum would be to have a comment or some helpfully named types and constants, perhaps both. <\/p>\n\n\n\n
\/\/ https:\/\/en.wikipedia.org\/wiki\/Transmission_Control_Protocol\n\nstruct TcpSegmentHeader {\n...\n    uint8_t DataOffsetAndReserved;\n    uint8_t Flags;\n...\n};\n\nconst uint8_t gc_DataOffsetBitIndex(4);\nconst uint8_t gc_DataOffsetBitSize(4);\n...\nconst uint8_t gc_ExplicitCongestionNotificationBitIndex(6);\n...\nconst uint8_t gc_SynchronizeSequenceNumbersBitIndex(1);\n...<\/pre><\/div>\n\n\n\n
I could have used the abbreviated form of the flag names but\ngc_CongenstionWindowReducedBitIndex<\/code> appeals to me more than gc_SwrBitIndex<\/code>.\nThat lets me re-write the function.<\/p>\n\n\n\n
bool IsExplicitCongestionNotification(const TcpSegmentHeader& segmentHeader) {\n    const auto flags = segmentHeader.Flags;\n    return ((flags >> gc_SynchronizeSequenceNumbersBitIndex) & 1u) &&\n           ((flags >> gc_ExplicitCongestionNotificationBitIndex) & 1u);\n}<\/pre><\/div>\n\n\n\n
Now someone looking at the code has a hope of understanding it. Next you might be asked to start updating the flags.<\/p>\n\n\n\n
void ClearExplicitCongestionNotificationFlag(TcpSegmentHeader& segmentHeader) {\n    const auto& flags = segmentHeader.Flags;\n    flags = flags & ~(1u << gc_ExplicitCongestionNotificationBitIndex);\n}<\/pre><\/div>\n\n\n\n
This isn’t hard but making a mistake becomes more likely. Did you mask the right thing? Was that the right operator to use? If you do it enough times one of them will be wrong. To me it is also less obvious what the code is doing. If I was skimming the code I could tell it’s doing bit manipulation but what sort? I mean, it says<\/em> it’s “clearing the explicit congestion notification flag” but is that what it’s actually doing?<\/p>\n\n\n\n
Bit manipulation library<\/h2>\n\n\n\n
So when you have a few of these it’s useful to make a functions to help. For the real thing I’d use template functions based on a Value<\/code> type but I’ll skip that here for simplicity.<\/p>\n\n\n\n
namespace Bits\n{\n    Value MaskUpTo(uint8_t index);\n    Value Mask(uint8_t index, uint8_t size = 1);\n    \n    bool IsSet(Value value, uint8_t index);\n    bool IsClear(Value value, uint8_t index);\n    \n    bool IsAllSet(Value value, uint8_t index, uint8_t size = 1);\n    bool IsAnySet(Value value, uint8_t index, uint8_t size = 1);\n    bool IsNoneSet(Value value, uint8_t index, uint8_t size = 1);\n    \n    Value Clear(Value value, uint8_t index, uint8_t size = 1);\n    Value Set(Value value, uint8_t index, uint8_t size = 1);\n    Value Flip(Value value, uint8_t index, uint8_t size = 1);\n    \n    Value GetAt(Value value, uint8_t index, uint8_t size);\n    Value SetAt(Value value, uint8_t index, uint8_t size, Value subValue);\n} \/\/ namespace Bits<\/pre><\/div>\n\n\n\n
And I’ll take advantage of my new library and add some more functions.<\/p>\n\n\n\n
uint8_t GetDataOffset(const TcpSegmentHeader& segmentHeader) {\n    return Bits::GetAt(segmentHeader.DataOffsetAndReserved,\n        gc_DataOffsetBitIndex, gc_DataOffsetBitSize);\n}\n\nvoid SetDataOffset(TcpSegmentHeader& segmentHeader, uint8_t size) {\n    return Bits::SetAt(segmentHeader.DataOffsetAndReserved,\n        gc_DataOffsetBitIndex, gc_DataOffsetBitSize, size);\n}\n\nbool IsExplicitCongestionNotification(const TcpSegmentHeader& segmentHeader) {\n    const auto flags = segmentHeader.Flags;\n    return Bits::IsSet(flags, gc_SynchronizeSequenceNumbersBitIndex) &&\n           Bits::IsSet(flags, gc_ExplicitCongestionNotificationBitIndex);\n}\n\nvoid ClearExplicitCongestionNotificationFlag(TcpSegmentHeader& segmentHeader) {\n    const auto& flags = segmentHeader.Flags;\n    flags = Bits::Clear(flags, gc_ExplicitCongestionNotificationBitIndex);\n}<\/pre><\/div>\n\n\n\n
This is much better. The function names tell you what’s happening. It’s easier to write and easier to read. The type constraints from the functions make it a safer as well. Unit tests can make sure at least this part of your logic is right.<\/p>\n\n\n\n
Bit manipulation classes<\/h2>\n\n\n\n
If you continue to expand your codebase you can make it even easier. The API is functional but the code it produces could be more concise.<\/p>\n\n\n\n
namespace Bits\n{\n    template <typename Value>\n    class Index\n    {\n    public:\n        Index(uint8_t index);\n\n        bool IsSet(Value value) const;\n        bool IsClear(Value value) const;\n\n        Value Set(Value value) const;\n        Value Clear(Value value) const;\n        Value Flip(Value value) const;\n    ...\n    };\n\n    template <typename Value>\n    class Range\n    {\n    public:\n        Range(uint8_t index, uint8_t size);\n\n        bool IsAllSet(Value value) const;\n        bool IsAnySet(Value value) const;\n        bool IsNoneSet(Value value) const;\n\n        Value Set(Value value) const;\n        Value Clear(Value value) const;\n        Value Flip(Value value) const;\n\n        Value GetAt(Value value) const;\n        Value SetAt(Value value, Value subValue) const;\n    ...\n} \/\/ namespace Bits<\/pre><\/div>\n\n\n\n
Which requires some new constants.<\/p>\n\n\n\n
const Bits::Range<uint32_t> gc_DataOffsetBitRange(4, 4);\n...\nconst Bits::Index<uint32_t> gc_ExplicitCongestionNotificationBitIndex(6);\n...\nconst Bits::Index<uint32_t> gc_SynchronizeSequenceNumbersBitIndex(1);\n...<\/pre><\/div>\n\n\n\n
And I can simplify the functions some more.<\/p>\n\n\n\n
uint8_t GetDataOffset(const TcpSegmentHeader& segmentHeader) {\n    return gc_DataOffsetBitRange.GetAt(segmentHeader.DataOffsetAndReserved);\n}\n\nvoid SetDataOffset(TcpSegmentHeader& segmentHeader, uint8_t size) {\n    return gc_DataOffsetBitRange.SetAt(segmentHeader.DataOffsetAndReserved, size);\n}\n\nbool IsExplicitCongestionNotification(const TcpSegmentHeader& segmentHeader) {\n    const auto flags = segmentHeader.Flags;\n    return gc_SynchronizeSequenceNumbersBitIndex.IsSet(flags) &&\n           gc_ExplicitCongestionNotificationBitIndex.IsSet(flags);\n}\n\nvoid ClearExplicitCongestionNotificationFlag(TcpSegmentHeader& segmentHeader) {\n    const auto& flags = segmentHeader.Flags;\n    flags = gc_ExplicitCongestionNotificationBitIndex.Clear(flags);\n}<\/pre><\/div>\n\n\n\n
Library performance<\/h2>\n\n\n\n
While there are benefits to this code: easy to write, easy to read, easy to test.\nYou might have concerns about performance.\nDirect bit manipulation is about fast as you can get.\nThere are assemble equivalents for most of this.\nIs this library going to use them?<\/p>\n\n\n\n
Performance is often less of an issue than people think but, if it is, you can check. Modern optimisers are pretty good but if the compiler isn’t using the best assembly then you can do something about it. Everything that uses these functions will gain the benefit. I’ve done some quick benchmarking<\/a> for this and my library looks neck and neck with direct manipulation.<\/p>\n\n\n\n
Skipping to the end<\/h2>\n\n\n\n
I’ve suggested building up this library in a series of small steps.\nHowever if you know<\/em> you’ll be using something a lot\nit might be better to go for a good interface with a bad implementation.\nI could have gone straight for the Bits::Index<\/code> and Bits::Range<\/code> classes but\nimplementing it with std::bitset<\/code>.\nIt would probably have involved a lot of conversion between types and\nlooping through arrays of bits.\nThat means it would have been slow but\nit would have still worked.\nI could have let me get on to the urgent packet inspection work.\nIf you can get a good interface the implementation can always be fixed later on.<\/p>\n\n\n\n
In the end<\/h2>\n\n\n\n
It matters less whether you take it in small steps or skip to the end.\nThat comes down to how much you can predict about where the code is going.\nIt’s about noticing patterns that occur in your codebase and then simplifying those patterns.\nI’ve posted about this before when thinking about\nDRY<\/a>.\nThere is a bit of extra work to notice and formalise the pattern but\nfuture work is then easier in many ways.<\/p>\n","protected":false},"excerpt":{"rendered":"
It can be useful to develop in a series of small steps. At each point trying to write the code that you need but not too much more. Building a huge library that you think you’ll need might be a waste of time but building something small that lets you simplify you current code is […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_import_markdown_pro_load_document_selector":0,"_import_markdown_pro_submit_text_textarea":"","footnotes":""},"categories":[1],"tags":[7,11],"class_list":["post-257","post","type-post","status-publish","format-standard","hentry","category-general","tag-performance","tag-planning"],"_links":{"self":[{"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/posts\/257","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/comments?post=257"}],"version-history":[{"count":2,"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/posts\/257\/revisions"}],"predecessor-version":[{"id":262,"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/posts\/257\/revisions\/262"}],"wp:attachment":[{"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/media?parent=257"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/categories?post=257"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/permutationcity.co.uk\/bp\/wp-json\/wp\/v2\/tags?post=257"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}