When you broke code, finding back where it got broken is easier if you have small change increment (i.e. bisection and binary tree search)
Posted by jpluimers on 2024/09/26
A while ago [Wayback/Archive] b0rk (Julia Evans [Wayback/Archive) wrote an interesting Tweet on finding back where you broke code of which the OCR text reads like this:
strategy: change working code into broken code
If I have a working version of the program, I like to:
- go back to the working code
- slowly start changing it to be more like my broken code
- test if it’s still working after every single tiny change
· ⬊⸳⬈˙⬊⸳⬈˙⬊⸳ OH THAT’S WHAT BROKE IT!!!I like this because it puts me back on solid ground: with every change make that DOESN’T cause the bug to come back, I know that wasn’t the problem.
by JULIA EVANS @bork wizardzines.com
This is similar (her arrows were of varying length) to using a binary search algorithm hunting for where the code was broken using bisection: repeatedly halving your search space to quickly zoom into the problem.
Another important aspect is that small commits while fiddling to solve an issue can help you determine what small commit was actually solving the issue.
This was phrased well by [Wayback/Archive] Chuck Larrieu Casias on Twitter: “@b0rk What I like about this is learning things that weren’t the problem. I feel like a lot of times I find a fix, but I’m not exactly sure what was necessary and what was incidental. I may remove too much or keep extra things in, and then I’m too scared to change bc it works now”.
Questions you should ask for each of these small commits are like:
- Does this commit help solving the problem?
- If not, what is the value of this commit in the code-base?
That helps lead to find out which commits are really needed to fix the problem, for which you can use the bisection method to determine the relevant commits.
So now we are…
Back to bisection
Bisection in software engineering is very similar to the bisection method in math.
Quite a few people responded with that especially because version control systems like git can do a bisect to find the exact commit where code got broken each iteration then just needing a (hopefully short) test that indicates “yes it still works” and “no it is broken now” result.
This is a big argument to commit buildable code often as that allows for finding back smaller code changes that has a particular “broken” feature in it.
A few years back I showed a demonstration of that in Binary search for finding problematic versions: install a specific version in homebrew and git bisect.
Some of those tweets are below. The cool thing is the link back to Julia Evans writing about strace.
Another demonstration was linked from [Wayback/Archive] gaeel on Twitter: “@b0rk As others have said, git bisect is a great tool for this @qyliss’ talk at Brighton Ruby 2019 illustrates a great usage of that tool to track down an arcane bug in an unfamiliar codebase: “: which links to a short introduction paragraph with a half our conference video (both by [Wayback/Archive] Alyssa (@qyliss)) that everyone should watch even when outside the Ruby field.
[Wayback/Archive] You may have encountered a bug in the Ruby interpreter
People sometimes say “it’s never a compiler error”. They don’t mean it literally — what they mean is that it’s very tempting to blame the compiler or interpreter for a bug that is actually in our own code. But sometimes, when the stars align, there it is. The mythical interpreter bug.I’m going to show you how I narrowed down from “the website is crashing on my computer” to a real, live bug in the Ruby interpreter.
We’ll look at how we can use techniques we already know, like unit testing and git, in the unfamiliar context of Ruby’s C internals. And, when we’ve finally figured out what’s causing our bug, we’ll go through the bug reporting process and learn how to share our findings and help make Ruby better for everyone.
Relevant tweets
- [Wayback/Archive] Julia Evans on Twitter: “debugging strategy: change working code into broken code”
- [Wayback/Archive] # cat /dev/random on Twitter: “@b0rk … git bisect?”
- [Wayback/Archive] Julia Evans on Twitter: “@00mnein git bisect seems so cool but I’ve never actually used it for some reason”
- [Wayback/Archive] Jeroen Wiert Pluimers on Twitter: “@b0rk @00mnein Please try it. It’s the force of binary search at work. I was amazed how much easier git bisect is than I thought it to be. See for instance my blog post
wiert.wordpress.com/?p=52000(yup, my blog queue back then was ~2 years, and I’m back on track getting it that deep again)” - [Wayback/Archive] Justin Azoff on Twitter: “@b0rk @00mnein I’ve rarely used it for my projects, but it’s useful for finding regressions in large codebases. Last year I used git bisect to understand why a glibc upgrade broke things:
corelight.com/blog/tracking-down-a-glibc-regressionI link to yourstracepage too, I used that first to work out what was broken :-)”
- [Wayback/Archive] Jeroen Wiert Pluimers on Twitter: “@b0rk @00mnein Please try it. It’s the force of binary search at work. I was amazed how much easier git bisect is than I thought it to be. See for instance my blog post
And because alt-text is important:
- [Wayback/Archive] cmcurry on Twitter: “@b0rk Would it be possible to get alt text for this?”
- [Wayback/Archive] Julia Evans on Twitter: “@the_dr_blue (1/2) strategy: change working code into broken code If I have a working version of the program, I like to: * go back to the working code * slowly start changing it to be more like my broken code * test if it’s still working after every single tiny change”
- [Wayback/Archive] Julia Evans on Twitter: “@the_dr_blue (2/2) I like this because it puts me back on solid ground: with every change I make that DOESN’T cause the bug to come back, I know that wasn’t the problem.”
–jeroen
The Wiert Corner - irregular stream of stuff 
Leave a comment