Okay. This is probably a jumble. It’s also a question disguised as an explanation of my experience. It’s possible working through this all has answered my question, but…
I would really like to know if I am missing something more obvious about juggling npm dependencies and a build process when using git bisect to identify a breaking change. Help?
Some background: I have used
When a build process was officially introduced to WordPress, each commit to the development repository also generated a corresponding commit to the core repository. You could use either to track down the original changeset. I believe it was mostly possible to run
Okay. Now with Gutenberg.
I spent some Time last week figuring out how to track down a change in the Gutenberg repository that was incompatible with some code in my Shortnotes plugin. Quite a bit of that Time was spent battling npm, probably in a non-efficient way.
That said, I’ve turned a day lost to running
npm install way too many times into a lesson that I think is accurate, but that I would love to be more confident in:
- Checkout the project’s most recent commit and run
- Run the build process (for Gutenberg,
npm run devor
npm run build)
- Checkout an old commit
- Run the build process
- If the build fails, run
npm installto resolve dependencies using the old
npm run devagain.
- If the build fails, run
- Stash any dirty
package-lock.jsonbefore attempting to checkout another commit.
- Repeat in some form or another.
Okay. Whether or not that’s correct (please tell me if not!), I made things more difficult on myself this time around by letting several months pass before trying to fix this bug, so there are a lot of commits to sort through.
For this issue I had to go back to Gutenberg 11.3.0 before my code was no longer broken. I shortcut my normal process by performing my own initial bisect with
wp plugin update to determine that a change between Gutenberg 11.3.0 and 11.4.0 caused my issue. Now I only need to look for a change in some 122 commits.
Alas, when I checkout
v11.3.0 and run
npm run dev, I’m told that a dependency discrepancy exists in Typescript. That’s a good notice because it effectively tells me to run step 4.1 from my flowchart above.
But then! When I run
npm install, something fails in the
firstname.lastname@example.org prepare script that prevents it all from working. I figure I’ve been relying on my initial
npm install too much and should probably perform one from scratch. Nope, same error.
Here is where I spent a bunch of time assuming that I was doing something wrong.
npm install with a new cache directory. I tried
npm ci, I tried googling some stuff that wasn’t relevant. I probably downloaded a good terabyte of data trying to get
npm install to work.
Jeremy, work smarter, not harder.
Finally, I looked up
react-native-gesture-handler in the Gutenberg repository. Sure enough an issue was raised in August when
npm install failed in the CI configuration. An attempt to fix it was made then, but I think the true fix came in September when all of the WordPress mobile dependencies were switched to tarballs rather than git repositories.
After some poking and prodding, I was able to check out Gutenberg v11.3.0, edit the
package.json in Gutenberg’s
packages/react-native-editor to use the tarball for
package-lock.json and run
npm install successfully. 😅
Now I can run
npm run dev, verify the issue did not exist in v11.3.0, check out v11.4.0, run
npm run dev, and verify the issue did exist there. Progress!
Okay. So I’m slightly more confident when using
git bisect on the Gutenberg repository. I can (hopefully) assume I stumbled on strange problems with npm that won’t always exist. The monolith is mighty, but not untraversable.
The biggest bummer now is that the process of
git bisect good,
npm run dev, reload the web page, type a character, and save the post to see if an error is shown takes roughly 2.5 minutes, of which almost the entirety is the build process. An issue that could take only a few minutes to identify instead takes at least 15 even if you’re staring at the screen and ready to pounce when the build is done.
I guess this could all be better if I wrote my own test for Shortnotes that checks to see if a post publishes successfully through the browser. Then I could script this all to run
npm run build, run my tests by launching a headless browser, then mark the bisect as good or bad depending on the result, but yeesh… that would add another handful of minutes to the process and hobby projects should be fun! 🙂
I have some open items / takeaways / assumptions / maybe still questions:
- Having automated tests for my plugin would eliminate much of the manual work in many cases.
- Automated tests would not help when npm configs get all flaky. I would have to catch the error in an automated nightly test and then troubleshoot it while the npm config was still working. If I caught it on the next commit to my plugin, I’d probably go through the same pain.
- There has to be a good way when running
git bisectto watch for changes in
npm installcan be run with more confidence. I need to do more research here.
- I think it would be useful for WordPress to maintain a repository of built Gutenberg that is updated whenever a pull request is merged to the trunk branch. Testing and troubleshooting would be significantly easier in some cases.
Please: if you have any tips or opinions, share away here or in a post of your own! 🍻