When I'm working with a client, I respect their time and energy by keeping things simple and easy for them. I make as many decisions as I can, and try not to bog them down with every complex decision I have to make.

I've learned to trust my instincts, and to escalate important choices only when the trade-offs are truly business decisions.

This is a story of when this instinct led me in the wrong direction.

A static link to a static zip file

I often work on projects with fixed-price and flexible-scope. What this means is my clients know exactly what a project will cost, but I'm also okay with the odd surprise.

That day, I was working on a web app that was powered by static data files. I was working on the simplest task of the project. The client wanted to add a link so users can download all the data files in a zip file. Sounds straightforward, right?

Commit #1: "NEW: add data download"

<a href="data.zip">Download Data</a>

I could have just zipped the data files up on my laptop and called it a day. However, I didn't want to be in a situation where I forgot to update the zip file if the data changed. I decided to automate the process by creating the zip file during the automated deployment pipeline.

Commit #2: "NEW: create zip file in deployment pipeline"

"scripts": {
"build": "npm run build:download; ...",
"build:download": "cd public/static/data; rm -f data.zip; zip *.csv data.zip"
},

I pushed that change and the pipeline failed immediately.

sh: 1: zip: not found

Seriously? I looked at the pipeline configuration in bitbucket-pipelines.yml. It was running off a seven-year-old container that didn't have zip installed.

I could have made a new, custom container that had everything we needed. I decided that would give the project another thing to manage over the long run, which I thought it'd be best to avoid. Instead, I thought I'd simplify things by using the official node.js container.

I added a line to the pipeline to install rsync (to upload files to the staging server) and zip. It would only add a few seconds to the pipeline, but then we wouldn't need to maintain a custom container.

Commit #3: "FIX: use latest node.js container, install rsync and zip"

image: node:latest
pipelines:
  branches:
    main:
      - step:
          script:
            - apt-get update
            - apt-get install -y rsync zip

The pipeline failed again.

> npm run build:download; webpack --mode=production --env.router hash --env.appPublicPath public --config ./production.config.js

Error in bail mode: Error: callback(): The callback was already called.
    at context.callback (/opt/atlassian/pipelines/agent/build/node_modules/loader-runner/lib/LoaderRunner.js:106:10)
    at processTicksAndRejections (node:internal/process/task_queues:103:5) Error: callback(): The callback was already called.
    at context.callback (/opt/atlassian/pipelines/agent/build/node_modules/loader-runner/lib/LoaderRunner.js:106:10)
    at processTicksAndRejections (node:internal/process/task_queues:103:5)

What the heck?

The web app was being built with Webpack v4, a very old build tool. Turns out Webpack v4 breaks on newer versions of node.js.

I decided to try using an older node.js container. After some experimentation, I found that Webpack v4 needed node.js v16 or older. Of course, once I had the right version of node, the build in the container still failed!

E: The repository 'http://deb.debian.org/debian buster Release' does not have a Release file.
E: The repository 'http://deb.debian.org/debian-security buster/updates Release' does not have a Release file.
E: The repository 'http://deb.debian.org/debian buster-updates Release' does not have a Release file.

This time it was because this old node docker container is built on an old and outdated version of Linux. The operating system is so old that I couldn't even install rsync and zip on it without some serious workarounds.

If you come to a fork in the road, take it

At this point I knew I was going in the wrong direction. Getting old versions to work together wasn't the simple & quick solution I had hoped it was. I wasn't going to spend any more time getting five-year-old software to play nice together. The best solution was to modernize this legacy project by replacing Webpack with Vite.

My client had surely never heard of Webpack or Vite, and they certainly would never have asked me to make the switch. I knew it was way outside the scope of this project. I also knew it would take me way longer than I estimated.

But more than all of that, I knew it was the right solution for this moment, and for the project in the long term. If I didn't deal with this properly today, the system would remain fragile, and only get worse. More problems like these would surely arise in the future, and I didn't want that burden and risk to persist.

I decided to push forward.

Commit #4: "FIX: replace Webpack with Vite"

Lines updated +2778 -8272

This was the deepest part of the rabbit hole, but I was happy to finally be rid of Webpack. The project was much better off for it, no longer stuck on outdated versions and tools. I was bracing myself for a huge task, but it didn't take as long as I feared, and I was very happy with the result.

Finally, I was able to use the latest versions of everything in the deployment pipeline, and it worked!

Delivering the download button

After all this, I sent my client an email and let them know the download button was in place. I explained that the zip file was being built automatically in the deployment pipeline. Best of all, I had modernized the pipeline so it was running the latest versions of everything and would even be a bit faster.

The client thanked me and asked, will the zip file update automatically when they upload new data files?

Damn, of course not. The deployment pipeline is used during development only, when I push changes to the staging server. Once the web application was live, I had assumed the client would create zip files manually whenever they uploaded new data files. It was a bad assumption on my end, and of course I was happy to deliver a better solution that works well for my client.

Commit #5: "NEW: use jszip to generate the zip file in the browser"

async function download() {
  
  const JSZip = (await import('jszip')).default;
  const zip = new JSZip();

  for (const filename of CSV_FILES) {
    zip.file(filename, await fetchFile(filename));
  }

  const zipBlob = await zip.generateAsync({ type: 'blob' });
  const url = URL.createObjectURL(zipBlob);
  const link = document.createElement('a');
  link.href = url;
  link.download = 'data.zip';
  document.body.appendChild(link);
  link.click();
  document.body.removeChild(link);
  URL.revokeObjectURL(url);
}

I deleted all the zip stuff from the pipeline, but of course I kept the modernization. I switched to creating the zip file on the fly in the browser using the jszip library. The download button was working great.

Simplifying things for my clients

When given a problem, I'm always looking to deliver the simplest solution. I'm willing to do the hard work to keep things simple for my clients, and for the users of the software.

At the start, having a static zip file seemed like the simplest solution. In the end, the best solution was the one that worked best for my client, and I was happy to deliver that instead.

By keeping things simple for my clients, I took on a lot of complexity. I went further down this rabbit hole than I had to, but I'm glad this legacy web application ended up being modernized in the process.

This easy task ended up taking way longer than I had expected, but my client never needed to know that. My client was very happy with the result. From their perspective, their project had been modernized, but I still finished everything on budget and on time.