How My Terrible Memory Makes Me a Better Developer
Last week, I wrote a script to parse a Word document into JSON. I wrote this in Python and, to avoid having to deal with different Python environments on each user’s machine and having to install dependencies, I used Docker. This lets me have the same environment everywhere the script runs.
I actually started on this script the prior week and ended up putting it down over the weekend and for the start of this week. Thanks to my laughably bad short-term memory (I can still remember my ICQ number from 23 years ago — 1590438 — but I can’t remember anything that happened last month. 😆), I couldn’t remember how to run the script once I got back into it.
Thankfully, I had my terminal history to refresh my memory. If you’re not familiar with the terminal, you can hit the up arrow to cycle through your previous commands. I’m running Oh My ZSH and its history substring search plugin which allows me to type part of the command and hit up to search my history for commands containing whatever I’ve typed. I did recall the command to run the script is a “docker” command, which made it easy to get back to the right command.
I’ve outsourced my memory to my terminal’s command history. This is great since I couldn’t keep that memory in-house, but it’s bad for two reasons:
- I’m the only one who has access to the command. My command history is only on my machine. That means, if someone else needs to run this script, they’ll have to first figure out how to run it.
- The command will eventually drop off my history as I run new commands and they are added to the history. This means eventually no one will have access to the command without working it out for themselves.
The history is great, but, for these two reasons, it’s not a long-term solution for my memory problem.
How I Get Around It
As soon as I realize my memory has failed me, I pause whatever I’m working on. My new job is to make sure I don’t have to remember again later. By extension, this means the next person that works on my code won’t have to figure it out for themselves.
For this project, my first step was writing a readme for the repository. Every Git host I’ve used looks for a “readme.md” file in the root of your project and renders that underneath the directory tree when someone visits your project. The “.md” extension is used for Markdown. (If you’re not familiar with Markdown, check it out. It’s really easy to learn.)
I start with a list of software that must be installed to use the project (for this project, it was Docker) under a “Pre-Requisites” heading. Next up, I have a “Setup” section that tells you how to install any project dependencies. In this project, Docker takes care of all that, so I skipped “Setup.”
Now, I need to tell people how to use whatever I’m building. That goes under the “Usage” heading. This is another one that doesn’t apply for every project. If I’m building a simple web site, I don’t need to worry about it. This section is useful for development tools I’m building to be used by other developers (like this Word to JSON parser).
Finally, I’ll add a “Development” heading where I tell people how to work on this application. In the case of this Python script, I’ve described how to get a shell on the Docker container so the developer can run the script through pdb (the Python Debugger)
Here’s an example of what that readme.md file looks like for the Word to JSON script:
# Word to JSON Parser
## Requirements
* Docker
## Usage
Run `./generate.sh ` in this project's directory root
**Note:** The Word document must be under the project directory. Other paths on the host system are not available to the script's Docker container.
## Debugging
Most debugging of the script should be done via shell on the container. This will give you tools which allow more visibility into what the script is doing.
To start a shell on the script's container, run `./shell.sh` from the project directory. The project directory is mapped to `/usr/src/app` on the container. This is the container's default working directory, so when the shell is started, you'll be in this directory.
Copy any files you want to use into the project directory, and you can access them on the container. You can then run the script with this command:
```
python generate_json_from_word.py <path-to-storyboard>
```
Note that `<path-to-storyboard>` is a path on the container.
For more intensive debugging, it's useful to run the script with the pdb debugger using this command.
```
python -m pdb generate_content_from_storyboard.py <path-to-storyboard>
```
Bonus: Script Your Steps
Those of you who do a lot of work on the command line can level up with this trick. For a lot of my projects, I notice that the instructions under many of my headings come down to lists of commands people need to run to install prerequisites, run development servers, or debug. These lists of commands could be reduced down to a single command each through the power of shell scripting.
Shell scripting is a simple thing with a scary sounding name. At its simplest, a shell script is a list of commands you would run in your terminal. Drop them into a text file with a newline between each command and save it with a .sh extension (or any extension really) and you have a shell script.
Here’s an example shell script from my project that gives the user a shell on the Docker container:
docker build -t word-to-json .
docker run -it --mount type=bind,src=`pwd`,dst=/usr/src/app --entrypoint sh word-to-json
The first command builds the container. (Think of a container as kinda like a virtual machine.) The second one starts a shell on the container.
I have this text saved as a file in the root of my project as “shell.sh” and that’s a shell script. It’s just two commands I could run manually (by typing them out in the terminal) to get the same result. Instead, I’ll make this file executable by typing chmod +x shell.sh and execute it with ./shell.sh to achieve the same thing with a single short and sweet command. (StackOverflow user neuro has a great explanation of why you need to lead your shell script commands with ./ in case you’re curious.)
Even though this shell script doesn’t do much, it does keep you from having to copy and paste that nasty looking Docker build command every time you want to get shell on the container. The more complicated the commands people need to run and the more of them, the more time and hassle you’ll save with shell scripts.
How Does the Bad Memory Help?
Anyone can go through these simple steps to make sure your projects are documented and even script any commands people need to run. How does my bad memory actually help me?
Ironically, my bad memory serves to remind me that the things I knew at one point because of repetition were not intuitive and were waiting to be forgotten. I’m reminded by the fact that I now have to hunt down the processes because I’ve already forgotten them.
My memory shines a spotlight on exactly what I need to document. Even if your memory is great, you can still follow my guidelines for documenting projects, but, if you have a terrible memory like me, you’re much more likely to catch everything that will in time be forgotten even by those with average or great memories.
You: But Your Documentation Sucks!
I know this probably isn’t the pinnacle of documentation, but it works for me right now. My method is not meant to be the last stop on your software documentation journey. Many projects have elaborate documentation sites built with the latest static site generators. Some companies have stringent requirements for documentation that my piddly readme.md would never meet.
If you’re already doing better documentation or just want to skip over this to the good stuff, great! Go do that. If you’re not doing any documentation, this simple formula is a great start.