Iain Dillingham

Weeknotes

2026-04-10T00:00:00+01:00

Weeknotes for the week finishing Friday, 10th April 2026.

This week, I:

Fixed a bug in production. The SSL certificate didn’t automatically renew. It expired, leaving a Cloudflare “Invalid SSL certificate” message in place of OpenPrescribing v2 for about 40 minutes early this morning.
Started work on a new feature: the ability to query by form/route and ingredient in OpenPrescribing v2’s backend. (Peter added the ability to fetch/ingest dm+d data and to fetch/ingest a mapping from dm+d codes to BNF codes.) I didn’t finish work on it this week, because of the bug in production.
Finished reading Simon Willison’s guide to Agentic Engineering Patterns.

Another bug in production

I fixed a bug in production last week, too. I’m not concerned that they exist or that they caused long outages; not today, anyway. OpenPrescribing v2 is still beta, signalled by the subdomain and the banner at the top of the page.¹ It takes time to fit together two reverse proxys (Cloudflare, Nginx), SSL certificates (Let’s Encrypt), a web server (Gunicorn), and a web application (Django). Sometimes, a long time: it’s likely that today’s bug was caused by a change I made nearly two months ago in response to an issue with OpenPrescribing v1.

More than last week, this week I leaned heavily on ChatGPT. I asked it to suggest reasons for the error, based on the message and code. I asked it about Dokku and the Let’s Encrypt plugin, because it wasn’t clear to me whether the error originated at the system-level or the app-level. And when the fix we — yes, ChatGPT and me — agreed on didn’t work, I asked it about Cloudflare’s reverse proxy. At this point, however, I ignored its suggestions to disable Cloudflare’s reverse proxy and to use DNS-01 rather than HTTP-01, and instead added a Cloudflare security rule.

Time’s up!

Respectively, https://beta.openprescribing.net/ and “This is a beta version of a new OpenPrescribing”. ↩

Weeknotes

2026-04-03T00:00:00+01:00

Weeknotes for the week finishing Friday, 3rd April 2026.

Two weeks’ worth of weeknotes this week.

Last week I was in Oxford, celebrating the award of the Queen Elizabeth Prize for Higher and Further Education. This coincided with my five year anniversary at the Bennett Institute for Applied Data Science, which led me to reflect on my longest tech job. Team Prescribosaurus also met up to talk about our next cycle.

Last week and this week, I added a feature to OpenPrescribing v2: thumbs up/thumbs down buttons for quick feedback. Clicking either button prompts you for more information. It took me a couple of hours to add the feature with help from Codex, so this was the second time I’ve used Codex to write code. (The first time was in late February.) But it took me a couple of days to debug why it wasn’t working in production, between meetings and code reviews. Even now, I can’t pinpoint the root cause. I have a hypothesis, but the quick fix makes testing it prohibitively expensive.

It seems to me that this kind of bug — works in development, doesn’t work in production, insufficient context — is resilient to coding agents. Codex and ChatGPT helped, definitely. Looking back through my questions, I see that I very quickly learnt a lot about Cross Site Request Forgery (CSRF) and how Django protects against it. However, neither Codex nor ChatGPT were able to pinpoint the root cause. And neither was I.

I don’t think this kind of bug signals a win for humans or a loss for coding agents, or vice versa. I think it signals that coding agents can support humans by making debugging faster, more accurate, and more enjoyable (there are fewer dead ends). Human judgement is still important, though. Martin Fowler talked about this in a recent blog post. I’m going to read the blog post he linked to; “The Expensive Thing” by Ajey Gore.

Weeknotes

2026-03-20T00:00:00+00:00

Weeknotes for the week finishing Friday, 20th March 2026.

This week was emotionally draining. It ended with a moment of clarity that made me rethink the last three months.

For now, I’m going to link to three posts that Ben B.C. recommended:

“Creating momentum when an employee is stuck” by Lara Hogan;
“First agree on the tradeoffs” by Jade Rubick;
“How I estimate work as a staff software engineer” by Sean Goedecke.

And to two posts that I found when I was trying to understand what a feature factory is:

“12 Signs You’re Working in a Feature Factory” by John Cutler;
“12 Signs You’re Working in a Feature Factory — 3 Years Later” by John Cutler.

I also read “Resilient Management, An Excerpt” by Lara Hogan and was saddened to learn that A Book Apart closed in 2024.

I did more than read posts this week, but as I say, it was emotionally draining.

Weeknotes

2026-03-13T00:00:00+00:00

Weeknotes for the week finishing Friday, 13th March 2026.

This week, I:

continued to write KPIs;
added some extra meetings to the calendar: several short daily standups and a longer weekly team meeting;
gave a show-and-tell about our second three-week cycle on OpenPrescribing v2;
read a paper about how to design effective and intuitive data visualisations.

Fixed time, variable scope

As I mentioned last week, we’ve been applying techniques from Shape Up. Last week, I talked about pitches. This week, I’m going to talk about fixed time, variable scope.

Fixed time, variable scope means a team holds the time they spend on delivery constant and varies the scope of what they deliver. Contrast this with fixed scope, variable time. I have much (much) more experience of the latter than of the former. In every case, variable time meant taking more, not less, time. And taking more time meant missing deadlines, again and again.

Shape Up calls the time a team spends on delivery a cycle. It suggests adopting either six-week or two-week cycles, with a cool-down period between each cycle. We adopted three-week cycles on OpenPrescribing v2 and, for historical reasons, call the cool-down period deck-scrubbing.

It’s been hard to finish what we’ve been working on by the end of each cycle. Although we finished our first cycle more or less on time, we did some delivery work during deck-scrubbing and we paused some delivery work that we started but didn’t finish. This “overhang” and “unfinished business” were demotivating. We had much less of both with our second cycle, but we discarded work that wasn’t getting good feedback. This was also demotivating, but for different reasons. Nevertheless, at the end of each cycle we were able to give a show-and-tell about a coherent set of features and fixes. We were also able to describe what we discarded and explain why we discarded it, without feeling guilty for spending too much time on it.

Time’s up!

Weeknotes

2026-03-06T00:00:00+00:00

Weeknotes for the week finishing Friday, 6th March 2026.

This week, I:

was reminded (again) that people issues are harder than technical issues;
started writing some KPIs;
started writing another pitch;
read a blog post about LLMs by the creator of htmx. (Thank you for the link, Simon.)

Pitches

In early November last year, the tech leads started talking about Shape Up. You might have heard of it. Shape Up contains a collection of product development techniques that are, according to the book’s introduction, tried and tested by Basecamp, now 37signals.

Central to the book is the idea that a delivery team should only start a piece of work when it has been properly shaped. Shaping is less like a planning activity and more like a design activity. The result of shaping a piece of work is a pitch. A pitch should:

describe the problem;
state the appetite, or the amount of time a team should spend delivering the solution;
contain a low-fidelity representation of the solution;
state any rabbit holes and no-goes.

I’ve written four pitches for OpenPrescribing v2. None are original; all draw on existing material. However, I’ve found that by using more-or-less the same structure, it’s getting easier to write, and to discuss, each pitch.

My main difficulty, so far, has been to identify both a problem and a solution that will occupy three senior software engineers for 270 hours. Estimation was hard before LLMs arrived; it’s even harder now, because writing code is quick and easy (for a senior software engineer, at least). Things that seem easy end up being hard; things that seem hard end up being easy.

Time’s up!

Weeknotes

2026-02-27T00:00:00+00:00

Weeknotes for the week finishing Friday, 27th February 2026.

This week felt like a tipping point: I started using an LLM to write code.

Before this week, I’d used ChatGPT’s web app to ask “How do I?” questions.

How do I count the number of NaN values in a numpy array?
How do I parse a date string into a Date object in JavaScript?
How do I run a command for every commit in a range?

These are questions that at some point or other I’ve answered myself. Sometimes I’ve read the docs. Sometimes I’ve Googled some keywords and sifted through the results. Often I’ve done both. Alas, I’ve forgotten the answers and whilst I’d like to refresh my numpy, JavaScript, and Git knowledge, I need the answers now. In some cases, that’s because I want to do the right thing before moving on: a mistake when counting the number of NaN values would mean more time spent debugging; incorrectly parsing a date string would mean a bug. In others, that’s because I’m scratching an itch: I could checkout each commit and run the command, but is there a better way? I’m not going to spend my evenings and weekends finding out. I’m 44 and have a young family. I don’t have evenings and weekends.

That was before this week. This week, I used Codex to write code. You’re looking at it now. I used Codex to write the HTML and CSS for my website. I could have written it myself, but I don’t write much HTML and CSS day-to-day and I don’t have a clear picture of what good front-end code looks like.

At work, I also used Codex to write front-end code, but with Bootstrap. The Bootstrap docs are excellent: easy to navigate, loads of examples. But I wanted a modal and as with the “How do I?” questions, I wanted to do the right thing before moving on. Incorrectly nesting a .modal within a .modal-dialog would mean a bug (they’re the other way around). I don’t have evenings and weekends. I need the answers now.

I reviewed the code for my website and the modal just as I would review the code in a PR. My sense is that Codex has a much clearer picture of what good front-end code looks like than I do. I learnt a lot, but I feel uneasy. Some thoughts:

What are the environmental impacts of using LLMs?
And the social impacts? There are haves and have-nots.
And the legal impacts? LLMs can write good front-end code because they’ve been trained on open-source projects.
Do LLMs disproportionately benefit me, a senior software engineer? It seems so. (Thank you for the link, Katie.)
What are the security implications of using LLMs? Codex can read the contents of my .env file.
What are the blub studies implications of using LLMs? Detailed mental models are valuable; fiddly technical details aren’t?

Time’s up!

Weeknotes

2026-02-20T00:00:00+00:00

Weeknotes for the week finishing Friday, 20th February 2026.

We started our second three-week cycle on OpenPrescribing v2 this week. We’re focusing on “more effective visualisations”. However, I spent much of this week fixing, or trying to fix, OpenPrescribing v1. Its VM frequently runs out of CPU and memory, and heavy traffic probably isn’t the cause.

This week, I also read a post about host leadership by Martin Fowler. It reassured me that my misgivings about servant leadership weren’t misplaced.

Weeknotes

2026-02-13T00:00:00+00:00

Weeknotes for the week finishing Friday, 13th February 2026.

As I mentioned last week, we’re working on OpenPrescribing v2. We finished a cycle last Friday and will start a cycle next Monday. This week was a deck-scrubbing week; time between cycles to tidy up.

We’ve certainly mixed and matched our terminology. Cycle comes from Shape Up; deck-scrubbing from a time when we thought a nautical metaphor was a good idea and referred to a cycle as a voyage. Terminology aside, we were asked to spend three weeks working on a feature, then show the feature to the Bennett Institute. I’m really pleased that we did just that, more or less.

In our retro (more mixing and matching our terminology), we talked about how we’d changed our behaviour to finish the cycle on time. Not quite: we did a small amount of work on Monday, so we also talked about how we wouldn’t have changed our behaviour, had we known. During the retro, we talked about how we’re going to taper our next cycle to make the three-week cycle to one-week deck-scrubbing transition easier.

I feel much better prepared for our next cycle. It’s good to start it knowing that we finished our last cycle more or less on time.

This week, I:

read the Management 101 and Tech Lead chapters in The Manager’s Path, by Camille Fournier;
learned that sequencing and prioritising are different;
started writing another pitch.

Time’s up!

Weeknotes

2026-02-06T00:00:00+00:00

Weeknotes for the week finishing Friday, 6th February 2026.

This week, I:

thought about when a user is a user;
started writing a pitch;
thought about what happens when “there’s not enough space on the dance floor”;
learned about servant leadership;
was reminded (again) that people issues are harder than technical issues.

When is a user a user?

We’re working on OpenPrescribing v2. Dave and Peter built a walking skeleton at the end of last year, which Peter, Tom, and I took over at the start of this year. We’ve shown it to the OpenPrescribing network, to Rich, and to the Clinical Informaticians (a word my spelling dictionary refuses to recognise). Next week, we’re showing it to anyone in the Bennett Institute who can spare an hour on Tuesday afternoon.

During one of these show-and-tell sessions Chris, one of the aforementioned Clinical Informaticians, recreated a measure from v1 in v2.¹ This was really encouraging: when someone does something in the new thing that they could do in the old thing, then that’s evidence that the new thing is, in some sense, working. This made me think about whether Chris was a user of v2. He’d learned how to recreate a measure in the new thing; he’d used the new thing to complete a task! And this made me think: how much effort should we go to, to ensure that Chris doesn’t have to relearn how to recreate a measure in the new thing when we inevitably make changes to the new thing?

As it happened, Peter was midway through making changes to the new thing. He was replacing a text-based interface with a point-and-click interface that had a couple of modal windows and modifier keys. I suggested placing the changes behind a feature flag. I felt that asking Chris to relearn the interface, at least until we’d thought more about the modals and modifiers, would cost a small amount of goodwill. Peter felt that doing so was unnecessary. Asking Chris to relearn the interface wouldn’t cost that much goodwill. And after all, OpenPrescribing v2 isn’t a mature product.

In the end, we compromised. We asked Chris to recreate a measure using the point-and-click interface. Chris did so: he used the even newer thing to complete a task that he completed with the new thing and with the old thing! We shared the point-and-click interface using zrok, so we didn’t need to place the changes behind a feature flag. (Maybe zrok is a feature flag?) We may have earned a small amount of goodwill, too.

Time’s up!

Chris didn’t completely recreate the measure, because it required more information than was available. However, he came as close as he could. ↩

Weeknotes

2026-01-30T00:00:00+00:00

Weeknotes for the week finishing Friday, 30th January 2026.

This week, I:

realised that I’d started, and not finished, half a dozen books on tech leadership/management;
read a lovely blog post from Will Larson about including engineers in leadership meetings;
realised that the Eisenhower Matrix works well for people, but not so well for teams;
read a lovely blog post from Julia Evans about learning an Old Boring Technology;
felt as though I couldn’t keep up with the management and engineering sides of being a tech lead;
felt as though I didn’t know what a tech lead was, anyway.

Tech leadership/management books

Although I’ve been a tech lead for over a year, most days feel like my first day. I don’t feel I’m bad at the job; but I don’t feel I’m good at the job either. My tried-and-tested approach to getting better at something is to read a book about it. So, over the last year I’ve started half a dozen books on tech leadership/management. In alphabetical order by title, they are:

Debugging Teams, by Ben Collins-Sussman and Brian Fitzpatrick
Engineering Management for the Rest of Us, by Sarah Drasner
Leading Effective Engineering Teams, by Addy Osmani
Scaling People, by Claire Hughes Johnson
Smart Management, by Jochen Reb, Shenghua Luan, and Gerd Gigerenzer
Team Topologies by Matthew Skelton and Manuel Pais
The Manager’s Path, by Camille Fournier

Alas, I’ve not finished them. I don’t wish to criticise these books: they’re well written,¹ well produced, and draw on the authors’ experience. However, after several hours in each other’s company, the book and I drift apart.

Why?

I don’t want a book-length treatment of tech leadership/management. I want something like untools (“Tools for better thinking”).
I want something that’s well written and thoughtful. I have Thinking of Answers, by A.C. Grayling in mind. Good blog posts can be both, but tend toward first-cuts.
I want something that isn’t over-fitted to the authors’ experience. Whilst there are valuable lessons from Google and Stripe, I work for neither Google nor Stripe. My current team has three software developers; my last team, two. One of those software developers is/was me.

Alas, I don’t have time to write more. But something is better than nothing.

Okay, mostly well written. ↩

Weeknotes

2026-01-23T00:00:00+00:00

Weeknotes for the week finishing Friday, 23rd January 2026.

My weeknotes for this week are shorter than usual. I try to start writing them on Tuesday or Wednesday, but it’s Friday afternoon and I’m just starting.

This week, I:

was an incident coordinator;
learned even more about BNF codes;
helped establish some norms about code reviews;
wondered where my week had gone.

Incident coordinator

When a software developer notices that there’s something wrong with one of our systems, they declare an incident in a dedicated Slack channel. Other software developers then help to resolve the incident. To keep them focused, the incident coordinator communicates with everyone who isn’t helping to resolve the incident: mostly users, but also other software developers.

This week, Katie noticed that there was something wrong with OpenSafely Jobs. She declared an incident and, after a short while, resolved it. I was the incident coordinator. So far, so unremarkable. But that’s the point: It takes time and effort to make things unremarkable.

Weeknotes

2026-01-16T00:00:00+00:00

Weeknotes for the week finishing Friday, 16th January 2026.

This week, I:

went to Oxford for the OpenPrescribing kick-off meeting;
wrote two ADRs (Architectural Decision Records);
learned more about DuckDB;
learned more about BNF codes;
wrote issues;
caught a cold.

ADRs

I want to write a little about ADRs, because I spent a non-trivial amount of time this week thinking about what they are and why we should write them. You can find our definition and rationale in the codebase. I’m pleased that ADR-000 is self-exemplifying; in other words, the team’s decision to record architectural decisions is captured by an ADR.

If it’s not clear, then I’m an advocate for writing ADRs. Indeed, I’m such an advocate that I made the team an offer: When we make a decision, I will write the ADR. The team accepted, although I’m regretting that my offer didn’t have an expiry date.

What I’d like to focus on here is the questions software developers often ask about ADRs and the reasons they often give for not writing them. First, some questions.

What distinguishes architectural decisions from other decisions about a codebase?
How detailed should an ADR be?
How long should it take to write an ADR?
When should I write an ADR?
Is it okay to write an ADR retrospectively?
Is it okay to change an ADR, once it’s written?
Do the benefits really outweigh the costs?

And second, some reasons:

Information about code should live as close to the code as possible. ADRs are not close to the code.
Large organisations are advocates for writing ADRs, but that doesn’t mean that small organisations should write them, too. Small organisations and large organisations are different.
Like Amazon, one of our Leadership Principles is Bias for Action. ADRs hinder this leadership principle.

I could reword the questions as reasons (“It takes too long to write ADRs”), but you know that already.

Alas, I don’t have time to write more. But something is better than nothing.

I’d like to thank Lucy, Peter, and Suzanne for their questions and comments about ADRs last week and this week.

Weeknotes

2026-01-09T00:00:00+00:00

Weeknotes for the week finishing Friday, 9th January 2026.

This week, I was occupied by setting up the new team. We’re working on OpenPrescribing, but as there’s already a group of people who are associated with OpenPrescribing,¹ and as that group of people refer to themselves as a team, calling the new team “the OpenPrescribing team” could be confusing.

A couple of candidates were “the OpenPrescribing delivery team” and “the OpenPrescribing development team”. Alas, delivery means different things to different people: people outside the new team will almost certainly be delivering OpenPrescribing in some way and it would be unfair to suggest that our work delivered whereas their work didn’t. Neither candidate is consistent with the names of the other other tech group teams: the Researcher Experience team, or REX; the Reproducible Analytical Pipeline team, or RAP; and the Research Software Innovation team, or RSI. Consistent is hardly well-defined: “about the thing, not the people” and “sounds like the name of a dinosaur, mostly” are probably what we mean by consistent.

I asked my colleagues for their views and “sounds like the name of a dinosaur, mostly” was as close to the consensus view as I could find. There aren’t any dinosaurs with “prescribing” or “rx”, meaning prescription, in their names, so we invented the Prescribosaurus. After some prompting by Thomas, an LLM generated the new team’s mascot.

Welcome, Team Prescribosaurus!

What else happened this week?

I’ve been reading the new codebase.
I had a lengthy discussion about supporting claims with evidence.
I had a lengthy discussion about Architectural Decision Records.
I learned about what front-end development looks like in 2026.
Peter and I discussed what Team Prescribosaurus might bet on next week.

Jess suggested calling that group of people “the OpenPrescribing network”. I like the name, but it would take a lot of effort to socialise. ↩

Weeknotes

2026-01-02T00:00:00+00:00

Weeknotes for the week finishing Friday, 2nd January 2026.

After another hiatus, weeknotes are back. A lot has happened in the last six months. Nevertheless, these are weeknotes. What happened this week?

I’ve been reading about prescribing data, BNF codes, and dm+d codes in preparation for starting work on OpenPrescribing next week. We have a new team, a new codebase, a new GitHub project, and a new Slack channel.

I asked my colleagues what I should read to get up to speed and they suggested:

These took me to many, many pages on various NHS websites. The page I bookmarked, however, was English Prescribing Dataset (EPD) with SNOMED Code.

What have I learnt? Well, that an indication is a symptom that suggests that particular medical treatment is necessary. That prescribing data represent reimbursement claims from dispensing contractors; prescriptions issued and prescriptions dispensed are separate, but related. That BNF codes once formed a hierarchy; now they don’t, unless you’re the NHSBSA. That a thing called ePACT2 gives authorised users access to prescribing data. That ePACT2 relates to a thing called the One Drug Database. That the dm+d contains an alphabet spaghetti of terms: VTM, VMP, VMPP, AMP, AMPP. And that my colleagues were helping the NHS identify cost savings over eight years ago. Considerable cost savings: a theoretical maximum of £410 million over 12 months.

Time’s up!

Weeknotes

2025-06-20T00:00:00+01:00

Weeknotes for the week finishing Friday, 20th June 2025.

I’ve been reading about continuous integration (CI) and continuous delivery (CD), and trying to imagine software development without pull requests (PRs). Although Ben B.C. has talked about this before, my prompt was returning to a PR with unresolved comments. It took me a non-trivial amount of time to remember the context and I’m sure it was no different for Alice, who opened the PR.

The aspect of CI/CD I don’t yet understand is how to break a feature into a logical sequence of commits, without first completing the feature. I’m used to creating the sequence by selectively staging and committing, and to rearranging the sequence by rebasing before merging. I’m not used to merging a part-created sequence; that is, to merging what Martin Fowler calls latent code.

Time’s up!

Weeknotes

2025-06-13T00:00:00+01:00

Weeknotes for the week finishing Friday, 13th June 2025.

As last week, additional paternity leave!

This week I learnt about git rebase --committer-date-is-author-date. Normally after a rebase, the author date is that of the original commit but the commit date is that of the new commit. With this option, the author date and the commit date are that of the original commit.

I’ve been using this option to tidy up rebases, such that the topological order of a set of commits is also the temporal order, both by author date and commit date. Honestly, I don’t think a different temporal order matters that much but as I’ve said before, blub studies are important!

Time’s up!

Weeknotes

2025-06-06T00:00:00+01:00

Weeknotes for the week finishing Friday, 6th June 2025.

Paternity leave! More specifically, additional paternity leave!

The University of Oxford Additional Paternity/Partner Leave scheme means I have 10 additional weeks of paternity leave. Needless to say, the scheme is amazing.

Last week I started thinking about pair programming as an alternative to pull requests. This week I read “Patterns for Managing Source Code Branches” by Martin Fowler. I’d like to learn more about continuous delivery and “Patterns for Managing Source Code Branches” gave useful and interesting context.

Time’s up!

The victory of form over content

2025-06-06T00:00:00+01:00

You’re a junior academic at a British university. You know it’s important to publish your research in high-impact journals. You’re already the first author on several papers. But you’re worried: you need more. And the journals aren’t high-impact enough.

Someone from the university’s Department of People is organising a half-day workshop called “The hitchhiker’s guide to getting your research published”. Other junior academics will discuss their perspectives on publishing their research. A senior academic will give a short presentation. It’s just what you’re looking for!

You arrive early. There’s tea and coffee. Someone mentions pastries. The senior academic gives a short presentation — more of an introduction, really — and departs, muttering something about a committee. You’re put into a group with two other junior academics. One is a historian. You learn that historians don’t publish papers in journals. Instead, they publish books. The other is a computer scientist. You learn that computer scientists publish short papers in conference proceedings. You’ve been told that conference proceedings are low-impact, but not in the fast-paced world of computer science, where you learn that they’re high-impact.

The three of you end up discussing how much teaching you’re doing, how hard it is to find a parking space, and how yeasty the beer is at the recently-opened micro-brewery. When the person from the university’s Department of People asks each group to report back, the historian and the computer scientist look at you. When it’s your group’s turn, you say something vague about different disciplines having different norms and make a joke about not needing a car parking space: if you like a yeasty beer or two after a day spent teaching, then it’s better to take the bus home.

You promise to stay in touch with the historian and the computer scientist. Whilst you’re grateful for the coffee and the pastry, you’re left with the nagging feeling that you don’t have much to show for your time.

You have experienced the victory of form over content.¹²

It’s easy to spot form-over-content events. Is the form, or structure, of the event planned in advance? Does it emphasise group work? Is the content not planned in advance? Should it emerge from within the groups? Are there several rounds of group work, bookended by “Opening remarks” and “Closing remarks”? If you answered “Yes” to these questions, then you’ve spotted a form-over-content event.

It isn’t expensive to organise form-over-content events. The organiser need only understand the structure, and the structure of the event is the same as the structure of each round of group work: someone says something to start; people do something; and someone says something to stop. The organiser need not understand the content. Indeed, the organiser need not have much interaction with anyone who understands the content in advance of the event.

Successful form-over-content events rely on the participants, who should be able to compare their perspectives quickly and clearly. If the event is about getting your research published, then the participants should be able to compare their perspectives on, for example, reading, writing, and note-taking strategies. Ideally, the participants should have diverse perspectives, both in kind and in degree. In the previous example, some participants should have well developed note-taking strategies — index cards, pin boards, a means of cross-referencing — others, less so. Perhaps they have a stream-of-consciousness Google Doc, into which they pour whatever they’re thinking, whenever they’re thinking it.

All this is to say that form-over-content events are more likely to benefit the organiser than the participants. What to do? As the organiser, ask yourself what you can do to ensure the participants have diverse perspectives. You could randomly assign participants to groups. As a participant, ask the organiser how the content should emerge from within the groups. Who will report back? Could the task of reporting back be rotated within each group?

Good luck!

I didn’t name the victory of form over content; my former colleague, Dr Tim Stojanovic, did. He must have mentioned it to me sometime between 2013 and 2014, when I was a junior academic at the University of St Andrews. ↩
Peter suggested that the victory of form over content was synonymous with cargo cult science. I’m not sure. Whilst the organisers of form-over-content events “focus on superficial aspects of a process without understanding the underlying principles”, the process (the form) and the underlying principles (the content) are unrelated. ↩

Weeknotes

2025-05-30T00:00:00+01:00

Weeknotes for the week finishing Friday, 30th May 2025.

I updated my script for creating weeknotes. It now creates an article for a given date, rather than for today’s date, which means I’m less likely to run it on Friday and struggle to remember what I did that week (I’m writing this on Wednesday!). Expect longer, but not necessarily more interesting, weeknotes.

I read “On Pair Programming” by Birgitta Böckeler and Nina Siessegger. The article describes how to pair, the reasons why you should and shouldn’t pair, and concludes that you should pair judiciously. It isn’t a quick read — it’s over 9,000 words long — but it’s well-structured, with clear headings and sub-headings.

I wasn’t surprised by the article, but I’m keen on pairing. It demonstrates to people who are less-than keen on pairing that well-respected people who work for well-respected organisations have thought about the benefits and challenges of pairing, and conclude that pairing (mostly) works.

Lucy recommended the article in Slack; not to me specifically, but to the tech group in general. Pairing isn’t the “sensible default” in the tech group. I’m coming to the conclusion that it should be.

I’ve been reading the sed and grep manuals (GNU). I picked sed because I don’t often use it outside Neovim and even then in a very limited way (:%s/regexp/replacement/g). I was curious about what I was missing. I picked grep because I want to use it more, and more effectively: I’ve found it hard to learn on the job. Blub studies are important!

Reflections on my first six months as a tech lead

2025-05-25T00:00:00+01:00

I became a tech lead in December 2024. My team is small: just Alice, a junior developer, and me. We were asked to work alongside researchers and health informaticians on the OpenPathology project with the aim of designing, implementing, and evaluating dashboards. Here, I’d like to reflect on my first six months as a tech lead, rather than on the OpenPathology project.

What does the Bennett Institute expect of its tech leads? We describe the role as about 80% leadership and about 20% hands-on engineering, I haven’t gone through my calendar, but I feel that on average I’ve spent about 40% of my time on leadership and about 60% of my time on hands-on engineering, ±10%. However, both my team and the project are small, and my team isn’t responsible for all aspects of the project; just the tech aspect.

We describe leadership as a mix of three activities: line management, providing support (coaching, mentoring, training), and owning the architectural direction of the OpenSAFELY platform. It’s structured around three weekly meetings:

a one-to-one with Alice (about 30 minutes)
a tactical meeting with the other tech leads (about 40 minutes)
a strategic meeting with the other tech leads (about 40 minutes)

I was apprehensive about line management; thoughts of filling in Word forms and approving annual leave depressed me. For the most part, though, line management has been straightforward. There are Word forms to fill in and there is annual leave to approve, but I’ve found I can manage the former by blocking out time in my calendar (and forgetting about the Word forms until the blocked out time arrives). Similarly, managing the latter means clicking “Approve” every so often. Hardly onerous!

The leadership activity I’ve found most rewarding is providing support. I was Katie’s buddy from last September to this January and I enjoyed helping her become an effective junior developer. Being a buddy and being a tech lead are different; I have more direct influence over Alice’s career progression than I had over Katie’s, for example. However, providing support is common to both roles and I’m enjoying helping Alice just as much as I enjoyed helping Katie. Answering questions about our systems and processes, and having to think about how I know what I know, or whether I actually know it, has helped me reason more effectively. We’ve also taken some useful and interesting deep-dives that I wouldn’t have taken otherwise: nginx configuration and Django’s AppConfig with Katie; just shebang recipes and Python design patterns with Alice.

I don’t think I’ve fully taken ownership of the architectural direction of the OpenSAFELY platform, although I am writing a proposal for a feature that will touch several of our systems and processes. Writing a proposal is hard, and there always seem to be more important and urgent things to do. Maybe the things are important but aren’t urgent, so I should schedule them rather then do them? If I did, then my leadership/hands-on engineering split would move from 40/60 to 80/20.

Project management doesn’t fit into any of the leadership activities. I’ve found it rewarding, nonetheless. Writing good issues, like all writing, is a skill. There are more opportunities to practice this skill as a tech lead than as a senior developer (someone has to write the issues, after all!). There’s also something about setting up a Kanban-style board and watching the issues move across it from left-to-right that I find rewarding.

To summarise: providing support and project management are rewarding; line management isn’t nearly as onerous as expected; and taking ownership of the architectural direction of the OpenSAFELY platform needs more work.

Time’s up!

Weeknotes

2025-05-23T00:00:00+01:00

Weeknotes for the week finishing Friday, 23rd May 2025.

Katie showed me a code snippet within which I noticed the following idiom:

del a_list[:]

I was sure that [:] copied the list, but I wasn’t sure what del did to it. Deleted a copy of the list? Deleted the contents of a copy of the list? What happened to the list?

After experimenting in the Python shell, I realised that del and [:] deleted the contents of the list. In other words, they’re equivalent to the following:

a_list.clear()

I find the use of del and [:] confusing. If you slice a list, then you get a new list. But if you delete a slice of a list, then you don’t: the elements contained by the slice are deleted from the list.

When we describe code as “idiomatic”, we mean “a characteristic mode of expression” (Oxford Languages). In this case, however, I mean “a group of words established by usage as having a meaning not deducible from those of individual words” (Oxford Languages).

This week, I also finished writing a proposal for a feature that will touch several of our systems and processes; talked about the future of the OpenPathology project; and helped Alice update the case-control studies documentation.

Weeknotes

2025-05-16T00:00:00+01:00

Weeknotes for the week finishing Friday, 16th May 2025.

Annual leave!

For several reasons I found myself going to bed and waking up early. I started reading about Rust (it’s a programming language, dad) and working through the Getting Started section of The Book. I was impressed by the quality of the documentation and the friendliness of the community. I should make time to learn Rust properly.

Weeknotes

2025-05-09T00:00:00+01:00

Weeknotes for the week finishing Friday, 9th May 2025.

After a hiatus, weeknotes are (hopefully) back. In the last six months, I’ve become a dad, taken a month off, and become a tech lead. Nevertheless, these are weeknotes. What happened this week?

Several things happened this week, but I only have time to write about one: I was frustrated by an idea that I keep hearing at work. Namely, that you need a copy of production data for local development. Or, if you’re not using a copy of production data when you’re doing local development, then you’re not doing local development right.

Certainly, it’s handy to tinker with a copy of production data; you can tailor a user interface to fit a long string or to fit domain-specific words and expressions (“jargon”). Having a copy of production data also makes replicating bugs easier. However, production data can be large. They can also contain personal data, which must be removed before a copy is made. And once the personal data are removed, the copy is no longer a copy of production data.

Time’s up!

Weeknotes

2024-11-08T00:00:00+00:00

Weeknotes for the week finishing Friday, 8th November 2024.

I wrote these on Monday, 11th November. I struggled to remember the previous week!

The REX (Researcher EXperience) team started a new initiative: “Onboarding OpenCodelists”. Lucy and Mary asked us to split into two squads. One squad will focus on maintaining OpenCodelists; the other squad will focus on discovery.

From my perspective, maintaining OpenCodelists is important; discovery, less so. It’s not clear to me what happens after discovery, because the initiative doesn’t seem to allocate any time for developing new features. Lucy also said that she wasn’t clear whether the REX team would be responsible for OpenCodelists for much longer.

Weeknotes

2024-11-01T00:00:00+00:00

Weeknotes for the week finishing Friday, 1st November 2024.

Annual leave!

We went to on holiday. Some walking, some paddleboarding, some running.

Weeknotes

2024-10-18T00:00:00+01:00

Weeknotes for the week finishing Friday, 18th October 2024.

I’m back after a two-week hiatus.

Katie and I have reduced the frequency of our meetings, which I think has helped each of us focus on our other tasks. We’re now meeting on Monday, Wednesday, and Friday mornings for a “short” hour and Monday to Friday afternoons for a “short” half hour.

We covered:

Types of test (MDN’s Django Tutorial lists 16!)
Variable scope
Closures
Nerd sniping

We also took a code walk through job-server. Katie and I reviewed an excellent pull request from Mike, and Katie reviewed a small (also excellent) pull request (also from Mike) on her own.

Besides buddying, I was on tech support. Like last time, it was great to be busy again. Lucy was away, so I took over her non-line management tasks. With buddying, tech support, and non-line management tasks, there wasn’t much time left.

Weeknotes

2024-09-27T00:00:00+01:00

Weeknotes for the week finishing Friday, 27th September 2024.

This was my third full week as Katie’s buddy. We covered:

When QuerySets are evaluated
The nature of the SQL statements generated by Django’s ORM
List comprehensions (listcomps)
Breakpoints, the Python debugger, and pdb++
Django Debug Toolbar
Django’s requests, responses, and middleware (within the context of views and authentication)
Callables (within the context of a URLconf)
And Dracula

Katie also started MDN’s Django Tutorial.

There were many great learning moments with Katie this week. The best? On Monday, I watched Katie find out that MyModel.objects.count() caches the result. On Friday, I heard Katie ask “Why does this view take less time to render, when I refresh the page?” and immediately answer “Ah, because the results are cached!”

Besides buddying, I was on tech support this week. It was great to be busy again, after several months of quiet. I revisited how we restrict access to datasets, and had a couple of productive conversations with Lucy and Ben B.C.

Finally, Simon reminded me of “Hypermedia Systems”. I’m really enjoying it. If nothing else, then it’s helping me understand why I have an aversion to SPAs.

Weeknotes

2024-09-20T00:00:00+01:00

Weeknotes for the week finishing Friday, 20th September 2024.

This was my second full week as Katie’s buddy. We covered:

the Django shell (manage.py shell)
the database shell (manage.py dbshell)
displaying two shells in one window with Terminator
creating and saving models with managers and instances
environment variables and their use in the twelve-factor app methodology
file permissions

I gave an hour-long talk on “Writing and running study code”, a 10 minute talk on job-server, and another 10 minute talk on the OpenSAFELY command-line interface.

I revisited how the OpenSAFELY platform restricts datasets, tracing a line from the OpenSAFELY command-line interface to GitHub issues, via research-template and research-action. I’m hoping to discuss how the OpenSAFELY platform restricts datasets in next week’s “Cabinet of Curiosities”.

Weeknotes

2024-09-13T00:00:00+01:00

Weeknotes for the week finishing Friday, 13th September 2024.

This was my first full week as Katie’s buddy. We covered:

setting up a GPG key
the differences between a GPG key and an SSH key
signing commits
the pros and cons of HTTP and SSH for communicating with a remote repository
a couple of useful key bindings; Ctrl-l (clear screen) and Ctrl-r (reverse search history)
piping to xclip (not bad for week one!)

I was pleased we talked about empirical method (“hypothesis-driven development”). Katie seemed to embrace it! We used it to test that Katie could sign her commits, although this took longer than it should have taken because I misunderstood GitHub’s UI.

I decided to learn more about Tig this week. I got confused, because Tig wasn’t doing what I wanted it to do. But then I realised that I was thinking of it as a tool you open once per session, when I should have been thinking of it as tool you open many times per session, once for each task. I set up a couple of useful key bindings, nonetheless.

Getting confused prompted me to learn more about revision selection and to find a well-hidden page in the Git docs called “gitrevisions”. ^ and ~ started to make more sense, which meant git log started to make more sense too.

Finally, I learnt more about Git internals (Chapter 10 of Pro Git). Understanding Git references better helped me work out why git branch didn’t report the branch name, but git status did, for a newly-initialised repository. Katie spotted the discrepancy when we were testing that she could sign her commits. Thanks, Katie!

Less neat, less tidy, and more unpredictable: Improving your code with property-based tests

2023-10-18T00:00:00+01:00

I gave a talk today at the NHS-R conference. It was called Less neat, less tidy, and more unpredictable: Improving your code with property-based tests.

Abstract

As conscientious data scientists, we know that it’s good practice to write unit tests to check that our code is working correctly. However, unit tests cover single scenarios and, despite our best intentions, these scenarios are often far neater, far tidier, and far more predictable than those we encounter in the wild.

How can we cover less neat, less tidy, and more unpredictable scenarios? One answer is to complement unit tests with property-based, or generative, tests. Popularised by Haskell’s Quickcheck library, property-based tests can cover tens, hundreds, or even thousands of disparate scenarios by generating large amounts of input data to match a specification, and by checking that the results of operating on these data are consistent with one or more guarantees. As conscientious data scientists, our job becomes one of carefully crafting the specifications and guarantees, rather than one of carefully crafting the input data.

In this talk, I’ll introduce property-based tests with examples in Python and R. I’ll also discuss how to switch from thinking about input data to thinking about specifications and guarantees. Finally, I’ll describe how we use property-based tests in our work at the Bennett Institute for Applied Data Science to improve our research and to improve ehrQL, the Electronic Health Records Query Language.

Code Reuse in a Trusted Research Environment

2022-11-16T00:00:00+00:00

I gave a talk today at the NHS-R conference. It was called Code Reuse in a Trusted Research Environment: Reusable Actions in OpenSAFELY.

Abstract

OpenSAFELY is a secure, transparent, open-source software platform for analysis of electronic health records data. It can be deployed to create a Trusted Research Environment (TRE) or, alternatively, as a privacy-enhancing layer on any existing TRE.

When undertaking a project within OpenSAFELY, a researcher will typically write one or more scripted actions; logical units of analytic Python, R, or Stata code. However, to encourage the development of well-tested, well-documented code that can be shared between projects, OpenSAFELY supports reusable actions.

Reusable actions are similar to packages in Python or R: versioned collections of analytic code, tests, and documentation that can be made available to other researchers via https://actions.opensafely.org/, the OpenSAFELY equivalent of PyPI or CRAN.

In this talk, I will discuss the challenges of code reuse within a TRE. I will then discuss how reusable actions in OpenSAFELY overcome these challenges, situating reusable actions within OpenSAFELY’s strict security protocols.

I will describe the structure of an OpenSAFELY project, and how a researcher can convert a scripted action, which promotes reusing code within a project; to a reusable action, which promotes sharing code between projects. Although I will give an example in Python, the same principles apply to R or Stata. Finally, I will survey the landscape of existing reusable actions, from the vantage point of https://actions.opensafely.org/.

What is OpenSAFELY?

OpenSAFELY is a secure, transparent, open-source software platform for analysis of electronic health records data.

It can be deployed to create

a TRE
a privacy-enhancing layer on an existing TRE

Why reuse code?

Avoid duplicating identical lines of code in different locations (e.g. by copying-and-pasting)
Instead, use the same lines of code in each location
Over time, improve the functionality that those lines of code implement
Tests
Documentation
Bugs
Features
Do this within, as well as between, projects

Code reuse in practice

Code reuse within a project:

Factor code into functions, classes, methods
Factor these into modules or packages

Code reuse between projects:

Release these to software repositories, such as PyPI or CRAN

TREs and code reuse: some challenges

Skills challenges

Releasing code requires a different set of skills to writing code.

We need to make choices:

Tooling (Flit, Poetry, Setuptools)
Versions of our chosen programming language (3.8, 3.9, 3.10)
Operating systems (Unix-like, Windows)
Versioning scheme (SemVer, CalVer)

Security challenges

You may be familiar with this

pip install matplotlib

or this

conda install matplotlib

or this

install.packages("ggplot2")

In each case we’re downloading a package from the Internet to a computer. What if the computer is within a TRE?

If packages can be downloaded, then can data be uploaded?
→ TREs probably shouldn’t have unrestricted access to the Internet

Where are packages hosted? Which software repository? Which mirror?
Software repositories and mirrors can be compromised.
→ TREs probably shouldn’t have unrestricted access to software repositories and mirrors.

Did we get the package we asked for?
Packages can be compromised.
→ TREs probably shouldn’t have unrestricted access to packages.

Summary

The skills and security challenges can be addressed, but they distract us from achieving our aims:

Avoid duplicating identical lines of code in different locations
Instead, use the same lines of code in each location
Over time, improve the functionality that those lines of code implement
Do this within, as well as between, projects

What are scripted actions?

An OpenSAFELY project is a set of steps, which when run, produce a result. We call each step an action. Each action is run within a Docker container, which doesn’t have access to the Internet.

Some actions address specific tasks: a cohort-extractor action, for example, allows a researcher to extract a cohort.

The researcher analyses the cohort by writing one or more scripted actions in Python, R, or Stata. For example, the researcher may write a scripted action to round counts in a contingency table to the nearest seven.

Scripted actions help us achieve our aims, in part, but they are project-specific: we achieve code reuse within a project, but not between projects.

What are reusable actions?

Reusable actions are scripted actions that have been decoupled from their projects. They help us achieve code reuse within, as well as between, projects.

Whilst we could think of reusable actions as packages, it’s better to think of them as reusable projects.

From scripted to reusable

First step: refactoring

Typically, project-specific information is hard-coded in a scripted action. For example, we may hard-code the location of an input file like this

path_to_input_file = "data/input.csv"

def my_function():
    my_other_function(path_to_input_file)

which we may run like this

python my_scripted_action.py

In a reusable action, project-specific information is passed as arguments to a command-line interface. For example, we may rewrite our scripted action as a reusable action like this

def my_function():
    parser = ArgumentParser()
    parser.add_argument("--path-to-input-file")
    args = parser.parse_args()

    my_other_function(args.path_to_input_file)

which we may run like this

python my_scripted_action.py --path-to-input-file data/input.csv

Refactoring requires some knowledge of how to make a command-line interface in our chosen programming language, but that’s a writing code skill, not a releasing code skill.

Remember: we use the same workflow whether we are working on a scripted action or a reusable action.

Next steps

Create a new project in https://github.com/opensafely-actions
Copy the scripted action to the new project: the scripted action is now a reusable action
Add README.md
Add action.yaml, which contains the run command
Point the existing project to the reusable action like this

my_reusable_action:v0.0.1 --path-to-input-file data/input.csv

Remember: we use the same workflow whether we are working on a scripted action or a reusable action.

Why create reusable actions?

By decoupling scripted actions from their projects we:

Create a location for making improvements (a new project)
Access a mechanism for sharing improvements
Access a mechanism for receiving credit for our work (potentially!)

Security redux

OpenSAFELY restricts access to the Internet; only

are accessible, by proxy.

We have more control over these GitHub organisations than we do over software repositories and mirrors.

Reusable actions have a similar security profile to scripted actions.

Where can I find reusable actions?

Actions Registry: https://actions.opensafely.org/
Code: https://github.com/opensafely-actions
Documentation: https://docs.opensafely.org/actions-reusable/

A Constellation of Data Visualization Tools

2022-06-23T00:00:00+01:00

I gave a short talk today called A Constellation of Data Visualization Tools: Vega, Vega-Lite, and Altair.

The first part of the talk was a brief history of data visualization tools. I described how the ideas in Leland Wilkinson’s The Grammar of Graphics are related to those in ggplot2; and those in D3.js, Vega, Vega-Lite, and Altair.

The second part of the talk was a demonstration of Vega, Vega-Lite, and Altair. Using the editor and the examples, I showed that the JSON objects of a Vega specification are transformed into WebGL or SVG graphics by the runtime; that the JSON objects of a Vega-Lite specification are transformed into the JSON objects of a Vega specification by the runtime; and that the Python objects produced by Altair are serialized into the JSON objects of a Vega-Lite specification.

The third part of the talk related Vega, Vega-Lite, and Altair to the Bennett Institute and OpenSAFELY. Vega and Vega-Lite specifications are well suited to manual (human) and automated (machine) output checking. They can be themed, so a house style can be developed by a designer and applied by a researcher. Finally, they have expressive power and are effective: the former means they are capable of generating a large variety and quantity of ideas; the latter means they perform well, when compared to other notations, on the cognitive dimensions of notations framework.