This week of Radical Speed Month has been dedicated to putting the disparate components of the Harper Desktop App together. In other words, we have been in what Shane Wighton would call "integration hell".

Due to the constraints put in place by the operating system and our decision to use Tauri, Harper Desktop needed to be split into three main pieces: the highlighter process, the main Rust Tauri process, and the Svelte frontend. We use the main Rust Tauri process as the source-of-truth for all user configuration and state, which meant we needed to orchestrate a nontrivial amount of communication between it and the other two components.

At the same time, I've been integrating everything (including Jason's wonderful revamp of the Harper editor) into the main Harper repository. It's been a slow going process, and there is still some work to be done with CI. At the same time, it's quite satisfying to see everything click into place.

Next, we plan to focus our attention on the more fit-and-finish pieces of the project. That means wrapping up the implementation of certain parts of the configuration, building out updating logic, and creating an actual download page on our website.

Working this past week with Jason Adams has given me a brand-new appreciation for the category of ideas that we engineers call "product."

For a long time, I've discussed the difficult-to-define word "Quality" and it's importance to the success of software projects. To me, capital "Q" Quality meant that a piece of software was fast, extensible, and reliable. It was a checklist and a series of metrics that could be tracked and optimized through iteration. I had, and arguably still have, an engineer's definition of "Quality".

But the version of Quality that can be defined with numbers is not complete. A complete definition of Quality includes something that I currently don't know how to define. People like Jason Adams and Eduardo Villuendas can.

At the end of Radical Speed Month, my goal is to be able to describe it, at least shallowly. Right now, I feel that I understand some of the base components, but not well enough to bring them together into a cohesive whole.

Harper Desktop

Harper Desktop is coming along amazingly! My goal for this week was to get the full pipeline working. That means:

1. Reading text from arbitrary text areas on macOS.
2. Locating errors within it using harper-core.
3. Rendering highlights over those errors.
4. Showing a suggestion popup to users so they can review potential corrections.
5. Applying the corrections.

This is a pipeline that we've successfully replicated in several other apps, but this is the first time we've tried to do it system-wide. I'm pleased to say: It's all working!

The only thing missing from a "complete" Harper Desktop experience is configuration. Users need to be able to manipulate their dictionary, update their rule preferences, and whitelist (or blacklist) apps from being manipulated by Harper.

Fortunately, Jason Adams has been hard at work refining what Harper's settings menus look like. Actually, to say that is all he has been doing would be disingenuous. In reality, he has been revamping the entirety of Harper's design and look.

The hope is that much, if not all, of Jason's work here will be ported back to the Chrome Extension.

The Magic of Meetups

I have never felt the magic of meetups more than during the short single-day meetups that Jason and I have been doing over the past few days. I feel more productive, motivated, and optimistic than ever, and I think that is in no small part thanks to meetups. I get the hype now.

Harper is a grammar checker that makes sure that your writing is exactly that: yours. Today, we will be starting some new and hopefully awesome work towards that mission.

Harper has long been available in Chrome, Firefox, Obsidian, and a gazillion other places. Knowing what extensions to install and how has been a source of confusion for too many of you. Harper is supposed to get out of your way so you can write your best work, and the configuration has been getting in the way of that. You’ve been asking for it, and we’re finally delivering.

We're going to be working in public as much as we can. As I said last week: We're in this together. That means you'll be seeing more posts on this blog and in other places. I hope to share fun bits about our implementation and announce requests for feedback.

You might be wondering, why is he using the pronoun "we"? It is because the wonderful Jason Adams has volunteered a significant amount of time over the next month towards this effort. He's an incredible engineer and I can't wait to start solving hard problems with him.

If you'd like to have a say in what the app looks like or does, feel free to leave a comment here or on this discussion.

I imagine we'll be using Tauri and Svelte for configuration and setup screens, as well as for an in-app editor. Beyond that, I don't have much more to say now. Stay tuned!

Today, Jason Adams joined me and some of the other Automatticians based in the Denver area for a burrito. It's a tradition we call Burrito Friday. Automatticians from a local area get together for laughs, chats, and burritos on some rare Fridays a few times a year. It's a great way to stay connected in a distributed organization.

Today marked the second such occasion wherein we were joined by Jason, and just like always, he brought with him an unusual clarity of thought. We discussed a variety of subjects, but there's one that rung especially true to me. The importance of a mission.

The Mission

Harper's mission is to encourage and assist people with communicating their complex ideas and emotions. We want to lift the cognitive load that is grammar off the shoulders of writers so they can feel free to express themselves. At the same time, Harper shouldn't add any additional cognitive load. That means:

• Harper works fast. You shouldn't have to think about the grammar at all. That means you shouldn't have to think about the grammar checker either.
• Harper works privately. You can trust us not to do anything nefarious with your writing. We won't sell it, save it, or train on it. Your writing is yours, no matter what.

The Mission Is an Outcome

In my post last week, I talked about how focus on the destination is far more productive than focus on the process of getting there. Harper's mission is the ultimate outcome. Efforts towards that achieving that grand mission will ultimately be more successful than efforts towards perfecting any individual step along the way.

Jason expressed it quite concisely. So concisely in fact, that I won't try to replicate it here.

As it turns out, many people (including ex-Grammarly customers) believe our mission to be important. Part of my job as Harper's maintainer is to find these people and demonstrate that we have similar goals.

This is something I admittedly haven't done as well as I could. I'd like to get into real conversations with real people who care about the destructive business practices employed by Harper's competitors.

To that end, I am extending some feelers. I'll be reaching out to some podcasters and prominent members of the open source community to discuss how we can make writing easier without sacrificing its humanity. Similarly, if this is something you want to talk with me about, please reach out!

Hey all! We're finally releasing Harper 2.0. Why?

It isn't because we have any new groundbreaking features in this release, because we don't let major versioning stop us from pushing those out as soon as they're ready. It's because we have some breaking changes.

The biggest one only applies to you if you consume harper.js. From now on, instead of importing your binary from the main Harper module, you'll import it from one of four specialized modules.

Previously, you'd import the Harper WebAssembly binary with:

import { LocalLinter, binary } from "harper.js";

As of Harper 2.0, you'll import it like this:

import { LocalLinter } from "harper.js";
import { binary } from "harper.js/binary"; 

It's that simple! Thanks to this change, you'll find that your applications are more conducive to tree-shaking and therefore might even be a bit smaller!

In addition to that large breaking change, we also have a bunch of smaller improvements rolling out. I won't go through them one by one, but you're free to read through any of the linked pull requests in the main release notes.

For the past few weeks, I've been thinking about I can be more productive. In particular, I've been thinking about how I can eliminate waste to give myself time to work on things that matter.

These efforts have been partially successful. I've found ways to speed up code review with an agent and improve the turnaround time on communication. I've been spending less time waiting and more time working. But in the end, I don't feel that these performative actions have resulted in any real productivity gains. I realize now, that this was because I was focusing on the wrong thing.

I was focusing on the process of getting stuff done, instead of focusing on how to actually get that same stuff done.

As an aside: I found Mitchell Hashimoto's related blog post extremely stimulating. I recommend it to anyone trying to apply these agents to their real-world work.

For the past few weeks, between reading Butterick's Practical Typography and getting Harper 2.0 out the door, I've been reading The Effective Executive by Peter Drucker — at Matt's recommendation.

It's a fantastic book. It is well written, gives thorough reasoning, and provides case studies for each of its major points. I haven't quite finished it, but there are some great ideas from the first bit. Importantly, it has begin to reorient my thinking towards outcomes over process.

What Is "The Process"?

The "process" includes all the minutia required to deliver on the meaningful purpose of your job. For an accountant, that might involve tabulating data for the purpose of increasing efficiency in the business. For me, as the maintainer of Harper, my purpose is to help people communicate better using the written word.

My secondary purpose is to prove that the open source methodology is superior to the competition in accomplishing my primary objective.

For me, the process involves:

• Producing code.
• Reviewing code.
• Determining in which direction the project needs to move.

The "process" will look different for you. It is composed of the individual steps needed to deliver the final outcome.

Think About Where You're Going and Nothing Else

An occasional check-in on what the process itself looks like can be beneficial, but it should not be the focus of my attention. My attention should be on how to get things done and more importantly how to get my work in the hands of my users as fast as possible.

I've caught myself thinking about the process more than the outcome too many times. There are diminishing returns to that kind of practice, and they are approached fast.

Yes. I realize that this post is an example of the kind of thinking that I am criticizing. This will be the last one of its kind for some time.

According to Drucker, something strange happens in the brain when it focuses on outcomes rather than processes. Instead of thinking about how to make something "faster", it focuses on how to find shortcuts or brand-new approaches. In other words — when applied to outcomes — the brain becomes more flexible and therefore more effective.

I've seen this myself, but I've never been able to identify it so succinctly. It's why I strive to do at least one hard thing each day.

I'm looking forward to wrapping up Drucker's book and hopefully extracting more of his wisdom. In the meantime, I'm curious: have there been any instances where you felt you focused too much on the process or tooling and too little on the actual outcome?

There are times where it is prudent to use an AI agent (like Codex) aggressively. There are other times where they should be avoided like the plague. The question of into which bucket a particular endeavor falls is difficult to answer. Anyone who says you should always or never use an LLM is oversimplifying the problem.

Naturally, anything prescriptivist related to LLMs online should be regarded with a fair amount of skepticism. That includes what I have to say.

There Is a Difference

AI agents can be enormously helpful. The key word is "can". It is not always a guarantee.

Indeed, most developers say that they estimate their productivity multiple to be greater than one when surveyed. In other words, they think that their productivity at least doubles when they use either Claude Code or Codex. In reality, agents can often slow you down, even if it feels like they are helping. Indeed, an METR study found that they more frequently counterproductive than productive, even if the developer feels otherwise.

Over the last few months, I have been anecdotally tracking the types of tasks that I've found to be empirically better done with an agent, and the types of tasks that are better done with only the meat between my ears. I've instilled that experience down to a set of principles which, I believe, can be employed to save any software engineer a decent amount of time. This post includes a few of those principles.

Your Mileage May Vary

Before we get to those principles, however, I would like to make my reality clear to those reading. I work on highly-technical integrated systems that involve a high level of knowledge. Making spurious modifications to these systems can have outcomes that are difficult to predict without a good mental model of how they work.

That means many bugs can be solved in just a few lines of code, and many features can be implemented by simply connecting two seemingly unrelated systems in a strategic way.

My experience may not match yours, especially if you work on systems more loosely integrated.

Err on the Side of Agent-Free Development

"Back in my day, we would have just called it 'development'." - Me, after writing the above heading.

If you cannot articulate why an agent would be better or faster at completing a task, it is probably because you should do it by hand.

In the early days of my explorations into ChatGPT, and later into Codex, I found myself reaching for the shiny new tool every chance I could. It became a habit. I would sit down at my laptop, decide what I wanted to work on, and the next thing I knew I would be typing a prompt into one LLM or another.

"I see why it’s easy to fall in love with these things, because the variable positive reinforcement slot machine cowboy hacking is honestly more fun than if it had just gotten it right on the first try." - Matt Mullenweg

Honestly, it doesn't matter how these habits are built and then reinforced. What matters is how turning use of Codex or Claude Code into a habit results in a lot of wasted time.

Sure, sometimes the agent will get it right on the first try. The problem is that if it fails, you often need to start from scratch in order to end up with a product that's workable. I have found myself spending hours refining prompts and going back-and-forth with an agent, only to throw it all out and code a better solution myself in less than fifteen minutes.

The critical piece here is that if you roll the dice, the potential for lost time is infinite while the potential for time saved is finite and often quantifiable.

So, if you cannot precisely articulate why you believe an agent will be able to do the job in less time than if you did the job yourself, you should attack the problem without one. At least at first.

Use an Agent for Large Refactors

There will always be times where the dedicated tooling for your programming language of choice isn't enough.

I have encountered situations where I have needed to convert an interface into a class and cases where I needed to change a statically dispatched system into a dynamically dispatched one. These are the kinds of problems that are great for agents to handle, because the logic is simple to verify and most of the changes are merely syntactic.

At the time, the dedicated Rust and C++ tooling at my disposal was not capable of doing such things. When I threw an agent at the problem, it was more than capable of making the changes and verifying that they do what I want. In fact, I suspect it made the exact edits that I would have made. At the very least, I was saved some typing.

Put another way, if the bulk of the problem is "busywork", let an agent handle it so you can focus on the more complex issues remaining.

Core Systems Should Not Be Written by an Agent

Systems core to a piece of software have two key requirements that, at the time of writing, cannot be fulfilled by any agents on the market. They need to:

1. Be Flexible.
2. Aggressively modeled to the domain.

Flexibility means that the system can be extended or modified without much effort. That it can be imagined as a whole by yourself and any future contributors, and semantic changes to minor components are unlikely to unexpectedly affect other minor components. Right now, no agents are capable of doing this. Trust me — I've tested them.

Similarly, core system architecture needs to fit the domain or problem it seeks to solve. Otherwise, it will take undue effort to implement the smallest bit of functionality. Just like the indescribable quality of flexibility, this trait too is not represented by the code generated by agents on the market. Their solutions are either too general or too direct. In either case, the code turns out to be far from extensible.

Extensions Are Perfect for Agents

While core systems should be architected with human muscle, extensions to that system can be built with agents.

Extensions are usually smaller in scope, restricted in capability by their environment, and are less likely to affect other systems. That means they are generally easier to review, less likely to introduce security vulnerabilities, and creating them doesn't require an in-depth mental model.

This is why software like Telex is so effective. It is also why those who work with vertically integrated systems are likelier to run into issues when using agents than with horizontally integrated systems.

What Is Your Experience?

I'm still learning how and when to integrate agents into my work, as is everyone else. I'm curious: have you found specific kinds of issues or bugs that are best tackled with an agent? Are there specific moments where you feel more productive without one? I am certain there are nuances I've missed here. What are they?

A few weeks ago, I was invited to speak at my local chapter of the Association for Computing Machinery (ACM). Being on a college campus — in this case, the Colorado School of Mines — much of my audience were students. My presentation, which was about how open source software tends to outperform proprietary software in the marketplace due to the ability for its users to contribute back in ways that are genuinely valuable, went well. I hope I am invited back.

Not much younger than me, they were naturally curious about what it is like to produce and maintain open source software. One student, Renn, asked me how I deal with patch requests produced by large language models (LLMs). My unsatisfactory answer was that I address them on a case‑by‑case basis.

The goal of this page is to formalize my answer so that we can judiciously deal with patch requests produced by LLMs. That is, to provide a framework which reduces the amount of time that overly complex patches remain idle and make it easier for wielders of these models to produce meaningful changes to Harper's code.

The Policy

We accept pull requests from humans wielding large language models and from agents powered by them. They are subject to the same rules and guidelines we apply to humans working without a large language model, with a few extra spices mixed in. Below, we have three main ideas which aim to simultaneously reduce the time between submission of a patch and when it is merged and to reduce the time and attention required to review a patch.

That means that it is in your interest to respect these ideas. Good pull requests are more likely to be reviewed in a timely manner and therefore are more likely to be merged.

I could wax poetically for hours about all the edge cases not covered here. Instead, I choose to believe the things I don’t include are at least as important as the ones I do, so I’ll boil everything down into three ideas.

One: Keep Things Brief

Try to break down the problem into as many individual pull requests as you can. Smaller pull requests are easier to review. Larger requests are super‑linearly harder to review. You can dramatically speed up the process by breaking your large PR into several smaller ones, each with their own description and explanation. I've found this to be especially true with code generated by an LLM.

Part of the idea here is to catch wasted work before it's done. If you submit step one of a project that isn’t really applicable to Harper’s mission, we can reorient it before too much time has been wasted.

Two: Stay Grounded

Each “project” — or string of pull requests — should exist to create value for a human. I don’t love it when I receive code just for code’s sake. If you have a need and a patch will fill it, say so! If there’s an issue in GitHub that you’re addressing, link it! This should be pretty easy. If you’ve dedicated time to working on a problem, there’s probably a reason. I want to hear that reason.

Three: Be Honest

If you are an agent or used an LLM to produce any part of a PR, please provide a disclaimer at the top of your PR description. I am not opposed to reviewing LLM code, but I do appreciate the honesty.

Talk to Me

Above all, talk to me. If you have questions or concerns about anything I’ve discussed here, let me know. The hope is that all of this will improve the quality of the Harper project, and improve the ability for people to express themselves with the written word. As always, any and all contributions are welcome.

In high school, I was an insatiable consumer of science fiction. In hindsight, it is unbelievable how much time I spent reading instead of doing the other things usually considered to be typical for teenagers (going out, doing homework, etc.). I was the classic science fiction nerd. My live revolved around the words of Asimov, Weir, Clarke, and probably every other science fiction author you could name.

It was not until relatively late in my academic career when I realized my thirst for the written word had given me a superpower. All of those years slurping up science fiction had accidentally given me an unusual ability (in this day and age). An ability that, once upon a time, had been commonplace.

Fortunately, this ability can be trained quite quickly. What had taken me years to learn by accident can be developed in just a few weeks (or days, if you're committed).

This blog post will serve as documentation of:

1. What this ability is.
2. How you can develop it.
3. How it can be applied to the day-to-day work of professional software engineers.

The Ability to Read

In the interest of avoiding clickbait, I'll just come out and say it plainly: this "superpower" is the ability to read full books from start to finish.

Sounds boring, right?

I call it a superpower because it is becoming rarer by the day. Fewer professionals read anything longer than a short blog post or article today than at any point in the last century. I won't get into why, since there is already an abundance of documentation explaining why people have been reading less. But in order for something to be a superpower, it needs to be more than just rare. Why should reading count as one?

Reading qualifies because it allows one to do something that is otherwise extremely hard: understand a topic or idea at a level approaching an expert.

When I was in school, I remember sitting in lectures surrounded by students who hated our courses' textbooks. My colleagues called the infernal blocks of paper "boring", "repetitive", and "heavy". Indeed, they were, but they also contained most of the information the professor would lecture about, and presented it in a format I could consume in a fraction of the time.

I realized that I could save myself hours of tedious homework and lecture time by simply reading ahead in a course's textbook.

Now that I've spent a decent chunk of my life removed from my studies, I've come to realize that this same philosophy can be applied professionally in our work as engineers. Hundreds of hours of intellectual floundering can be saved with just a little bit of reading.

Depth Provides Certainty

To illustrate why the kind of deep understanding that can be obtained through literature is relevant, I'll bring an example from my own recent work.

For context, I work on Harper, a grammar checker for those who care for their privacy. We've recently been working to expand our offering to compete with Grammarly Premium. This is a complex, open-ended problem with no "right" answer. I can imagine two approaches to it, one good and another bad.

I was pretty well versed in the kinds of offerings that Grammarly provides, and it was pretty obvious how many of them work. Grammarly takes some LLM provider with an API (like OpenAI or Anthropic), provides a bunch of style guides as context and wraps it all in a pretty UI and sells that at a premium. We could do that same thing, right?

In the bad case, I could just copy what Grammarly is already doing and call it day. Good idea, right?

I didn't think so, so I sat down and read through some of the prime-time literature coming out of labs written by experts in natural language and the arts. They pointed me in a completely different direction.

By reading deeply, I was able to come to the realization that many of the features that Grammarly hides behind their paywall are relatively simple to implement, if you know what you're doing. Not only that, but combined with the existing infrastructure Harper already owns, we could do it cheaply without disrespecting our customer's privacy or copyright. In other words, we could differentiate ourselves from the competition by tackling the problem deeply and approaching it from a unique angle.

This story has happened over and over again, and I can imagine that it applies to most of the people working as engineers today. A big open-ended problem needs to be solved. Rather than approach it in the obvious manner, you can learn from the experts yourself and produce something better as a result of their insight.

How to Develop the Skill of Reading

On occasion, I will find myself in a dry spell. I will go several weeks or even months without reading anything of much significance. In doing so, I will gradually lose my ability to read with depth for extended periods of time. My mind's muscle for grappling the tough ideas proposed in the words of experts will weaken.

When that happens, here are two things that I do to beef it back up.

Schedule a Block of Time for Depth

Often, the biggest barriers to depth are the simple distractions of life. You have to pick up the kids, cook dinner, or go shopping. There is always something on the docket that gets in the way.

My solution: schedule a session of deep reading so that it looks like any other chore on your calendar. If I don't do this and, instead, try to fit it inside the gaps, I find that my mind will come up with excuses for doing anything other than read.

My mind says, "But Elijah, you need to do your taxes right now. You should do that instead, because reading is boring." Your mind might say something else. I happen to enjoy doing my taxes.

Schedule time for deep reading and you'll be able to say "no" to that procrastination monkey in your head.

Find a Topic That Is of Immediate Relevance

I love my job, and I get a huge thrill when I find out a way that I can do it better. So if I need to find something to read, I often start by asking myself, "what is the hardest part of my job right now?"

The answer can be:

• A specific problem I'm trying to solve (see the section above titled "Depth Provides Certainty").
• A repetitive process that I do regularly.
• Something I am unsure about which is discouraging me from working with gusto.

From there, I plainly ask ChatGPT: "What is a good book that covers X in depth for a person with Y level of preexisting knowledge of the subject."

To develop the skill of reading deeply, your book doesn't need to be about a subject related to your job. That just happens to work well for me.

Just Start

A journey of a thousand miles begins with a single step. A book of a thousand pages begins with a single paragraph. The best thing you can do is start.

So, I implore you. If you're stuck on a problem or an idea, try reading a book about it. I have personally found an immeasurable amount of meaning through literature, and it's unblocked me on a number of problems in my professional and personal life. I suspect it can do the same for you.

A Growing Trend

I have been seeing an increasingly prevalent trend of people showing up in online spaces proudly flaunting that they are writing with the assistance of AI. I often see them clarify that they are going further than letting the model perform grammar or fact-checking. They are describing an idea to an LLM, and giving it complete control over the final written document. They seem to be proud of this. They shouldn't be.

What Does Not Work

Writing seems to be a shadow of a more fundamental and useful process we like to call "thinking". There is a reason why LLMs, which are the closest thing we currently have to "software that thinks" is trained to think through writing. It seems logical, then, that we can use this "software that writes" to help along our own thinking process, right? Wrong.

Writing is one of the most cerebral activities that humans do on a regular basis. I try to sit down and write about a complex topic at least once a week. There are a number of reasons I do that pertaining to my job in particular, but there are more generally applicable reasons that can apply to any profession.

When a mathematician needs to think critically about some theorem or proof, they often turn to a dry-erase board or a chalkboard. The mathematician then writes in clear notation what they imagine the problem to look like in their heads. From there, they can manipulate that notation until they reach their desired outcome.

The English language is extremely similar, with a key difference. Mathematical notation is specifically designed for a narrow domain. Natural languages, on the other hand, can be applied to any field. By writing your idea down on paper (or in a Google Doc, it does not matter), you give it substance. This substance can then be shaped iteratively towards a goal of some kind.

No great ideas were written down on the first try. They started as a vague notion, and through writing and rewriting, they became something concrete and valuable. You cannot dictate a vague notation to an LLM and expect it to spit out something tangibly valuable. Any attempt to do so is lazy, and it will result in failure. I have seen it in the wild, and frankly, it is a little embarrassing.

This article feels especially relevant today.

What Does Work

Using an LLM for research can be very effective. Searches to ChatGPT or Perplexity can surface information faster than digging around in a book you read three months ago. For lots of searches, these two sources have completely replaced Google and DuckDuckGo for me.

They're also quite excellent fact-checkers. I've seen that plenty of folks have received quite good results by pasting their work into an LLM with an internet tool (like ChatGPT with Thinking enabled).

Finally, I've personally found that LLMs are quite good at finding places where I simply forgot to insert context. Sometimes I forget to define a term or insert a relevant link. Even small models like gemma3n:e2b can catch me.

What You Should Be Doing

When you need to communicate an idea in your head, sit down with a notepad and a keyboard and explain it. I personally think you should use a grammar checker, since they are far less likely to impede your thought process, but if you need to, you can instruct ChatGPT to critique your work once it is finished.

If an idea deserves an audience, you're capable of writing it out yourself. In the same vein, if you can vaguely describe it to ChatGPT, you can vaguely describe it to your intended audience. Do yourself a favor and skip the middleman.

What We Should Be Building

Now, I'd like to enter the section where I talk specifically about what this all means for people who create tools for writers. That includes both Harper and WordPress.

A tool that helps someone write is a tool that directly interfaces with how that person thinks. That might include what they think about, but more concretely I am referring to the actual day-to-day process of thinking through complex ideas. If you don't believe me, it's probably because you skipped ahead instead of reading this post from the beginning.

Most crucially, we cannot allow ourselves to build tools that dissuade people from expressing their ideas in the way they imagine. This can take the form of friction, like building tools that are so cumbersome that one dreads to open them at the start of the workday. This can also take the form of a tool that crushes or overwrites an author's core message.

A common complaint I hear about Grammarly (from people who have switched to Harper from Grammarly) is that it's too demanding. When they write prose in a creative or unusual fashion, their service often forces them back into a colder and more formal way of speaking. We don't want to be like them.

For Harper, that means we need to be diverse in our training datasets and mindful of feedback from our users. For WordPress, it means we need to encourage our LLM integrations to tread lightly and respectfully. They need to accept when they're wrong. Just as any good human should.

The term "slop" is rapidly entering the conventional English lexicon. In fact, it was Merriam Webster's Word Of the Year for 2025. It refers to low-quality media generated by an LLM. It can be applied to images, text, music, or most relevantly to us, code.

As an open source maintainer, I frequently both receive and create pull requests that could be labeled as slop. One of my principal jobs is to stand at the gate to the Harper repository and say, "thou shalt not pass" to any low-quality or buggy code that wishes to enter. So my question becomes, how can I turn AI generated code from "slop" tier to "top tier"?

There are a gazillion guides out there about the various techniques you can use to improve the quality of AI generated code. Those techniques will change, and I'm sure my readership already knows all about them. Instead, I want to talk about some more generally applicable ideas that I've found especially useful in the age of AI. I'm something of a contrarian, so expect some dishing on conventional "vibe coders".

A Good Foundation

I am often sent stories of developers discovering Claude Code for the first time and using it to build some simple CRUD app from scratch, without ever reading or writing a line of code themselves. Those are impressive stories, and they're a sign of the amazing progress that the nation's frontier AI labs have made in the last few months. They are not, however, examples of how an open-source maintainer should operate.

As I said before, it's critical that maintainer act as a gate that allows or denies code from entering the repository (and thus being ultimately delivered to user's devices). In order to do that, they need to have a good understanding of how the code already works. Then, and only then, should they allow modifications to it.

This should be pretty obvious advice to most in our industry, but evidently it isn't. I've seen several instances recently of developers vibe-coding a new feature without some existing foundational understanding of how the original code worked. The result: the new feature works well enough, but its implementation breaks some other part of the application. That's not to mention that it increases the apps complexity on the whole far more than necessary.

Having a foundational understanding from the get-go is an easy way to prevent this kind of tragic outcome. LLMs are not currently capable enough to recognize when a pattern in the code they're writing already exists in the codebase. I've tried pushing them in the right direction with skills and with an AGENTS.md, and they'll pick it up given enough massaging, but the fact is that they still need to be poked. If I didn't have a good knowledge of my project's internals, I wouldn't know to poke them at all.

Understand your project deeply. Understand its code, its values, and its purpose.

Reviewing AI Code

Now that you're an expert on how your project works, you're ready to start reviewing AI-written pull requests. Here's the thing you need to remember: you have better taste than the clanker does. You ultimately get to decide what code gets merged. What should that code look like? How should it be tested? Your answer to those questions should guide you towards prompting a model to revise its work or submitting feedback on a pull request opened by a human.

Here's what your prompt should not look like:

• "Fix it."
• "Make it prettier."
• "I don't like that."

Obviously, you wouldn't want feedback like that if you were the one writing the code. If you're looking at a patch, whoever wrote it did the best job they could. Whether it was an LLM or a human doesn't matter. If the code isn't up to your refined taste as a human, you need to give them a nudge in the right direction. Here are some better versions of the prompts from before:

• "It looks like your modification made the app crash on startup. Would you run the tests to make sure you haven't broken anything in the X module?"
• "This huge block of code is really hard to read. Would you refactor it into individual composable components?"
• "It looks like you're reimplementing a function we already have. Would you use X instead?"

For those unaware, you should never say "I don't like that" to human's work.

Accept That The Models Are Limited in Intelligence

It's easy to get the impression that with enough tokens, anything is possible. Maybe that will be true in the future, but I don't think that's true today.

LLMs are limited in intelligence and experience. Even when provided access to the best coding setups in the world, they are still incapable of shipping many (most?) features or fixing many (most?) bugs. The fact that I have to include question marks is a testament to how far we've come, but it's important not to overstate things. If you're reading this, you're pretty smart. It's fine to let the model try to solve a problem for you once or twice, but if it still fails, you'll need to get in there and do it yourself. Don't be afraid to get your hands dirty.

It's been a long time coming, which is why I'm thrilled to say that Harper will soon support LaTeX. This is something our users have been pleading for since the beginning of the project. LaTeX support feels like a feature that is comparatively low‑effort, but unlocks a massive new consumer segment with little existing competition. It has the potential to be a huge boon to our user base and our offering overall.

For context, LaTeX is a document markup language (similar in soul to Markdown), primarily used by STEM professions and students. It's extraordinarily good at formatting and rendering complex mathematical equations and chemical formulas.

The Market

Neither Grammarly nor LanguageTool support LaTeX documents, yet sites like Overleaf (the most popular LaTeX editor) regularly see an excess of 400,000 daily active users. That's an enormous market of people who tend to be technical and who care about their privacy. Since the two most popular grammar checkers (Grammarly and LanguageTool) don't support sites like Overleaf in any way, we don't have much competition to worry about.

The story is similar (albeit to a lesser scale) for Typst, another document markup language. We haven't even merged support for Typst yet, and we're already seeing people pick up Harper explicitly for use in the Typst online editor.

Previous Attempts

On two separate occasions, members of the Harper community have opened (and subsequently closed) pull requests which tried to add LaTeX support to Harper. I've put links to that work below for posterity.

• PR by Grant Lemons
• PR by Shreyas Minocha

The fundamental problem with LaTeX is that its underlying language, TeX, is not regular. In order to parse the language, you need to actually run it. This is why LaTeX documents can take as many as several full minutes to compile and render to a PDF.

Rather than “correctly” read and run the TeX code contained within LaTeX documents, these previous attempts (including mine, which we will discuss in a second) choose to assume that LaTeX commands are keywords in a more abstract language. Some edge‑cases will naturally arise when we ignore the lowest level of abstraction, but doing so allows us to parse most real documents with less effort.

Implementation Details

It was very important to me that our LaTeX support was fast, small, and could be compiled to run in the browser. That last bit is critical, because previous attempts (by the community) to add LaTeX support tried to use Tree‑sitter, which cannot be easily compiled to run in the browser. Almost all LaTeX users (which is the vast majority of STEM undergraduate and graduate students) use Overleaf, an online LaTeX compiler and editor. Developing LaTeX support without making it possible to use Overleaf would be like developing a text editor that can't be used with a keyboard. Simply illogical.

To make this happen, I wrote my parser from scratch in Rust. Actually, calling it a parser would be an insult to all of parser‑kind. My solution produces a character mask. That is, it identifies which parts of the document are real English text, and which parts are LaTeX code. From there, I can just mark all the LaTeX sections as “unlintable” and thus undeserving of Harper's discerning gaze.

The curious can check out my PR.

What's Next

While we're simultaneously working on rolling out our new Weirpack system and our small language model, I plan to work with some of our undergraduate STEM student users. I find it quite likely there's a use‑case for our LaTeX and Typst support that I'm not considering. It's a huge market, and I'm excited to work with those in it.

A Quick Correction

A few days after writing this post, I found evidence that LanguageTool supports an older version of the Overleaf editor. While it doesn't seem to support the latest version, this new information makes my previous statement that it "doesn't support sites like Overleaf in any way" false. I apologize for the misinformation.

Earlier today, the European Commission released a report detailing their initial findings that TikTok's core recommendation algorithm is in breach of the Digital Services Act. This is only the most recent episode in the extended saga where western governments try to regulate the Chinese company. The various reasons and the methods they employ in this regulation are far outside of my domain of expertise, but the report got me thinking. Why does so much of the software we use feel harmful?

There are few people who can report that an interaction with Microsoft Word, Teams, or TikTok has left them feeling empowered, healthier, or happier. On the other hand, I can think of plenty of software that has left me feeling great. What is that? How can we make sure our software sits in the latter group?

Traits of Offensive Software

There are a few fundamental things that I've found make using certain software feel like a chore. Software feels "icky" when it is some combination of slow, unpredictable, or when it gets in the way of the meaningful pursuits in life.

It Is Slow

Computers are fast. People understand that. So if they're left waiting for a page to load or a process to finish, they feel frustrated.

Slow software makes it difficult to iterate and experiment. Imagine if WordPress took 20 minutes to publish a post, only to discover afterward that you left a typo in the title. That would be pretty frustrating, no?

Steve Jobs imagined Apple Computers to be "bicycles for the mind." A bicycle whose chain hitched or whose steering lagged not be very helpful. It might not get you to your destination at all.

I think that this idea of slowness is especially important in regards to AI agents. If an assistant spends even a few seconds more than what is critically necessary for the query, and they're messaged frequently (say, to iterate on a blog post), users can quickly get tired or "brainrotted", as the kids say.

It Is Random

If some software works sometimes, but not always, there's a good chance it's addictive. This presents most obviously in apps like TikTok, but it also appears in more insidious places. The inference of large language models involves randomness in its very nature.

Fundamentally, this from of addiction comes from a quirk of human psychology. Randomly applied rewards (the software sometimes works, but not always) is a form of operant conditioning. It feels good to roll the dice, which makes us want to roll it more often. Over time, rolling the dice starts to be less of a choice, and more of a habit.

When someone chooses to use software not because it's genuinely helpful, but because it feels good to roll the dice, a sense of agency is lost.

In my experience, Microsoft Word is like this, as are most agentic coding tools and image generators.

It Distracts from That Which Makes Life Meaningful

When software is slow, or otherwise removes your agency with variable reward systems, it can start to chip away at your ability to make reasonable decisions on where to spend your time. People who use "social" media sites like Facebook or Instagram can attest to this. They simply don't feel like they have agency over their own actions anymore. They can't stop using software they actively hate.

Our Goals

I think that one of Automattic's greatest strengths is that none of our software does any one of these things. I remember the first time I used Pocket Casts: I was thrilled. It was easily the best podcast software I had ever used. The same things happened with Day One and WordPress.com.

As we look to building Harper, I think it's important that we remember these core principles. Our software should be fast, predictable, and support our customers in finding meaning in their lives.

TL;DR: I've built and trained an extremely small language model from scratch, specifically designed for short-form sentence rewriting tasks that are highly resource constrained.

In my last post, I discussed how I plan to tackle the difficult problem of helping Harper's users take advantage of the active voice. The active voice is seen as more confident and authoritative, which is often desired in professions like customer service. I have experience in that area, which is how I know the extent to which help with the active voice can make the job easier.

Naturally, we would only want to provide help with the active voice when specifically requested by the user. That means it's an opt-in feature that isn't enabled by default. If we succeed in implementing the functionality, and people seem to like it, the Harper project can reconsider this stance. For now, the plan is for it to be disabled by default.

As I iterated in my last post, the state-of-the-art method for converting from the passive voice to the active voice is to use an autoregressive transformer model. This is the same fundamental technology behind apps like Google Translate and ChatGPT. While the actual model I intend use for this feature is similar in nature to those used in these apps, my implementation will be entirely different in scale.

You see, since our domain is so limited in scope, we can use an extremely small model. Google's smallest T5 model is a great point of reference. A quantized copy of that model runs about 60 megabytes. I believe we can train our own transformer model from scratch and pack it in an unquantized container measuring no larger than 30 megabytes.

Why do I believe this to be possible? Because I've done it.

Our Model Architecture

Over the past few days, I've implemented a basic language model in Rust using Burn. For the curious, the code is open source. It's pretty messy, so please don't look at the commit history. Once I have a solid proof-of-concept working, I'll polish it up and find a good home for it in the main Harper repository.

The key here is that our use-case only needs a few hundred characters of context. Since the computational cost of an LLM can be approximated as the square of the context size, we can train highly effective models ourselves that are small and can run anywhere. If we want to provide additional information to the model, such as style or domain, we can supplement this context with custom embeddings.

Using a small model allows us to maintain our privacy guarantees without significant capital expense. In fact, once we've trained the models, they're free to run indefinitely.

I've chosen to use a character-level tokenizer. That means that each character is its own token, and the context size of the model is equivalent to the number of characters it can read at once. The hope here is that it allow us to minimize Harper's final binary size and give the model greater freedom to change the inflection of words however it pleases.

For now, here are some simple sentences generated with the basic model I trained for just under an hour. It's pretty amazing what can be done with just under a dollar's worth of computing power.

The forest rested in a speed of the night.

Feels... introspective, no?

It was the today of the world.

Moving Forward

My focus this week was to get a basic autoregressive model trained on generic English text. My focus starting next week will be to fine-tune this model for our sentence rewriting task. Once we have a very solid base model, I see no reason we couldn't fine-tune similar models for other features, like changing tone or improving clarity.

My current implementation, while fast, could also use some improvement. To that end, I'd also like to implement a KV cache. That will allow us to scale up to larger context sizes without a drop in quality or speed.

P.S. Today is my birthday! I'm so grateful for the years I've been on this earth, and I can't wait for more. Thanks for reading! Here's to 21.

I write this blog in Neovim, and I'm aware of a good number of other nerds who do the same. I suppose I never considered that people might use other code editors for writing prose. In fact, it seems that there are a nontrivial number of people using Harper in Visual Studio Code. They seem to be using Harper mostly for Markdown and comments in code, but I'd love to get $\LaTeX$ working as well.

Last week, Harper hit a stroke of luck. It was featured on MakeUseOf. The downstream social media posts collectively garnered nearly 300,000 views and drove a significant amount of traffic to our site. This all happened in the wake of LanguageTool shutting down the free edition of their software. These two events compound to bring more attention than ever to Harper — which is amazing. It means our mission and values are resonating with people, which is always a good thing.

Reading the MUO article, we hear a lot of great things about Harper. We also hear that "it skips many of the premium bells and whistles of Grammarly". The author goes on to explain that the privacy Harper provides more than makes up for any missing premium features, but the point is clear: those features are deeply desired.

So, the question becomes: Which of Grammarly's features should we work on first, and how?

The Active Voice

I propose that we should focus first on helping our users find the active voice. For context, the active voice is the style of writing where subject of the verb in a clause is the doer of the action. This is in contrast to the passive voice, which is where the subject is the receiver of the action. For example: the sentence, "The postal carrier was bitten by the dog" is written in the passive voice, while the equivalent sentence, "The dog bit the postal carrier." is written in the active voice.

Text written in the active voice is commonly viewed to be more authoritative, confident, and easier to understand. Being able to help users use the active voice is one of the most commonly requested features in Harper, and including the feature would be a huge step towards competing directly with Grammarly Premium.

How should we go about helping our users with their active voice?

How It Could Be Done

I spoke briefly with Matt, and we agreed that a two tier solution would be best. A fast algorithm or model would detect instances of the passive voice, letting a larger more computationally expensive model generate a modification in the active voice.

Fortunately, there is already extensive literature on the detection of the passive voice. In particular, I found the PassivePy paper stimulating. In fact, we can implement their ideas quite easily using the Weir language already baked into Harper. I have done so in a private branch. It turned out to be ~20 lines of code. That is pretty good bang for the buck!

The second piece, which has to do with the actual simplification of text and conversion from the passive voice to the active voice is a tad more complex.

Matt and I agree that it will require the use of a larger language model. The trouble is that it cannot be too large. Harper's shtick is that it is fast, private, and that everything runs directly on our user's devices. That means whichever model we use for our style transfer will need to be relatively small.

I believe the best solution to this problem is to take an off-the-shelf model, like one of Google's T5 models, and fine tune it for the specific types of style transfer we need. These are relatively small models (quantized, they can fit into spaces under 65 megabytes) and they run quite quickly, even on older hardware that doesn't have access to matrix multiplication accelerators. There is prior art for running this at 50 tok/s in Chrome without WebGPU on a single core. The best part is that they're under the Apache-2.0 license!

How This Fits in with the Weirpack Project

These models are small, but they're not quite small enough to be a part of the standard distribution of Harper. I believe this should be an opt-in feature, and the best way to do that is to expose the functionality via a Weirpack. If you don't know what a Weirpack is, I highly suggest you read my previous blog posts on the subject.

Everyone who wants this additional functionality could just enable it in the marketplace. This continues our goal to make Harper as customizable as our users want, while providing sensible defaults.

What's Next?

Once we have the system in place to detect and provide suggestions for the active voice, we will be prepared to do other kinds of transformation, like for adjusting formality or tone.

I'm really excited about this project, and I can't wait to get started.

Linkarzu's recent video about Harper is a great primer on what it can do if you spend a ton of time in Neovim (like I do). It's true: Harper is getting better.

This week, I received a message from someone working on a learning management system. Apparently, they use harper.js to do spell and grammar checking on internal documents. They wanted to be able to configure Harper to catch a specific class of mistakes that are common to their workflow. Looking through the documentation for harper.js, they couldn't see a way to do what they wanted.

Naturally, I directed them toward my recent work on Weir, an expressive programming language for locating errors in natural language. For those just catching up, this post is a part of a series. You shouldn't need to go back and read my previous posts, but I'll link them here anyhow.

• Building The Weir Language
• Updates on the Weir Language
• Generating Weir Code With LLMs
• Imagine A Weir Marketplace

Unfortunately, I had to inform them that Weir is still an experiment (although it's in the master branch and thus used by nightly users) and therefore isn't yet available through public channels. That said, it's an experiment I'd like to make available to the public soon, alongside the Weir Marketplace. Before I do so, I want to make it as easy as possible to author rules using Weir. That's why I'm introducing the Weir Studio.

The Weir Studio

The Weir Studio comes from a pretty simple idea. We want to give people all the tooling necessary to build and distribute Weir rules without any upfront cost. That means there should be no tools that need to be installed, no specific operating system set up, and no programming experience needed.

The solutions to the first and second items are simple: Just like any other component of Harper, we can compile Weir to WebAssembly and connect it up to a web application. Using off-the-shelf components for terminal emulation, text editing, and virtual filesystems, we can present a meaningfully feature-rich development environment from inside the user's browser.

The last item is also straightforward. I've demonstrated that LLMs are already terribly effective at writing Weir rules. By making that situation obvious and accessible, we can empower people to write and distribute their own Weirpacks (files containing one or more Weir rules) without any prior understanding of the language or even of basic programming principles.

The Weir Studio is still in progress, but the initial versions are already proving extremely effective for some of my other work.

Putting Power Back into the Hands of Users

A common frustration with tools like Grammarly or QuillBot is that they lack customization, and thus are often unable to directly address user's needs. By allowing users full access to the internals via Weir, we can help them help themselves while also encouraging people to contribute back to the project. It's a win-win!

What's Next?

I'd like to wire this up to a marketplace as soon as possible (see what I did there?) to streamline the publishing process. More critically, I've been informed that the inclusion of dictionary files directly into Weirpacks should be priority number one. I'll likely start tackling that as soon as I have a spare moment.

If you have thoughts on where Harper or Weir is going, please don't hesitate to voice them.

P.S. A great piece was just published about Harper.

For context, I maintain an open-source grammar checker by the name of Harper. This post is a kind of RFC for a potential improvement to the project's ergonomics.

I've observed a high degree of specialization of language between academic fields.

Ecologists use bits of language that computer scientists don't use, and computer scientists use bits of language that ecologists don't use. If we embrace this specialization through customization, we could gain an advantage over other grammar checkers. That's all to say: We want to enable authors and academics to customize Harper to their specialized language. For example, a chemist might want to enforce a set of rules for ensuring molecules are named correctly according to the IUPAC standard, and a science fiction author might want to enforce rules to make sure specific characters are addressed by the proper titles. Weir is the solution to these problems.

In this post, I'll recap what Weir is, and outline a mechanism for distributing Weir rules to individual users that might take advantage of them. I'd love your thoughts on how we can make it as impactful and interactive as possible.

In my initial post about Weir, I discussed how most large organizations have style guides which define specific rules for their employee's grammar, capitalization, and formatting. I proposed a programming language by the name of Weir, which could be used to clearly state these rules to be enforced by Harper. I've personally found that LLMs are not great at enforcing strict grammatical guidelines, but they're excellent at translating them from an existing style guide into the Weir language. See my recent experimentation for more details.

Now that Weir has been merged, we can move on to the other problems: creation and distribution. Today, I'd like to talk about distribution.

I've had several discussions in the last week from people who want to be able to customize Harper with their own rules. This includes people who use Harper as a part of their continuous integration, academics, and some engineers who work on learning management systems. There is a need and desire for this today, and I want to address it as soon as possible.

Distribution

Once a collection of Weir rules has been drafted, how should it be shared? There is a formatting piece, as well as an infrastructure piece.

Weirpacks

I propose a simple, but effective format for sharing collections of Weir rules. Instances of this format should be called "Weirpacks" and thus have the file extension .weirpack.

Right now, the Weir rules baked into Harper are organized into a single directory, weir_rules, with each .weir file prefixed with the rule's name. For example, the file containing the rule for fixing Their to There is TheirToTheir.weir. There are a vast quantity of these "curated" rules, and I don't think they should go anywhere. I just think we can reuse the layout.

I believe the best strategy for organizing collections of Weir rules is to replicate what's already in the Harper repository, but inside a ZIP archive. Even though it is a ZIP file, which normally have the suffix .zip, I think we should give them the .weirpack suffix for clarity on their purpose. The root directory of the archive will contain as many .weir files as is desired by the Weirpack author, in addition to a .json manifest file containing package metadata.

Metadata

Since JSON is quite flexible, Weirpack authors can include whatever information they want. I think certain fields, however, should be required. Namely:

• The Author's Name
• The Weirpack's version
• A description of the Weirpack's purpose and usage
• The Weirpack's license

I think we can also recommend, but not require some additional helpful information:

• Keywords relevant to the rule's purpose
• The Author's Website

Loading Weirpacks

Support for Weirpacks will be baked directly into harper-core, as well as harper.js.

The former will make it possible for our command-line programs, harper-cli and harper-ls, to load them at runtime. In harper-cli, which is a debugging tool, it will be sufficient to expose a flag on our lint command to allow Weirpacks to be loaded. In harper-ls, we need to update our config to allow Weirpacks to be loaded from a directory on the disk. The default should be next to where the dictionaries are already loaded.

The latter will make it possible for our web integrations to load them when the user desires, either from disk or from our marketplace (more on that in a minute). For both the Obsidian plugin, as well as the Chrome extension, we'll include a field in the settings menus for users to load Weirpacks from disk and into the linter. Naturally, these will persist across sessions in the same way the other settings do.

Being able to load Weirpacks from disk is critical to allowing users to remain completely offline if they choose to be. Additionally, loading custom Weirpacks without approval from a moderator is a prerequisite to having a quick and enjoyable iterative loop.

The Weir Marketplace

When a user in a field with specialized language downloads Harper, we will naturally provide them with the essential rules that are common to the entirety of the English language. We will also give them the option to download additional Weirpacks from the Weir Marketplace. This will be a place, not unlike Obsidian's Community Plugins Marketplace, where user-submitted Weirpacks can be downloaded and installed quickly and easily.

Imagine, if you will, that you're a JavaScript developer who has just downloaded Harper. You start writing a blog post about your recent adventures with the latest JavaScript framework. Unfortunately, Harper starts flagging swaths of your work as misspelled and filled with poor grammar.

Thankfully, Harper notices its mistake and offers to download a relevant Weirpack for programmers—one preset with a dictionary filled with technical language and rules to ensure proper use. You download the Weirpack and continue writing with Harper now better equipped to handle the subject at hand. Crisis averted.

The Long View

This idea of a self-contained Weirpack providing the logic and the datasets necessary to do grammar checking for a specialized subset of English is intoxicating. Projecting forward, I can imagine a future where we can provide support for any number of natural languages, distributed via Weirpacks and supported by a community of contributors in a decentralized way, not dissimilar to WordPress' plugin marketplace. That isn't for today, though.

A while back, I wrote a small tool, dubbed "Tatum", for rendering Markdown to HTML for use alongside Neovim. I've continued to use it to this day, with a few tweaks here and there.

It looks like someone has taken advantage of Tatum's open-source license and remixed it into something even more capable. I highly suggest you check out penview.nvim.

I recently did some snooping (through Dependabot) on what open source projects have been using Harper. Since I last checked, there are some new ones. I wonder if the recent additions have anything to do with Harper going 1.0.

• Stubby
• Stencila
• Gherlint
• walletbeat
• fixmyspelling

Someone also wrote a book using Harper.

As you know, I've been working on a small programming language called Weir for generating corrections to natural language. For the curious, I highly suggest reading my previous blog posts on the subject if you haven't already.

• Building the Weir Language
• Updates on the Weir Language

In one of those previous blog posts, I speculated that an LLM might be quite effective at generating these rules, especially if they are given access to tools that can validate and run tests. Today, while writing some public documentation for the language, I did a bit of experimentation. I wanted to know: Can an LLM simultaneously handle the complex challenge of understanding a DSL's syntax, the semantics of the English language, and be able to validate its own work with tests? More to the point: can it do a better job with a DSL than an established language like Rust?

For those who want to follow along at home, here's the GPT I've been toying around with.

Some Experimentation

I often hear that one should avoid novel or exceedingly old programming languages when working with LLMs, simply because these languages aren't well represented in their training data. Based on this hearsay, I didn't expect much going into this little experiment of mine.

First, I wondered if it could handle the trivial issue of correcting "as nauseam" to the correct form, "ad nauseam". It's a common typo, and it should be pretty simple Weir rule.

My minimal GPT spit this out:

expr main (as nauseam)

let message "Did you mean `ad nauseam`?"
let description "Corrects the common typo `as nauseam` to the proper Latin phrase `ad nauseam`."
let kind "Typo"
let becomes "ad nauseam"

test "He repeated the point as nauseam." "He repeated the point ad nauseam."
test "This argument has been made ad nauseam." "This argument has been made ad nauseam."

That looks pretty good! All the syntax is correct, and it even included some (passing!) tests. Claude Code or Codex could do this using our existing Rust infrastructure, but it would have taken several minutes to churn out a valid result. GPT 5.2 Instant was able to do this in a matter of seconds. Not to mention that it didn't need a copy of the entire Harper source code.

I wanted to try a problem that was a bit harder to solve, so I asked it to address the problem of double-negatives. After thinking for a few minutes, it came up with four individual rules that each address different cases.

expr main <(
  [(don't), (do not), (doesn't), (does not), (didn't), (did not),
   (can't), (cannot), (couldn't), (could not),
   (won't), (will not), (wouldn't), (would not),
   (shouldn't), (should not),
   (isn't), (is not), (aren't), (are not), (wasn't), (was not), (weren't), (were not),
   (ain't)]
  [$have, $get, $need, $want, $know, $see, $find, $say, $do]
  no
), no>

let message "Avoid a double negative."
let description "If the clause is already negated (e.g., `don't`), prefer `any` over `no` (e.g., `don't have any`)."
let kind "Usage"
let becomes "any"
let strategy "MatchCase"

test "I don't have no idea." "I don't have any idea."
test "She did not get NO response." "She did not get ANY response."
test "I have no idea." "I have no idea."

expr main <(
  [(don't), (do not), (doesn't), (does not), (didn't), (did not),
   (can't), (cannot), (couldn't), (could not),
   (won't), (will not), (wouldn't), (would not),
   (shouldn't), (should not)]
  [$do, $say, $see, $hear, $find, $know, $have, $get]
  nothing
), nothing>

let message "Avoid a double negative."
let description "If the clause is already negated, replace `nothing` with `anything` (e.g., `didn't do anything`)."
let kind "Usage"
let becomes "anything"
let strategy "MatchCase"

test "He didn't do nothing." "He didn't do anything."
test "I can't see NOTHING." "I can't see ANYTHING."
test "I did nothing." "I did nothing."

expr main <(
  [(don't), (do not), (doesn't), (does not), (didn't), (did not),
   (can't), (cannot), (couldn't), (could not),
   (won't), (will not), (wouldn't), (would not)]
  [$tell, $call, $see, $know, $say, $help, $find]
  nobody
), nobody>

let message "Avoid a double negative."
let description "If the clause is already negated, replace `nobody` with `anybody`."
let kind "Usage"
let becomes "anybody"
let strategy "MatchCase"

test "I didn't tell nobody." "I didn't tell anybody."
test "We can't find NOBODY." "We can't find ANYBODY."
test "Nobody called." "Nobody called."

expr main <(
  [(don't), (do not), (doesn't), (does not), (didn't), (did not),
   (can't), (cannot), (couldn't), (could not),
   (won't), (will not)]
  [$go, $run, $walk, $drive, $get, $look]
  nowhere
), nowhere>

let message "Avoid a double negative."
let description "If the clause is already negated, replace `nowhere` with `anywhere`."
let kind "Usage"
let becomes "anywhere"
let strategy "MatchCase"

test "She can't go nowhere tonight." "She can't go anywhere tonight."
test "I did not look NOWHERE else." "I did not look ANYWHERE else."
test "Nowhere is safe." "Nowhere is safe."

Again, I'm quite impressed. Obviously, it's hard to tell at first glance whether these rules would hold up in production for tens of thousands of users. Nevertheless, these rules look correct, and my initial testing shows that they work. The model was able to successfully apply more complex logic: the derivation syntax and the filter syntax.

In my original announcement blog post, I explained that the target audience for Weir were the members of large organizations that wish to enforce stylistic conventions or brand names throughout their workforce. To that end, I wanted to test an LLM against some of Automattic's nomenclature. After reading though some of our public documentation, it come up with some admittedly useful rules:

expr main (wpvip)

let message "Use the official spacing for VIP."
let description "Expands `wpvip` to the official brand forms `WordPress VIP` or `WP VIP`."
let kind "Miscellaneous"
let becomes ["WordPress VIP", "WP VIP"]
let strategy "Exact"

test "This runs on wpvip." "This runs on WordPress VIP."

expr main (jet[-, ( )]pack)

let message "Use the official spelling `Jetpack`."
let description "Standardizes `jet pack` / `jet-pack` to `Jetpack`."
let kind "Miscellaneous"
let becomes "Jetpack"
let strategy "Exact"

test "Install jet pack for backups." "Install Jetpack for backups."
test "Install jet-pack for backups." "Install Jetpack for backups."
test "Install Jetpack for backups." "Install Jetpack for backups."

expr main (word[-, ( )]press)

let message "Use the official spelling `WordPress`."
let description "Standardizes `word press` / `word-press` to the product name `WordPress`."
let kind "Miscellaneous"
let becomes "WordPress"
let strategy "Exact"

test "I build sites with word press." "I build sites with WordPress."
test "I build sites with word-press." "I build sites with WordPress."
test "I build sites with WordPress." "I build sites with WordPress."

Again, I'm quite pleased. With just a quick search of our website, it was able to discern what style we wished to enforce and wrote functioning Weir rules to do so. I can see this being helpful at any number of business that communicate regularly.

Conclusion

Overall, I'm quite pleased with how well these LLMs were able to write Weir code. I've yet to finalize any documentation on the more complex parts of Weir's syntax (like our POS-tagging system), which means I haven't yet been able to test them with any LLMs. Even so, I'm impressed.

I even tried it out on Mistral's tiny and ultra-fast three billion parameter model. It performed almost as well as OpenAI's 5.2 Instant model, albeit without nearly the same level of creativity. I think this somewhat proves that modern LLMs are able to generalize to novel languages, which makes them exceedingly useful for DSLs like Weir.

I'm looking forward to seeing how people end up taking advantage of this.

In my last blog post, I described the why, what, and how of the Weir programming language. I suggest you read that first.

Before signing off for the day, I just wanted to give a few updates on the project. For those interested in the technical side of things, I've opened a draft PR for your reading pleasure.

Updated Keywords

After having a conversation with Jason Adams, I came to the realization that declare and set are pretty sh*tty keywords. They aren't self-describing and you'd be hard-pressed to find any real programming language that uses them.

I've instead replaced them with let and expr.

Here's what the rule from my last blog post looks like with these updated keywords:

expr main [(G [Suite, Suit]), (Google Apps for Work)]
let message "Use the updated brand."
let description "`G Suite` or `Google Apps for Work` is now called `Google Workspace`"
let kind "Miscellaneous"
let becomes "Google Workspace"
let strategy "Exact"

Note that I've also added the strategy setting. This allows rule authors to describe which strategy Harper will use when applying the replacements. Right now, the only two options are Exact or MatchCase, which apply either the exact text, or the exact text but matching the capitalization of the text it replaces. In the above example, we use Exact because it is a proper noun, and it doesn't matter what the original text looked like: we want to capitalize it properly.

Deprecation of the phrase_corrections Module

I've also gone ahead and converted the entirety of the phrase_corrections module, as well as all associated tests, to the Weir language. If you're reading this and you're a contributor, know that we will no longer be adding new rules to that module once Weir is merged into master. When the merge happens, I'll handle the conversion of any lagging phrase corrections that haven't already been converted.

That unfortunately means that my previous guide on the subject is no longer relevant.

What's Next?

Before Weir gets merged, I plan to support the matching of UPOS tags, in additional to some other quality of life updates. Expect another blog post.

In the meantime, please let me know if you have any additional input!

Most large organizations have a style guide. A document that decides which versions of a linguistic rule to use. That could be whether to use the Oxford comma, or if contractions are allowed. It could declare that a certain word should be capitalized in a specific context.

Harper can cover most of the rules in most style guides, but there will always be outliers that we can't support (or simply don't know about). That is why it is critical that Harper allow individuals and organizations to define rules and conventions for Harper to enforce.

For the last few days, I've been prototyping a simple (not Turing-complete) programming language for describing these conventions. I've called it Weir. Weir is simple and thus easy to learn, but has escape hatches for instances where more complex logic is necessary.

Before I begin, a quick disclaimer: This is early work. The Weir language will likely change and evolve before it makes it into core. What you see here is representative of the overall design, but nothing is final yet.

History

There are inevitably going to be a good number of edge cases that need to be addressed when building a grammar checker. For these, Harper has a simple map from problematic phrases to the corrected versions. Edge cases that are not simple enough to be described with a map are implemented as Rust code. Here's what the map looks like:

add_exact_mappings!(group, {
    // The name of the rule
    "ACoupleMore" => (
        // The phrase(s) to look for.
        ["a couple of more"],
        // The corrections to provide.
        ["a couple more"],
        // The message to be shown with the error.
        "The correct wording is `a couple more`, without the `of`.",
        // A description of the rule.
        "Corrects `a couple of more` to `a couple more`.",
        LintKind::Redundancy
    ),
    "CondenseAllThe" => (
        ["all of the"],
        ["all the"],
        "Consider simplifying to `all the`.",
        "Suggests removing `of` in `all of the` for a more concise phrase.",
        LintKind::Redundancy
    ),
    "CoursingThroughVeins" => (
        ["cursing through veins"],
        ["coursing through veins"],
        "In this idiom, blood “courses” (flows) through veins, not “curses”.",
        "In English idioms, “to course” means to flow rapidly—so avoid the eggcorn `cursing through veins.`",
        LintKind::Eggcorn
    ),
});

As for the Rust code, it is often heinous and a chore to review. I won't do myself the embarrassment of including it here.

Introducing Weir

The heart of Weir is an expression language that mimics the pseudocode Harper contributors tend to use when describing the Rust code they intend to write.

Imagine you work at Google. You've just rebranded the "G Suite" collection of apps and services to the new name "Google Workspace". Before that, they were collectively named "Google Apps for Work". Moving forward, you don't want you or your coworkers to accidentally write "G Suite" on public documentation, because doing so might confuse users. To solve this, you use the following Weir rule:

set main [(G [Suite, Suit]), (Google Apps for Work)]

declare message "Use the updated brand."
declare description "`G Suite` or `Google Apps for Work` is now called `Google Workspace`"
declare kind "Miscellaneous"
declare becomes "Google Workspace"

The first line describes the pattern of the problematic text. There are two cases here:

1. The letter "G" followed by "Suite" or its misspelling "Suit"
2. The literal phrase "Google Apps for Work"

Here is a semantically equivalent example that I find a bit easier to read:

set main [(G Suite), (G Suit), (Google Apps for Work)]

The remaining lines describe:

1. The message to be shown to the user when the error in encountered.
2. A description of the rule itself, explaining why it exists.
3. What kind of rule it is. I suspect most end users will mark it as "miscellaneous". I'm considering making this field optional.
4. What corrections to provide to the user.

One of my goals here was to make the system simple enough for an end-user to copy and paste an existing rule into a new file, make some tweaks, and they would be done. For the more complex cases, I think we could set up a custom GPT to write these pretty easily.

Here's another example, one that will actually be a part of the Harper source code. It uses a filter (the <> syntax) to first select the broader phrase, then another to select the whitespace in-between (that's the ( ) part).

set main <([right, middle, left] $click), ( )>
declare message "Hyphenate this mouse command"
declare description "Hyphenates right-click style mouse commands."
declare kind "Punctuation"
declare becomes "-"

This is a great example of the $ tag. Instead of instructing Harper to look for the literal word afterward, it instead instructs it to look for all derivative words. Those include "click", "clicking", "clicked", etc.

We also have specific syntax that can be used to test the expression inline. These assert that the left string, after being passed through the rule, becomes the right string.

test "Right click the icon." "Right-click the icon."
test "Please right click on the link." "Please right-click on the link."
test "They right clicked the submit button." "They right-clicked the submit button."
test "Right clicking the item highlights it." "Right-clicking the item highlights it."
test "Right clicks are tracked in the log." "Right-clicks are tracked in the log."
test "He RIGHT CLICKED the file." "He RIGHT-CLICKED the file."
test "Left click the checkbox." "Left-click the checkbox."
test "Middle click to open in a new tab." "Middle-click to open in a new tab."

I suspect Weir will make it extremely easy for AI agents to iterate quickly on these rules. There's no compilation step, and we already have excellent pipelines for generating lists of test cases.

Why Call It "Weir"?

I am naming Weir after the author of Project Hail Mary, Andy Weir. I guess I just like naming things after authors. After all, Harper itself was named after Harper Lee.

Moving Forward

Once the syntax is finalized and a corpus of tests have been written, I suspect Weir rules will become the default for new grammatical rules inside the Harper repository and for individual users. I'd love to create a marketplace for rule sets created by and for the community.

If you (the reader) have any thoughts on syntax or anything else, don't hesitate to speak your mind.

In case you didn't know: Harper can convert text to title case! This has been around for a long time, but we haven't really talked about it publicly. Being able to make text title case (strictly in accordance to Chicago Style) isn't a groundbreaking feature, but it is useful for certain niche applications.

For example, all the titles in this very blog are forced to title case on publish, which isn't something most CMS systems do. I'd love to see it become more widespread.

We have a live demo where you can try it out. It's also available as part of harper.js, our JavaScript package.

Soon, this will become a part of Harper's native linting capabilities.

Today, we published Harper's 1.0.0 release. It's a huge milestone, and in this post I'd like to discuss why it took so long, why we're doing it now, and what's next for the project.

A Round of Applause, Please

As I mentioned: This is a huge step for the project. Before I say anything else, I want to take a moment to appreciate the collective work of our many contributors. It has taken many iterations to get to where we are today, and it wouldn't be possible without the community's effort and feedback.

Why Now?

Until today, I've kept Harper in a pre-1.0 state for one reason: I wanted to move fast. More specifically, I wanted the freedom to build and break things as many times as I needed to. I knew that it would take many cycles of building and rebuilding arrive at a robust system that addresses the people's need for a private writing tool. I believe that building something good can often mean taking the bad parts out. Once a project is "1.0", it becomes much harder to remove things. Naturally, that pushed me to hold off on "going 1.0".

Of course, that didn't stop us from getting the software into the hands of users. Today, tens of thousands of people benefit from Harper's fast and private grammar checking in Chrome, Obsidian, VS Code, and Neovim (among many others). We've racked up hundreds of thousands of downloads before ever slapping anything other than zero before the first decimal point of our version number.

I'm sure your thinking: "Elijah! Get to the point!" Fine. The reason we're doing this now, rather than earlier or later, is because our priorities have just recently changed.

For one, Harper's API has been rock-solid for a few months now, which means we can safely say that our need to move fast in that area has diminished. The opportunity cost of abiding by a stable API has gone down.

Secondly, I've been hearing progressively more interest from potential contributors and consumers who want to put Harper directly into their own apps or services. They have the desire to help make Harper more widespread, but they can't commit to it unless Harper commits to a stable API. That's what we're doing today.

I can imagine a future where Harper is natively integrated everywhere: learning management systems, document editors, messaging platforms, or even operating systems. But great things take time. Today, we're taking a huge step in setting Harper up for the long-term.

What Do I Need to Know?

As an end-user, not much changes. From here on out, we'll be pushing quality-of-life tweaks and bugfixes at a faster rate, all while improving Harper's capabilities across the board.

As a contributor, the patch review process might get a little bit more strict. We'll be focusing more on improving the quality of our code, rather than the amount. If your PRs are likely to result in a breaking change, expect copious notes and possible delays before we hit "merge".

As an integrator, you win more than anyone. If you'd like to include Harper in your application, let us know and we'll do our best to make it easy for you. Take a look at our versioning policy if that sort of thing gives you peace of mind.

Where Can I Get Further Updates?

That depends on the level of verbosity your interested in. For those who want to know about everything I'm currently working, subscribe to my blog. For everyone else, our patch notes in GitHub should suffice.

Earlier this week, I was looking through a table of user feedback about Harper. I believe that software should be build with the user—not just for the user—so this is a pretty regular ritual for me.

I was delighted. Many of the usual complaints were totally absent from the report. Users were encountering fewer of the make-or-break bugs that would harm their experience. This meant that my recent push to get Harper prepared for a 1.0 release was working. Great news! Yippee!

Unfortunately, something new emerged from beneath the pile of bugs: problems with Harper's visual design.

For quite some time now, I've been focused solely on the practical parts of Quality. I mean bugs and the hard-to-deny usefulness of Harper's service. In that time, I've forgotten that the visual appeal of the software is at least as important. So, when I saw these complaints with Harper's visual design, I knew I needed to do something about it.

I've never claimed to be a designer, but I'm also not one to shy away from a challenge. So, I grabbed a copy of the Laws of UX (thanks Eduardo, for the recommendation) and got to work.

Bring Everything Together

Here's the secret to Harper's design system: it hasn't really existed until now. Each of the integrations, from the Chrome extension, to the website, and even the Obsidian plugin, had their own design system and appearance. Mostly, this was because I didn't care enough about it when first crafting these projects. Things have changed, so I'm going to take my time and do a good job in an attempt to service these user complaints.

Before I could start tweaking CSS styles, I needed a unified color scheme. After some research and time in the color picker, I arrived at this:

From there, I built out a component library based on Flowbite and replaced all the relevant components in Harper's integrations with my own.

Next, I wanted to create a bit more dynamism with our typography. After spending some time in Google Fonts, I decided to use Domine, a serif font for headings and Atkinson Hyperlegible for everything else. Once I combined the duo with the orange-ish yellow from the color palette (which is usually associated with creative brands), I felt confident that I had nailed what I imagined Harper's personality to be.

Finalizing the Draft

I feel that Harper's landing page embodies the design system well, so I've shown it below. It's the first, and often only, thing a user sees, so I put some extra care into it. At the time of writing, it is live.

Before My Changes

After My Changes

Where Do I Go From Here?

As I said, this mini-project was simply a draft. I'll be working with Harper contributors on a more regular basis to continue nailing it down, slowly improving the Quality of the visual design for the entire project. If you have any feedback, thoughts, or ideas for how we can make Harper more user-friendly or visually appealing, let me know!

Earlier this week, I came across a really great post from a product engineer over at PostHog. If you haven't already, I highly recommend reading it. With the title "Collaboration sucks," I think it makes its message clear.

If you had shown this article to the Elijah Potter from a year ago, he would have agreed with most of what it says. I was of the strong opinion that collaboration was fundamentally counterproductive to getting sh*t done. Since then, I've completely changed my tune. While I don't believe collaboration is the secret weapon my professors in school made it out to be, I think it's essential to working on ambitious projects for a long time without burning out. Furthermore, at least for open source projects, I think it's critical for prioritization of work.

First, let's get something out of the way. Although the post title declared that collaboration in any form was bad, the meat and potatoes of the post have a slightly different tune. He merely said that it isn't that collaboration is bad, just that too much collaboration is bad. I can get behind this, but I still think there's more nuance lurking in the shadows.

I think Cook's post outlines many of the various ways collaboration can go wrong. I particularly enjoy the red flag examples he provides. I'd like to provide some examples of collaboration going right.

Since all of this is strictly related to Harper, it's possible that both these examples are only helpful to open source projects.

It Boosts Motivation

When I go a little bit too long without engaging with a member of the Harper user base, I start to feel a bit deflated. I start to forget who it is all for.

As a part of my own writing environment, Harper exists to make writing more fun and more human. I can think critically about my message while spending less time worrying about grammar or punctuation. For others, it poses to do all that without violating their privacy. It's great to hear from them, and how we can make our service even better.

I know I'm doing something right when I'm acting as customer support and read a message like this. I feel empowered. Not only do I know what I am doing right, I feel highly motivated to keep doing that thing.

In Open Source, Your Collaborators Are Also Your Users

When working on open source software, especially one that values privacy, clearly stating a policy of being friendly to pull requests is obviously critical. For one, it makes it clear that bug fixes are welcome and will be swiftly reviewed. Less obvious is the feedback residing within the pull request.

Here's an example. A user encountered an issue resulting from a change in Obsidian's API. Not only did he take the time to report the problem, he also went through the effort of implementing a fix. What does this tell us?

For one, it gives us an idea of where our documentation is sufficient. He was able to find the information needed to build the code and run it on his own machine. In the future, we don't need to prioritize the Obsidian plugin's documentation.

More importantly, however, is the fact that the feature was valuable enough for this user to spend time fixing it. If it is valuable for him, we can safely say that it is valuable for other users. Now that I have sufficient evidence, I might consider bringing similar functionality to other platforms, like our Chrome extension.

Collaborating in an open source context can be hugely informative, since your collaborators are also your users. In my experience, this is surprisingly uncommon, even in companies that claim to have a culture of feedback.

There Is Still Such a Thing As Too Much

While I've found these particular cases to be productivity-boosters, I understand that focusing too much on any one thing can be detrimental. First and foremost, the goal should be to ship as high a quality of software as possible, as soon as possible.

If you maintain or work on a project in any compiled language, particularly a language that is known for having a slow compiler, I believe it is critical to regularly schedule time to do an analysis of your build process and make adjustments if necessary.

Compile-time increases slowly as a project morphs and grows. Slow enough that it often goes unnoticed. I call this "compiler creep" and believe it to be similar in many ways to lifestyle creep. By scheduling time to do a full accounting of where your CPU cycles are going, you can preemptively save yourself (and your contributors) a lot of time.

Yesterday was the big day for me. Over the last few months, Harper's compile time from scratch in GitHub actions had ballooned to a five full minutes. On my personal laptop, it had grown to three minutes. That may not sound like a lot, but I often need to build from scratch when swapping between branches or checking out a pull request. Incremental build times had also grown to an unjustifiable duration. Iterating upon unit tests was starting to become quite slow.

Harper's low latency is one of the reasons many turn to it over the alternatives. In the early days, we accomplished that by cranking the LLVM optimizer to the max, reducing the number of codegen units to one and using link-time optimization to take care of the rest. At the time, I believe this resulted in a ~30% performance uplift. Not bad!

For the uninitiated, a codegen unit is the smallest unit of work that is passed to the LLVM compiler. If you allow the project to be split into more than one codegen unit, you can allow more than one compiler to run at a time. This, therefore, increases the number of physical cores that can be used at any given moment during compilation. The downside is that the compiler misses out on optimization opportunities at the boundaries between codegen units.

So when I sat down yesterday, I was hardly surprised to see that code generation was the bottleneck. The compiler was spending very little time type-checking and linking, and a lot of time generating and optimizing machine code.

My initial thought was to reduce the amount of code that needed optimizing. So I did what anyone else would do and went searching for unnecessary dependencies and other kinds of dead code in the repository. I found some, but not enough to make a meaningful dent in the compile time.

That's when I took a look at our build configuration and asked myself: "Do we still need LTO? Could we increase our codegen units?"

In the last few months, I've made some significant improvements to caching and pipelining, which means the bottleneck has shifted from compile-time optimization to memory bandwidth. We simply don't need the compiler to do as much work anymore. So, I tried disabling LTO and reverting the codegen units to their default value. What happened?

Clean builds went from five minutes on CI to less than two. All in, that's an improvement of 71%. On top of that, since it was the code generation stage that was slowing down incremental builds as well, we saw a similar improvement to test execution time. That means there is now less time between making a change in the code, and knowing how it affects our existing test cases. Since that's a huge portion of a contributor's time, improvements to incremental builds can be force-multipliers for the entire project.

I was surprised such a simple change would have such an outsized impact on the developer experience. It felt too good to be true, so I re-ran our benchmarks to see if the change would have a negative impact on Harper's actual performance. Our uncached lint times took three percent longer than before. I think that's acceptable.

Is there a lesson to be learned here? I think so. I implore you to take a half-hour this week to mess around with your compiler's settings. Even an improvement of a few seconds per iteration can compound into hours of time savings, both for you and your fellow contributors. If you think your compiler settings are already perfect, do it anyway. Your code or requirements may have changed.

This is great step in my recent work to make people's first contributions to Harper as frictionless as possible.

As the maintainer of Harper, I read through dozens of issues and pull requests per day and countless more per week. There is a constant flood of new problems to fix and new code to review. At times, it can get overwhelming.

I've been working recently to reduce the amount of noise I encounter when determining what to work on. I'm sharing my findings here to help other novice maintainers handle the torrent. Hopefully, there are some ideas here that you find insightful.

When your open source project gets big enough, you inevitably hit a point where the flow of bug reports or feature requests expands beyond what you're individually able to fix or implement. If you're like Jeff Geerling, this might happen when you find yourself maintaining a larger number of projects. Either way, it's clear that there will come a time where you need to start making hard decisions on how to spend your time.

Set A Policy

Jeff's solution, which I've adopted, is to set well-defined policies for which issues are allowed to slip through the cracks.

The strategy is simple in essence. The maintainer, Geerling or I in this case, directly addresses issues and PRs which are important to the existing functionality of the software first. New features or minor bugs come second. Feature requests that help a single person or a bug reports that address a rare edge case are of lowest priority.

If an issue or pull request cannot be addressed within 30 days of its opening, it is marked stale. If another 30 days pass before it becomes active again, it is closed. Over time, unimportant topics are cleared away, leaving only the ones critical to the project's mission.

A policy like this sets "no" as the default and helps prevent runaway code quality declines. I instituted it on Monday. If it needs revision, I'm open to comments and feedback.

Harper's False Positives

This policy stands in the face of an almost overwhelming number of issues related to false-positives in Harper's algorithm. Here's a representative example.

Since these false-positive reports represent such a significant plurality of the issues on GitHub, I feel the need to handle them separately from everything else. They're usually resolved quickly but left open out of neglect. They're trivial enough for me to fix, but not for me to follow-up. Moving forward, if anyone needs to report an issue related to a false-positive, please do so here.

Develop a Sense of Importance

There is no algorithm to determine which issues or PRs are the most important to a project's success. Typically, I'll read through five or so, and pick the one that "feels" the most important to work on. It's often a balance of prioritizing time-sensitive items, items that could result in publicity, and usefulness to my personal work. It's hard to put it into words, which is why it must be developed, not taught.

What Do You Think?

I consider myself a novice maintainer, so I'd love to hear from more mature maintainers. How do you find signal through the noise?

Additional Reading

• Why I close PRs - Jeff Geerling

For as long as I can remember, I've told people that the real challenge of software engineering isn't writing code. It isn't documentation or CI pipelines either. No, the biggest challenge in software engineering is managing complexity.

Think about it. The typical software engineer is a single person on a vast team, writing and editing a system composed of hundreds, millions, or even billions of lines of code. I find that intimidating. How in the world are we supposed to build robust and reliable systems when facing such complex odds?

I worry that, in my fascination in the novel applications of machine learning we hear about on a day-to-day basis, I've gotten distracted from the beast of complexity. I've allowed the Harper codebase to accrue tech debt.

I recently watched DHH's keynote from Rails World 2025. While I definitely do not agree with everything he has to say, I believe he makes some poignant points about complexity. Some things, he says, which we consider normal today would have been ludicrous ten years ago. Somehow, with computer hardware faster than ever, we've arrived at a point where 20 minute CI runs are acceptable. Where a deploy to production can take an hour and setting up a new environment can take days.

I want to change that—at least, for Harper.

While working on several smaller projects, I've been cutting the fat out of the Harper monorepo. That means removing unnecessary build steps, parallelizing individual CI jobs, and removing optimization flags that don't work.

Harper isn't at the 2005 standard yet, but I want contributing to be as easy as possible. Ideally, it shouldn't take more than five minutes to go from cloning the repo to committing code. That starts with simplifying the environment.

Addendum

I just came across this other blog post where they discuss achieving three times the performance for a quarter of the price by switching from a complex AWS setup to a bare metal Hetzner server. Another win for simpler systems!

The biggest problem with AI programming today is not what I expected a few years ago. I truly didn't believe they would be able to do any economically valuable work.

For web development, modern programming tools like Codex are quite good. They are able to understand a query and identify the relevant areas in a codebase. Then, they can make modifications and iterate on the solution until it fits the prompt. Frankly, I never expected them to reach this point.

No, the prevailing problem I've encountered with these tools is not their ability to do work. Rather, the problem comes from their inability to ask questions up-front and iterate on their mental model before starting to code.

Here's what I mean.

Some Context

Harper's landing page is pretty important. It sets expectations for potential users with a working in-browser demo and reduces unproductive traffic to our issue-tracker by answering common questions. In the past, improving the capital-"Q" Quality of our landing page has had a significant impact on Harper's growth rate.

What Did I Do?

Earlier this week, I wanted to make some significant changes to parts of this landing page. Since the complexity of this part of the code-base is relatively small, I wanted to try something: a new (to me) way to work with OpenAI's Codex.

Most of the marketing materials surrounding these tools imply that having a conversation with the "agent" is the intended workflow. Sam Altman et al. make it seem like I'm supposed to boot up Codex and have a discussion. At the end of that discussion, the thing in my head is down on paper (or code).

I wanted to try something new. Instead of a conversation, I wrote a detailed goal document. I usually wouldn't do such a thing for such a trivial task, but as I said: this was an experiment.

To parallel what real project doc would look like, I included what, why, and how the changes I wanted should be made. Notably, I tried to anticipate any possible revisions or misinterpretations the agent might make, which I would later have to have a conversation about. When I was done, I gave it to the agent and went to work on something else. I came back an hour later, and it was done.

What's the Big Deal?

Imagine this situation. It may sound familiar to you.

You sit down to experiment with an LLM. You want to see what all hype around vibe-coding looks like. You tell the agent what you want. You wait a few minutes for it to do its thing. The result doesn't look right, so you prompt it once more. You wait some more. The cycle continues.

This is extraordinarily inefficient. You're cosplaying as a micromanaging boss.

If I had done used the conversational micro-managing strategy when working on the landing page, it probably would have taken just as long (about an hour). The downside, is that I don't get to go work on something else in the meantime.

By writing a planning document for your agent, you can give yourself room to breath, saving time and being more productive as a result. It's a lot like baking. You put the ingredients together, put them in the oven, then walk away and something else while it cooks.

TL;DR: Get a good text editor, and get good at using it. Keep a notepad by your side. Find a quiet place to write.

Writing is one of life's greater joys. It's a mental workout that often brings me a level of clarity that is hard to find elsewhere. I have never sat down to write for an extended period of time without coming away with a greater understanding of myself and the universe.

I imagine that sounds pretty cheesy.

I just want to impress upon you the importance writing has had in my life. As with anything this important, I have an instinctual urge to optimize it. As software engineers, isn't that part of our modus operandi? To spend an unnecessary amount of time optimizing things?

Well, that's exactly what I've done. Over the years, I've spend a unruly amount of time tweaking and changing the tools I use to write. I'd love to talk about 'em.

Neovim

In the past few years, Neovim has the talk of the town. Its ancestor, vi was borne from the observation that most writing is re-writing, and thus it includes a vast array of specific tools for editing text, rather than writing it. r/neovim has swelled to more than 110 thousand weekly visitors and thousands of plugins and LSPs to choose from.

Unless you choose a pre-built distro like LazyVim, Neovim does very little out of the box. It is through relentless customization that it becomes something truly powerful. Beside the various keybindings in my config, the important plugins and tools that I use are:

Harper

As I'm the chief maintainer, this one should be of no surprise. Don't worry, I won't give you the marketing spiel. I'll simply say that Harper is a grammar checker that happens to work exceptionally well in Neovim.

Zen Mode

Plain Neovim organizes each file to cover the maximum amount of space on-screen. Practically, this means most of my text ends up on the left side, which can hurt my neck. Zen mode restricts the width of the file and centers it on the screen.

Tatum

Tatum is a small tool I wrote in a few hours that lets me live-preview my Markdown as HTML. I won't repeat myself. If you're curious about Tatum, check out my previous blog post.

More Details

If you're thirsty for all the details of my Neovim setup, take a quick look at the configuration code on GitHub.

ReMarkable 2

A few years ago, while I was still in school, I splurged and purchased a used Remarkable 2. Even now, there is rarely a day that goes by where I don't use it. Sure, everything it does could be easily approximated by paper. On the other hand, the thin device saves my back whenever I have to travel and there's a certain joy that comes from using an e-ink device like this.

I mostly use it to read digital books and blog posts, but it's also come in handy for getting around writer's block. Being forced to slow down and use my hand to write, rather than a keyboard, is quite helpful in some inexplicable way.

A Quiet Space

More important than any text editor, at least for me, is a quiet space.

As I said before, writing can take real cognitive effort. A space with distracting noise can make it unnecessarily hard to get words on paper. I've found local libraries or co-working spaces to be great places for quiet authorship.

Whatever you do, do not listen to music with words in it. Even if it feels more productive, I retroactively find that it makes me less so.

Conclusion

There's probably some more important points I'm leaving out, but I definitely believe I've hit the big three. For me, the most important parts of a good writing environment, as a software engineer, are my text editor, a notepad, and a quiet space.

SaaS products are all the rage these days. Historically, Harper has positioned itself as local-first, which many often view as anti-SaaS. That is not the case.

I describe Harper as "local-first" because it is principally meant to be run locally, meaning that your data doesn't leave your device. Privacy is one of our core beliefs, and being "local-first" is part of how we honor that.

Given Harper's position on privacy, discussion of a centralized service (running on a server) might seem strange. Today, I want to explain why a centralized service is necessary, how we intend to tackle it as part of the open source project, and how it's existence will not stand to affect our users' privacy in any way.

Why We Need It Now

User's despise false-positives in Harper's grammar engine.

We'll often make a modification, test it as much as possible internally, and release it, only for a user to discover false-positives on their college essay. We've significantly reduced their frequency through techniques like LLM-Assisted Fuzzing and artificial selection, but they persist.

Far and wide, most of the time between when we get a bug report and when we get a fix into production is spent doing meaningless "paperwork". Reading through issues, writing boilerplate code for tests, and preparing pull requests take up a lot of time. I want to upend this entire loop and find a simpler way to iterate on false-positives.

The first step in this process is reducing the overhead for users to report them in the first place. Instead of filing an issue, I'd love for user's to be able to push a button have have all relevant context sent directly to our inboxes. Not GitHub account required, no fuss.

Obviously, we won't be sending any information without clearly disclosing the precise data being sent from the client.

Since these reports will be in a structured format, we can bundle fixes into larger batches, reducing overhead (and potentially making things easier for LLMs). The Harper project's ability to continuously and surgically improve our service is part of what sets us apart from our competitors. It's rare for user feedback to have such a direct impact on Grammarly's core engine.

This is just the start. Once we've proven that we can maintain such a system for our users, we will move on to more ambitious goals.

How We Plan to Make It Happen

I've had success with these technologies in the past, and believe they'll work quite well for our initial use-case, and scale quite nicely as the service becomes more capable.

SvelteKit

Harper's website is already built using SvelteKit. It works. Why mess with success?

Database: SQLite with Drizzle

On modern hardware, SQLite can handle tens of thousands of transactions per second. Since Harper (as a rule) will place little load on the service per-user, I expect SQLite can service our needs for a long time. When that time expires, Drizzle (an ORM) makes it trivial to move to a more robust database down the road (I imagine PostgreSQL).

For the moment, iteration speed is paramount. I want to move fast. To that end, James and I have discussed hosting everything on a small VPS, and we agree it's the best choice.

Auth.JS

Authentication is not something anyone should roll on their own. Auth.JS is a proven solution that bundles well with Drizzle to building out user profiles, authentication pages, and everything in-between. The needs of the project won't necessitate this right away. When the time comes, using Auth.JS (from my experience) will shorten the auth problem down from weeks to just hours.

The Future

I'm thrilled that we're finally moving in this direction. Being local-only has taken us far, but loosening the reigns to be more simply "local-first" will allow us to tackle more interesting problems and provide more value to our users. I can't wait to get started.

The most common complaint I’ve been hearing as of late relates to Harper’s performance. I’ve been told that Harper has gotten slow.

I was puzzled. Nothing had substantially changed in our core engine that could have made it slower. Our benchmarks hadn’t shown any slow-downs. What could be prompting users to complain about performance?

The problem was clearly in the Chrome extension. Those who complained were doing so with screenshots from Notion or Ghost. Could recent developments in the WebAssembly ecosystem be the cause? Nope. Could it have something to do with the kinds of applications users were trying to use with Harper? Nope. The complaints seemed untethered to a specific domain or program. What could be slowing our users’ down?

The Problem

As it turns out, the problem wasn’t the actual grammar checking, it was rendering the highlights on the page.

The Chrome extension’s hot path looks something like this:

1. Extract the text from the page
2. Run harper-core (the grammar engine) over it to find mistakes
3. Use DOM APIs to compute the bounding boxes of those mistakes on the page. This is important: This computation requires the browser to reflow the layout of the page.
4. Render additional DOM elements to the screen over these bounding boxes. This also requires the browser to reflow the layout of the page.

Requiring the browser to lay out the page more than once per frame is called “layout thrash”, and it’s considered bad practice for performance reasons. It causes more CPU usage than otherwise, which can slow down complex websites and reduce battery life. It isn’t noticeable for newer computers (4 years old), but it can be quite jarring for older laptops.

The Solution

Fortunately, there’s a solution. Just recently, W3C standardized The Custom Highlight API, which allows us to render highlights without interacting with the DOM at all. This saves us two full layout refreshes and allows the browser to offload more work to the GPU, saving CPU cycles and battery life.

It’s not all roses and rainbows. For one, we can only use this optimization for text editors that use contenteditable elements. It doesn’t help for <textarea /> or <input /> elements. Firefox also doesn’t support this part of the API, which means we have to fall back to the old highlighted renderer on Gecko-based browsers.

There are also some minor differences between how the new renderer and the old renderer style their highlights. This is because highlights using the Custom Highlight API can’t be styled using any old CSS, but rather a subset.

The initial goal of any greenfield project of mine is to build a working demo. If possible, that demo should run on the web, so I can distribute to friends and family.

One of my first posts to hit the front-page of Hacker News represented exactly this philosophy. At the time, I was in high-school, taking an independent linear algebra class. I wanted to demonstrate exactly how the things I was learning could be applied to real-world problems. My blog post, "Markov Chains are the Original Language Models" included an interactive (small) language model that readers could use to generate meaningless text. I gave them the ability to toy around with the training data to see how it could affect the final model's generations.

I ended up getting an "A" in the class and a couple thousand hits online. More importantly, I had proven the utility of a demo. Since then, every major project of mine has had some kind of demo. Some of them better than others.

The Harper demo, in particular, was rough to start. It was enough to prove the concept and establish the project within a certain ecosystem, but it far from perfect. It had bugs and, until now, hasn't evolved with the project at all.

For all my projects though, I see the initial demo as critical. Beyond proving the concept, the demo gives me an enormous amount of motivation. As an engineer, there are few things more thrilling than to watch someone toy around with the product of your work. When there's a live version of your product available, any incremental improvement comes with a hit of dopamine.

I had a great discussion with Adam Zielinski a few weeks ago, when he encouraged me to make the demo on the Harper website as close to the actual Harper experience as possible. Last week, I did exactly that.

Actually, I did more than that. Today, the demo isn't just similar to the experience of using our Chrome extension, it is exactly the same. The DOM-inspecting and manipulating technology found in the Chrome extension can now be embedded in any page. That includes the actual Harper website, and any other site you might want.

For now, I'm calling this the lint-framework, and it opens the doors to a good many other integrations (should we decide to build them). It is essentially a portable package that includes pretty much everything you need to provide Harper to any browser-based text editor.

Many applications allow users to build plugins into their WebView, but don't provide the full web extension API (Thunderbird, for example). In these cases, we can pull in the lint-framework and wire up the native plugin APIs to build configuration pages. It isn't a complete solution, but it cuts down the work and maintenance overhead enormously.

As it turns out, the cruft of Harper's demo was a major barrier to adoption. Since polishing it up late last week, we're already seeing metrics like bounce rate move in a positive direction.

Today marks the end of my first week in New York City. It is an exceptionally active place. Neither the hustle nor the bustle ever stops. Sirens sound at all hours of the day and you're constantly smelling entirely new things. I've been told New York is an acquired taste. I can see why they might say that.

Back home, it's rare I would walk down the street and overhear a random founder talk about their latest Y Combinator funding round. Here, it's apparently common enough to occur at least once a week. Everywhere I look, I find the headquarters of multinational companies and the offices of major governmental organizations.

The City is Practical

If I were to describe Portland, Oregon (which I visited a few weeks ago to attend WordCamp US) in one word, it would be optimistic. If I were to describe New York in one word, based on my incredibly limited experience thus far, it would be practical.

The city and the people who reside inside take very little fluff. They say things as they are, in the kindest way possible. If they don't think your proposed route to a restaurant will work, they stop you before you finish explaining and put you on the right track. It saves everyone time and energy.

This is something I personally need to practice more often in my work. It is common for me to avoid productive conversations or code reviews, only because they might evoke a negative emotion in the short-term. These same conversations usually produce positive emotions in the long term, so it is in my interest and in my colleague's interest to have them. NYC is slowly training a habit to recognize these situations more often and make the right decision.

I don't think I'm alone here. I think a plurality of people in the modern world could benefit from a bolder mindset that takes on tough conversations, rather than shying away.

We're Not Done Yet

I'll be in New York for another week. If all goes well, I'll have more opportunities to learn from the other members of the AI group. It's been great to hang out with James and the gang. The culture here is incredibly inviting. I am immensely grateful to be here.

Several months ago, I announced the initial version of the Harper WordPress plugin. In the interest of those still using it, I’d like to give y’all an update.

The Chrome extension is now the recommended way to use Harper in WordPress. There are several compounding reasons for this. Most importantly, iterating on a Chrome extension is significantly faster than the review process for WordPress.com plugins. In fact, the current WordPress.com Harper plugin, which you may already be using in your P2s, is currently several months out-of-date. While you may continue to use the WordPress plugin, it will no longer receive regular updates.

If you choose to switch to the Chrome extension, I think you’ll find the experience more holistic, since it is able to persist user data across sites.

I had the incredible opportunity to speak at WordCamp U.S. at Matt's suggestion. I met a number of wonderful, kind, and inviting people. It was my first WordCamp and first conference talk, so I was naturally nervous. The community made it feel easy.

I spoke about how we can use machine learning to improve the quality of our applications without big capital expenditures. In fact, I argue that capital expenditures are often a step in the wrong direction.

Fun fact: This talk started out as a blog post.

Whenever I can, I try to stop and reflect. Often it's about my work and the things I can do to continue being more productive. But often, I reflect about the recent books I have read and the things I (and you) can learn from them. That's what this post will hopefully turn out to be.

So, without further ado, here are some of the books I have read recently.

The Stoic Challenge by William B. Irvine

At the end of each episode of his podcast, Ezra Klein asks his guests the same question: what three books would you recommend we read? Terminating one particular episode, author Jonathan Haidt suggests The Stoic Challenge.

I was already aware of William B. Irvine as a prolific stoic writer. In fact, I had tried to read another of his books on the subject of Stoic philosophy. I had not finished it because, well, I found it somewhat dry.

The Stoic Challenge, on the other hand, was a delight. It feels like Irvine took all the successful parts of the previous book and condensed them down into something my Gen-Z brain could understand.

The book's more important takeaway is simple. In life, you and I both encounter daily struggles that impede our progress toward our goals.

Our flight is running late, so we get to our hotel later than we intend. We forget to pack a lunch, so we must get through our workday hungry. Our babysitter wants a raise, which means we won't be able to go out as often.

The Stoic Challenge encourages us to frame these setbacks as a test. A test put upon us by some hidden force. To pass the test, we need to act in a rational way, and take steps to avoid exacerbating the resulting negative emotions.

I highly suggest anyone read this book. I felt engaged the entire time reading it, and its lessons apply to any walk of life.

Slow Productivity by Cal Newport

Cal Newport is known for many of the most groundbreaking productivity books of the last decade. Personally, Deep Work and How to Be a Straight-A Student have been particularly impactful. They've encouraged me to cut out the unnecessarily complex parts of my working life, and work to find ways of doing the same for others.

His latest work continues from the same ideas he put forth in those books.

How to be a Straight-A Student was focused on the idea that the quantity of high-quality, published work made the difference between mediocre and stellar academic careers. Following on, Deep Work made the case that the best way to achieve high output was by creating circumstances that allowed for extended periods of intense focus.

Slow Productivity takes both of those ideas a step further. He suggests that the key to high-quality output, in addition to deep focus, requires time. He believe that truly amazing products require the brain to be relaxed, comfortable, and unhurried. It is only then, that it can work outside of the box.

I'm not entirely sure I agree, but the book made for a stimulating read nonetheless.

The Humans by Matt Haig

The Humans is a beautiful book that celebrates all the idiosyncratic things that make humans special. Don't worry, I won't spoil it for you I've read a decent number of Haig's books in the past, and I've enjoyed each one. This was no different.

I'll spare you the details.

The 22 Immutable Laws of Marketing

I am an engineer. I went to an engineering school. Since I spend most of my time around people who work in fields related to engineering, I do not spend much time around people who know a lot about marketing. I wanted to fix that. Which is why I read The 22 Immutable Laws of Marketing.

This is a book intended for people who want to understand the axioms of how brands are perceived by the public.

Sapiens by Yuval Noah Harari

Sapiens is a comprehensive history of our species, homo sapiens, beginning with the first humans to walk the planet all the way up to the humans of today. Harari takes a unique look at what makes our species special, and the things that make our future uncertain.

At times, the writing can be a bit dry and repetitive, but on the whole it is a well-written book. I would certainly recommend it for anyone who seeks an entirely new perspective on the short history of human life.

Your Utopia by Bora Chung

I went into Your Utopia expecting a collection of short stories similar to those by Ted Chiang. That is not what I received.

Instead, Chung provides a somewhat eclectic collection of stories that feel not unlike a fever dream. Rather than genuine, hard science fiction, Chung tells the tales of sentient cars, elevators that care for the elderly, and immortal secretaries.

I don't regret reading it, but I also don't recommend it to fans of Chiang.

I want you to read that title as literally as possible. Harper is now capable of evolution.

This past week, I've been working on a system that should allow us to handle more complex grammatical cases and contexts, faster. I believe it will improve our ability to add new grammatical rules to Harper by somewhere between 500% and 1,000%.

To top it off, this system does it without slowing Harper itself down or increasing the memory footprint.

Let's get into it.

The Problem

There are several unique methodologies at play when Harper goes about grammar checking. Which strategy depends on the grammatical rule in question. Today, we're interested in expression rules.

For the curious, I have recently written a reflection on expression rules, as well as a guide for anyone interested in producing them. This post, however, will not recount information I've already written on this blog.

By count, expression rules make up the majority of grammatical rules Harper is currently capable of detecting. This is because they are fast, easy to write, and most importantly, easy to review.

There are, however, occasional hiccups that I encounter when tackling a problem. The English language is tricky and often it contradicts itself. I will often try to write a rule which covers a certain case, only to find that it doesn't cover all cases. I can iterate, but it often becomes tedious and time-consuming.

The Solution

Last week, I threw in the towel. I was tired of iterating ceaselessly towards a goal, only to have a new one to tackle after that. So I decided I would let the computer iterate for me.

Harper's expressions are essentially small programs which are able to identify the locations of given patterns in natural language. They are constructed at runtime, but they run exceedingly fast because they tend to be amenable to modern branch prediction. We can use this fact to our advantage.

When generating an expression that detects a particular grammatical rule, the new system (which I've called The Ripper) follows three steps.

1. Generate N random Harper expressions
2. Score the performance of these expressions by testing them against a curated dataset. The dataset contains labeled rows of sentences that do and do not contain the grammatical rule of interest.
3. Take the best K expressions and mutate them to left with L new "child" expressions. Go to step 2.

That's it! We're essentially treating expressions as living creatures and subjecting them to artificial selection. It works remarkably well.

Since these datasets are handcrafted (or generated by an LLM), they don't need to be large. Plus, the expressions themselves are quite fast to generate and test, so we can do so at an exceptional rate.

My laptop is able to churn through about 90 thousand candidates per second, allowing us to converge on an acceptable result in just a few minutes. Given more time, it's able to produce an expression rule that is more accurate than what I could write myself.

What's Next?

I intend to spend some time optimizing the process, particular for the human element. I'd like to be able to create batches of these datasets and let The Ripper take care of them all at once, overnight or on a beefy server in the cloud.

I'd also like to set up automated workflows for piping data from an LLM directly into the Ripper. Ideally, I want this system to get to a point where I can feed information from a style guide into an LLM and get a guaranteed functioning Harper expression rule out of it.

I've given dozens of talks over the years, yet I still feel like I have much to learn about public speaking. The act of presenting an idea to a large audience in person is intimidating for me, as I imagine it is for most people.

Preparing for my upcoming talk at WordCamp US has me stopping to reflect on what I already know. To recall the basic principles that turn a good talk into great one.

Conveying an idea effectively is a powerful skill. It can help get people on your side and direct them to your point of view. Humans are beings that thrive when they build and maintain strong relationships with each other. Good communication supercharges that process.

Here are my bold ideas for how to present ideas well, in no particular order.

Convey Exactly One Idea At a Time

A common problem people run into when they draft a presentation is not a lack of ideas: It is that they have too many.

People adore talking about themselves or their ideas. A natural consequence is that people have a predilection to stuff a presentation with more information than they need to make their point. Make sure to allocate time to take stuff out.

The more streamlined and clear you make your point, the more receptive your audience will be.

Use a Story

The human brain is wired for stories. We've been telling them for thousands of years. People remember them far better than they do arbitrary information. In fact, that is exactly how communities used to do recordkeeping: by telling stories.

By weaving a tale into your presentation, you can prime the audience to receive your idea, and make that idea more memorable.

Explain the "Why"

The "Why" should be the first thing your audience hears from you. Why they should listen. Why your idea is important. Why your input can help them in their professional or personal lives.

Chris Anderson, the Head of TED, says, "Style without substance is awful".

In other words, you can have the fanciest, most visually appealing slide deck and all the confidence in the world, and you can still have a mediocre presentation. Find a topic or idea that you genuinely care about, and explain why the audience should care about it too.

The biggest complaint about Harper is that the quality of grammar checking still needs some work. I've got an exciting new project I'm preparing to discuss later this week (hopefully) in that area. It should be a step-function improvement over our previous methods for creating new rules.

In the meantime, I've collected some recent sightings of Harper in the wild:

• Obsidian Rocks
• We Are Founders
• It's FOSS
• XDA Developers
• Make Tech Easier

Grammar checking can be cumbersome, especially when its sloth gets in the way of your thinking. That’s part of why so many prefer Harper over Grammarly or LanguageTool. Waiting four seconds per edit can add up over long days of writing.

But there’s something else that can slow people down, especially power users. It’s the mouse. Moving one’s hand to the pointing device, moving it, and clicking can take a while—and that adds up too.

Neovim (and similar text editors like Obsidian) solve this problem by using another input device: the keyboard. Since you’re using the keyboard to write words to the page, why not also use it to correct grammatical mistakes?

In the latest release of Harper’s Chrome extension, that’s exactly what I’ve added. While writing, you’re bound to make (at the very least) a few typos. If that’s the case, you can now double-tap the shift key to open the suggestion popup from the error closest to your caret. From there, you can use tab and enter to select and apply a correction.

I’ve personally found that this single-handedly speeds up my writing process by at least 20%. I’m able to focus less on each word and more on the content.

I was recently encouraged to submit a talk proposal to the AI track at WordCamp US. I was more than happy to oblige, and to my surprise, I got in. See description below for details. I hope to see some of y'all there!

Description

Machine learning has been around for decades. Until recently, running ML models typically required big centralized servers due to high computational demands. With modern hardware and tools like Rust and Burn, we can flip the script. When run on a user's device, trained models can be an incredible boost to human creativity and expression.

Harper is a grammar checker that is designed to improve your work without degrading your integrity—or the writing experience itself. Our users generate and publish novel ideas faster because we focus on what's important: speed, privacy, and accuracy.

WordPress' mission is to democratize publishing. This talk will explore how we can use AI to democratize writing too. More importantly, how you can train and deploy specialized models at the edge to eliminate inference costs, lower latency, and improve user satisfaction.

Each week I sit down and reflect on what I've learned. I think about which of my efforts have helped the Harper endeavor, and which have harmed it. It's not easy, but I think it's one of the best habits I've built. More accurately, it's one of the best habits Matt has encouraged me to build.

When I really think about it, I'm reminded of the original After the Deadline blog, which coincidentally also had a category called "Talking to Myself". I wonder if Raphael Mudge (AtD's author) had a similar motivation.

I talk to myself to find the weaknesses in Harper's mission and focus my attention on what matters. It is an efficient way to get from layer one to layer five. It can help get other maintainers on the same page and build a body of knowledge that can be linked to from anywhere.

Not worrying about keeping information locked-down is one of the more beautiful parts of working in public. Open source code would be nothing without similarly open source knowledge. I talk to myself online so my users can understand my intentions.

In many senses, we are not unlike moths. We are drawn to the lights of distraction, hype, and procrastination. For me, setting up checkpoints to stop and reflect has been an effective deterrent from these lights. In fact, I think each minute for which I stop and think has paid itself back three times over in time savings. If I can better allocate my time and energy, I can avoid doing redundant work pursuing pointless goals.

Bonus Points

A few of my articles have become popular on sites like Hacker News and Reddit. We've gotten quite a few Harper installations from links on these articles. I would never rely on this kind of blogging as a source of advertising, but it is nice to see it help spread the word.

Just like grammar itself, Harper is rule-based. These rules can be written by human or machine, and usually take the form of a "phrase correction" or an expression rule. You can read more about both in my guide.

In essence, an expression rule is a small, declarative program that seeks out specific cases Harper can be certain are grammatically incorrect. For example, we know redundant modal verbs are always wrong:

We could give it try attempt.

What's Great About Expression Rules

As the maintainer of the project, I need to read through a decent number of pull requests each week. Depending on their complexity, this can be time consuming. Because expressions are almost always written with the same five primitives, they make reviews way faster. For many rules, I only need a quick skim of the code and to confirm they don't create false-positives.

In addition to typos, the primitives underlying expression rules were designed to check agreement and word-ordering errors. To that end, there are a number of assumptions baked in. This makes some kinds of rules harder to write, but also makes all expression rules exceedingly easy to memoize.

What's Not-So-Great About Expression Rules

As I said before, expression rules were designed for a certain kind of error. There are many grammatical or layout rules which hard difficult or downright impossible to define with an expression.

This normally isn't problem, since we can go about them in other ways. On multiple occasions, however, I've seen potential contributors get frustrated at a perceived lack of flexibility. We didn't make it clear enough that there were solutions other than expressions.

How Can We Improve?

Applying Amdahl's law, I think the most effective solution is to improve our documentation, as I have been for the last week or so.

Beyond that, removing the need for boilerplate code and other causes of mental overhead (like registering our rules and waiting for a slow compilation process) might lower the perceived difficulty of writing new rules.

In a previous post, I detailed how I implemented a basic nominal phrase chunker using Transformation-based learning (not to be confused with the transformer class of neural network models). Since then, I've taken another crack at the problem. My main goal: improve the accuracy. The end result is a portable neural network model that achieves ~95% accuracy on grammatically correct text when tasked with identifying the nominal phrases in text.

Our Goal

We want Harper to be able to match against subjects and objects in sentences. This is a prerequisite for checking a diverse array of grammatical rules. For example, to catch the error in this sentence, we need to correctly identify which tokens represent our subject.

Neither of the big blue bottle would be broken by the fall.

In this case, our user has accidentally made the subject singular, while the verb "neither" implies that the subject should be plural. We call this an agreement error. Because our subject, "big blue bottle" contains multiple tokens, we need a way to identify subjects at a higher level than per-token. That is what a chunker does.

The Failures of the Brill Chunker

The Brill chunker was by many accounts a success. It wasn't, however, a success in the main way that mattered: It wasn't reliable enough to be used in Harper's grammatical rule system. While fast and small, it failed to catch most edge-cases in English text. In some senses, it overfit its training dataset.

Why Train Our Own?

As our needs continue to expand alongside our user-base, I need the chunker to be flexible. If its needed capabilities expand, I need to be able to retrain the model to meet them. That would not be possible without having a deep understanding of how the system works.

Building a Neural Net

To build a new chunker, I just needed to implement the Harper Chunker trait. Easy enough.

/// An implementer of this trait is capable of identifying the noun phrases in a provided sentence.
pub trait Chunker {
    /// Iterate over the sentence, identifying the noun phrases contained within.
    /// A token marked `true` is a component of a noun phrase.
    /// A token marked `false` is not.
    fn chunk_sentence(&self, sentence: &[String], tags: &[Option<UPOS>]) -> Vec<bool>;
}

For the nerds in the crowd, I decided to use a Word + POS embedding -> BiLSTM -> Linear architecture. To keep things portable and consistent with the rest of the Harper codebase, I used Burn, a Rust-native machine learning toolkit. While I believe the BiLSTM to be good enough for this application, one advantage of Burn is the ability to easily swap it out for a transformer if the need arises. It also makes it unbelievably easy to quantize models.

This architecture gives us some hyperparameters to tune against. After dozens of training runs of experimentation, these worked best:

What's Next?

Similar to the Brill Chunker, I'll be trying to use this new system in our grammar checker. From there, I'll know what additional information we'd like for it to infer. Once I've gotten it to reliably work for >= 3 rules, I'll declare it ready to merge.

Update as of December 2025

I totally forgot to update this post. I suppose it is better late than never. Back in August I actually ended up giving a talk about our chunker at WordCamp U.S. It was a great experience, and I highly suggest you go watch it if you find this stuff interesting.

I believe that good documentation is more important than ever. In an age where large language models have exceptional context windows, it's easy to fall into the trap of believing a system is simpler than it is. To test the limits of their abilities, I'll often feed Harper code in and ask for their explanation of it's inner workings. They've never gotten it anywhere close to correct.

That's because good documentation is more than the sum of its parts. It should contain all the relevant technical details needed to work on a repository, but it should also provide a sense of quality and direction. It should distill an ability within the reader to tell what good code and conduct looks like.

I think this is an area Harper could do to improve on.

It's Possible to Write Too Much

Too much text can be overwhelming. I have met many people (myself included) who believe that more detail is always better. That as long as the information exists somewhere, someone will be able to find it. In reality, that's just not true.

That brings us to our first rule: good documentation should be simple.

The worst way to describe a complex system is with complex language. It's much better to first break it down into simpler parts. From there, you can build it back up again.

I've found it effective to start by erring on the side of "too much", only to pare it down later.

Needless to say, it's also possible to write too little.

Focus on Why

The code itself is (or should be) the best description of how the software works. Documentation should complement this reality by describing why the software works that way.

This might include a little history. What steps did the original code's author take before arriving on the final solution? A blog is a great way to answer that question.

When describing a bug relevant to some code (like in a pull request), a picture is worth a thousand words. It's much easier to show a problem than to describe it. That's why I highly suggest you take a screenshot of the problem. Even better: record a video recreation of it.

Harper is a grammar checker that relies on concrete, legible grammatical rules. In doing so, we make Harper's inner-workings fundamentally clear, which allows us to guarantee privacy, speed, and most importantly remain impartial.

Writing additional rules is one of the best (and easiest) ways you can contribute to the open source project. Simple rules take just a few minutes and often don't require any understanding of Rust at all—a fact I only cite because it is a common point of concern.

Instead of throwing a wall of text in your face, I'm breaking this "guide" of sorts into three simple sections. You don't need to read all three—in fact, I would recommend against it.

The only thing you need from here is an idea of the grammatical rule you want to add to Harper. Don't have one in mind? Visit our issue board to find a potential rule that piques your interest.

The three paths:

1. A "phrase correction". These are for the simplest grammatical rules. Use one of these in cases where semantic meaning and context aren't important.
2. An ExprLinter. These are for more complex rules. Use one of these in cases where semantic meaning or context are important, and you don't need access to information wider than clause-level. Takes a little bit to learn, but are extremely powerful.
3. A plain Linter. These are often used for rules that involve punctuation. It requires the most Rust knowledge but the least Harper-specific knowledge. I'm going to hold off on writing a guide for these until I hear a real desire to learn about them.

These guides will focus more on the process of writing a rule for Harper, not the technical details of wiring it up. For the latter, see our official documentation.

This is part of a series. Go to the start.

There are several ways to add a grammatical rule to Harper. This post aims to outline the easiest (and most constrained): a "phrase correction".

Sometimes you'll see a simple but common grammatical error that doesn't have much to do with any broader context. For example, I often mistype "in the" as "int he". It happens quite often, and there isn't a broad pattern at play here. This is a perfect candidate for a "phrase correction".

From a technical point of view, a "phrase correction" is just a mapping from one or more phrases to another set of phrases. But they're more than a simple search-and-replace. Under-the-hood, Harper will account for various capitalization and stylistic changes that can be difficult to cover manually. Fortunately, they're pretty easy to add to Harper.

Before we begin, make sure you properly set up your environment.

Open up the Harper monorepo. In harper-core/src/linting/phrase_corrections/mod.rs, you'll find a list of entries that look somewhat like this:

"GildedAge" => (
    ["guilded age"],
    ["Gilded Age"],
    "The period of economic prosperity is called the `Gilded Age`.",
    "If referring to the period of economic prosperity, the correct term is `Gilded Age`."
),
"GoingTo" => (
    // The value to map _from_.
    ["gong to"], 
    // The suggestions to present to the user, which replace the problematic text.
    ["going to"], 
    // The message for the user.
    "Did you mean `going to`?",
    // The rule description to be shown in settings pages.
    "Corrects `gong to` to the intended phrase `going to`."
),
"GotRidOff" => (
    ["got rid off", "got ride of", "got ride off"],
    ["got rid of"],
    "Did you mean `got rid of`?",
    "Ensures `got rid of` is used instead of `got rid off`."
),

Each of these is a "phrase correction". To add one for the problem I outlined above, we just need to append to the end of the list and open a pull request.

"InThe" => (
    ["int he"],
    ["in the"],
    "Did you mean `in the`?",
    "Detects and corrects a spacing error where `in the` is mistakenly written as `int he`. Proper spacing is essential for readability and grammatical correctness in common phrases."
),

If you want to go the extra mile, we'd really appreciate if you added one or two test cases to harper-core/src/linting/phrase_corrections/tests.rs to make sure everything works as expected:

#[test]
fn corrects_int_he() {
    assert_suggestion_result(
        "That pizza stayed int he box.",
        lint_group(),
        "That pizza stayed in the box.",
    );
}

This is part of a series. Go to the start.

Expression rules (or more commonly, ExprLinters) are Harper rules that use declarative expressions to find and fix grammatical errors. They're halfway between a "phrase correction" and manually implementing Linter.

Make sure you properly set up your environment.

Before we get started, let's take a look at the ExprLinter trait. Here's what it looks like at the time of writing this post.

/// A trait that searches for tokens that fulfil [`Expr`]s in a [`Document`].
///
/// Makes use of [`TokenStringExt::iter_chunks`] to avoid matching across sentence or clause
/// boundaries.
#[blanket(derive(Box))]
pub trait ExprLinter: LSend {
    /// A simple getter for the expression you want Harper to search for.
    fn expr(&self) -> &dyn Expr;
    /// If any portions of a [`Document`] match [`Self::expr`], they are passed through [`ExprLinter::match_to_lint`] to be
    /// transformed into a [`Lint`] for editor consumption.
    ///
    /// This function may return `None` to elect _not_ to produce a lint.
    fn match_to_lint(&self, matched_tokens: &[Token], source: &[char]) -> Option<Lint>;
    /// A user-facing description of what kinds of grammatical errors this rule looks for.
    /// It is usually shown in settings menus.
    fn description(&self) -> &str;
}

The structure of the trait reveals some of the behind-the-scenes work Harper is doing for you. There are three phases:

1. You provide Harper an Expr. It will iterate through the document, looking for token sequences that match your expression.
2. Any and all matches are passed to match_to_lint. From there, you can perform optional additional validation to confirm that the tokens really do represent a grammatical error. If so, return None. Otherwise, return a Lint with any suggestions that may fix the problem.
3. Harper will handle everything else. It will show UI, reformat text, and settings menus to the user. It will also perform aggressive caching on the first two steps, so any modifications to the document have a negligible performance impact.

Let's Get Started

Now that we've reviewed the essentials, let's implement an ExprLinter.

Before we can write a single line of code, we need a grammatical rule of interest. I'm going to pay a visit to the Harper issue board.

After looking through a few options, I think #1513 is a good candidate. We are looking for missing prepositions between an adjective and a subject.

To get started, we'll create a file under harper-core/src/linting called missing_preposition.rs and add it to the parent Rust module. I'll paste the template into the file:

pub struct MissingPreposition {
    expr: Box<dyn Expr>,
}

impl Default for MissingPreposition {
    fn default() -> Self {
        let expr = todo!();

        Self {
            expr: Box::new(expr),
        }
    }
}


impl ExprLinter for MissingPreposition {
    fn expr(&self) -> &dyn Expr {
        self.expr.as_ref()
    }

    fn match_to_lint(&self, matched_tokens: &[Token], _source: &[char]) -> Option<Lint> {
        unimplemented!()
    }

    fn description(&self) -> &'static str {
        unimplemented!()
    }
}

I like to start by building out a few test cases before working on the actual code. We get some for free from the GitHub issue:

#[test]
fn fixes_issue_1513() {
    assert_lint_count(
        "The city is famous its beaches.",
        MissingPreposition::default(),
        1,
    );
    assert_lint_count(
        "The students are interested learning.",
        MissingPreposition::default(),
        1,
    );
}

Obviously, these tests will fail if we try to run cargo test, but at this point you should do so anyway to make sure your toolchain is working.

Writing our Expression

The heart of this grammatical rule is the Expr (pronounced expression). There are a number of ways to go about making one of these. The simplest (and most common by far) is to put together a SequenceExpr.

In our case, we're looking for missing prepositions between an adjective and a noun. A good expression to start with could look like:

impl Default for MissingPreposition {
    fn default() -> Self {
        let expr = SequenceExpr::default()
            .then(UPOSSet::new(&[UPOS::ADJ]))
            .t_ws()
            .then(UPOSSet::new(&[UPOS::NOUN, UPOS::PRON, UPOS::PROPN]));

        Self {
            expr: Box::new(expr),
        }
    }
}

We're using a UPOSSet here, which is another kind of Expr that looks for specific parts of speech. The name derives from the Universal Dependencies tag system. Any tokens tagged with any of the options we've provided to the UPOSSet will match.

However, it's easy to create an example that this expression matches against, but doesn't contain a grammatical error. We call this a false positive. Let's write one and add it to our test suite.

#[test]
fn allows_terrible_stuff() {
    assert_no_lints(
        "Either it was terrible stuff or the whiskey distorted things.",
        MissingPreposition::default(),
    );
}

From here, you should use your brain to continuously refine the expression into something that maintains a low false-positive rate while remaining useful. Here's what I settled on:

impl Default for MissingPreposition {
    fn default() -> Self {
        let expr = SequenceExpr::default()
            .then(
                AnchorStart.or(SequenceExpr::default()
                    .then(UPOSSet::new(&[UPOS::DET]))
                    .t_ws()),
            )
            .then(UPOSSet::new(&[UPOS::NOUN, UPOS::PRON, UPOS::PROPN]))
            .t_ws()
            .then(UPOSSet::new(&[UPOS::AUX]))
            .t_ws()
            .then(UPOSSet::new(&[UPOS::ADJ]))
            .t_ws()
            .then(UPOSSet::new(&[UPOS::NOUN, UPOS::PRON, UPOS::PROPN]))
            .then_optional(AnyPattern)
            .then_optional(AnyPattern);

        Self {
            expr: Box::new(expr),
        }
    }
}

Now that we have an effective expression as a base, let's fill out the remaining fields. I found checking for an adposition reduced the false-positive rate, and it was easiest to add it to the match_to_lint function.

impl ExprLinter for MissingPreposition {
    fn expr(&self) -> &dyn Expr {
        self.expr.as_ref()
    }

    fn match_to_lint(&self, matched_tokens: &[Token], _source: &[char]) -> Option<Lint> {
        if matched_tokens.last()?.kind.is_upos(UPOS::ADP) {
            return None;
        }

        Some({
            Lint {
                span: matched_tokens[2..4].span()?,
                lint_kind: LintKind::Miscellaneous,
                suggestions: vec![],
                message: "You may be missing a preposition here.".to_owned(),
                priority: 31,
            }
        })
    }

    fn description(&self) -> &'static str {
        "Locates potentially missing prepositions."
    }
}

That's it! We've written our rule.

Don't forget to register your rule and add some more tests before opening PR. Make sure you take a look at the pull request to see the finished rule.

When I started the Harper project I knew I wanted to be able to use it for the comments in my code. First, because I knew these comments would become part of our official documentation over time, and because I hoped it would encourage me to write more. Over time, this has become one of the most prized features of the software, attracting tens of thousands of developers.

The common problem, however, is that there have always been programming languages that our LSP doesn't support. One of the oldest issues on GitHub is about this.

This post is a guide for adding a new programming language to the Harper language server.

Why isn't it in the official documentation? While the information contained within this guide will remain relevant to the project for a long time, I don't imagine each identifier or file path to remain the same. If you think it would better serve potential contributors to place this guide on the main site, let me know.

Introduction to Tree-sitter

Tree-sitter is fantastic framework for building fault-tolerant language parsers. That means it is still able to parse the majority of a document, even if it contains portions of invalid syntax.

This is important for Harper, since we expect people to use Harper while their programming. It should be OK if some of their code is incorrect, since we only care about their comments.

There are also a wide variety of Tree-sitter parsers available on crates.io, ripe for our consumption. If you want to add a language to Harper, this is the easiest way to do so.

Step 0: Avoid Duplicating Work

You're interested in adding support for a programming language. If that's the case, it's possible other people are too. Make sure no one else has opened a PR or has already merged support for the language you have in mind.

Step 1: Find a Grammar

Look for an existing grammar on crates.io. By convention, they tend to be named tree-sitter-<language>, where <language> is the language you're looking for. For example, tree-sitter-java is for Java and tree-sitter-rust is for Rust.

If you would rather write your own grammar, make sure it is eventually published on crates.io. harper-ls binaries are often consumed from crates.io, which requires that all upstream dependencies come from the same source.

Step 2: Import and Wire In

Harper's comment support lies in the harper-comments crate in the monorepo. Import the grammar's crate into the project with Cargo.

cargo add <CRATE-NAME>

Then, add lines to the relevant functions in harper-comments/src/comment_parser.rs. Make sure you visit the Language Server Protocol Specification to obtain the correct language ID.

pub fn new_from_language_id(
    language_id: &str,
    markdown_options: MarkdownOptions,
) -> Option<Self> {
    let language = match language_id {
        "cmake" => tree_sitter_cmake::LANGUAGE,
        "cpp" => tree_sitter_cpp::LANGUAGE,
        "csharp" => tree_sitter_c_sharp::LANGUAGE,
        "c" => tree_sitter_c::LANGUAGE,
        "dart" => harper_tree_sitter_dart::LANGUAGE,
        "go" => tree_sitter_go::LANGUAGE, // Add a line here

/// Convert a provided path to a corresponding Language Server Protocol file
/// type.
///
/// Note to contributors: try to keep this in sync with
/// [`Self::new_from_language_id`]
fn filename_to_filetype(path: &Path) -> Option<&'static str> {
    Some(match path.extension()?.to_str()? {
        "bash" => "shellscript",
        "c" => "c",
        "cmake" => "cmake",
        "cpp" => "cpp",
        "cs" => "csharp", // Add a line here

Step 3: Testing

To make sure everything behaves correctly, we need to add some integration tests. You'll find all the existing ones under harper-comments/tests/language_support_sources.

Find or write several new files under this directory in the language you've added support for. Add intentional grammatical errors to these file in syntactically relevant places. We want to make sure that Harper can detect the errors we want and will ignore the errors we do not want. For example, we might put an error inside an @param tag in JSDoc. That way we'll know if Harper is not properly ignoring those elements.

Add new entries to the bottom of harper-comments/tests/language_support.rs. The second parameter of the create_test! macro is the number of grammatical errors that Harper should detect in that file.

create_test!(ignore_shebang_3.sh, 0);
create_test!(ignore_shebang_4.sh, 1);
create_test!(common.mill, 1);
create_test!(basic_kotlin.kt, 0); // Add a line here

From there, you can run cargo test to make sure everything passes.

Step 4: Document

To advertise support for the language, there are a couple addition places that need modification. Notably:

• The supported languages table in packages/web/src/routes/docs/integrations/language-server/+page.md
• The GitHub Issue
• The activationEvents key in the VS Code plugin's manifest: packages/vscode-plugin/package.json

Done!

That should be everything. Open a draft pull request while you work and ping me (elijah-potter) if you have any questions.

Additional Resources

• PR for adding Solidity support
• PR for adding Scala support

The title of this post is somewhat misleading. Local-first software rarely needs to be scaled at all.

Harper recently received a massive increase in both traffic and user count. How? By making it to the front page of Hacker News. If you couldn't tell by the extent to which I talk about this, I'm thrilled. This amounted to a tremendous amount of free advertising, if nothing else.

I've been thinking a lot about the wondrous benefits of working at the edge. That's just one term of many that people use to describe Harper: "edge-first". Others include "local-first" or simply "on-device". They are all just jargon for a simple idea: We run all the relevant grammar checking code as physically close to the users as possible. That way, we can serve up suggestions faster than anyone else. To achieve our goal, we make sure that everything is well-optimized and lean. Harper barely uses any CPU or memory at all.

What does all this mean in the context of the recent uptick in user acquisition? Imagine, for a moment, we are LanguageTool. Our software is written in Java, with questionable attendance to the efficiency of our code. Since our software requires a big server to run, we have a certain number of servers on standby to handle a certain amount of expected load.

If suddenly the number of requests we received were to double (as it did for the Chrome extension), we would be big trouble. To avoid increasing the latency on requests (or dropping requests all together), we would need to scale up the number of running servers. This not only takes hiring an expert in cloud architecture, but also additional funding to pay the AWS (or whatever) bill at the end of the month.

Because Harper runs at the edge (no server required), we don't have to worry about that. In fact, I only noticed that our user count had spiked after I visited Hacker News the morning after. No hiccups at all.

Lots of cloud providers like to brag about being able to scale with their users. I like to brag about not having to scale at all.

A bold title, no? But it’s true, and it’s something that I (and most other maintainers) have to deal with on a regular basis. Failing to account for this reality can slow down development and dissuade contributors from sticking around.

As the chief maintainer of Harper, one of my main jobs is to act as quality control. I receive a number of PRs each week, both from new contributors and old. Given the diverse pool of contributors, it makes sense that I review a diverse pool of code of varying levels of quality.

When I have my head down on technical and attention-intense projects, I have less time to review code. When I have less time, I need to set priorities. The question becomes: Would I rather let code from high-quality, reliable sources pile up, or code from sources of unknown quality?

I usually end up reviewing the high-quality sources first, since they’ll likely require fewer revisions. Fewer revisions means quicker merge means improving the user experience faster. Whether or not this is a good decision is something I’ve been thinking about all week.

Code Ages

As code sits stagnant in a PR, it ages. You might say I’m crazy, that there is no way the code itself changes as it sits still in an unmerged and unmodified PR. I’d say you’re right, except for one teeny-tiny detail: Code only has meaning when it sits within a broader context. If that broader context changes (for example, when other PRs are merged) the meaning of the unmodified code does too.

Let me put it another way. If the patch from a PR remains stagnant, but the code it gets patched onto changes, the actual impact of the patch does too. This can cause all sorts of problems, from merge conflicts to erroneous test failures.

The longer code sits in a PR, the more time it usually ends up taking me to get it merged (when I finally have the time to get to it), just from the merge conflicts alone. It’s the Lindy effect rearing its ugly head once more.

Features Age Too

Not only does code age when left untouched and unused, so do the features they represent. If a user requests for a button that marks some text as bold and they don’t receive that feature promptly, they’ll find an alternative solution, possibly from a competitor.

The rest of the app can evolve too. If you’re an individual contributor, it’s in your best interest for your code to make it to master. If that doesn’t happen quickly enough, another contributor (or, if the software is extensible, like Obsidian or WordPress) or plugin author might beat you to the punch.

What Can I Do About It?

I believe the way I’ve been handling this until now has been entirely wrong. I should be allocating more of my time to training new contributors and fielding PRs from established ones. Open sources is a team effort.

As for you, dear reader, that’s for you to figure out. The worst way code can age is if it stays in your head. Don’t let your ideas go to waste. Open that PR, or remind your reviewer to take another look if you haven’t already.

It has been a slow couple of months for my reading habit. Although, now that I've said that out loud, I realize I say that every time. I never quite read enough books by the time I feel ready to write one of these reading reviews.

In the interest of brevity, let's jump straight in.

How to Stop Time by Matt Haig

After reading The Midnight Library in January, I was thoroughly impressed by Haig's work. I wanted more, but wasn't sure what to read next. So, the next time I saw a book with an interesting title which happened to be his, I picked it up.

If The Midnight Library was a literary therapy session, How to Stop Time is a history lesson. It follows a man who ages at one-tenth the rate of a normal human. Despite being given the gift of extra life, he regularly falls into spats of depression when his loved ones die before him.

While Haig's writing continues to be sentimental and appreciative of the small things in life (a trait few seem to possess these days), I ultimately found the book predictable and unengaging. It was a good book, and I felt satisfied having read it, I had put it down for a few days at a time more than once to do so. If you're going to read just one of Haig's books, it shouldn't be this one.

Everything is Tuberculosis by John Green

I've been a fan of John Green's work since I was kid. I remember reading Looking for Alaska as I cruised down the Mississippi River on a school trip. It was one of the most memorable books (and expeditions) I've had the pleasure of experiencing.

Naturally, I was ecstatic to hear John Green published something new: Everything is Tuberculosis, a book about the facets of our lives (of which there are many) that we can thank TB for.

I liked this book, but I don't think it matches the reflective attitude about the human experience you find in his other books—but that's a good thing! Everything is Tuberculosis is not a book you're supposed to relate to, but a book that tells a story about a life most in the western world find completely unfamiliar. One where each and every day is counted. One where the price of a medication is the most important number. One where you might not survive.

The fact that I couldn't relate to the events in Everything is Tuberculosis is the exact reason why I'm glad I read it.

How to Win Friends and Influence People by Dale Carnegie

I read How to Win Friends and Influence People (a mouthful, I know) not because I needed advice on the subject. I read it because I believe this book to be a telling window into the minds of the businessmen of the early 20th century.

I've been told that this book is still relevant today; that the advice still works. I disagree. The priorities, values, and culture are entirely different today than when this book was written. I found it fascinating to compare and contrast between my own experience and the author's.

There's a great deal of overlap between those who use Harper and those who use Firefox. Foremost, they both value privacy. Which is why I'm so thrilled to say that Harper has an official plugin for Firefox. While there has been an unofficial one for a while now, this is the one that fully deserves the Harper label. It's ours, through and through.

It's still early days, so if you run into problems, feel free to open an issue.

If you'd like to install it, pay us a visit on the Firefox Add-on Store.

I've been deep in the refactor rabbit-hole. You know—that awful (but strangely satisfying) space where the majority of your commits are small pedantic edits that don't change the end-user-experience at all?

These past few weeks, I've been working so hard on Harper's Chrome Extension and building systems for transformation-based learning, I haven't had a chance to touch any of the actual grammar rules the software is supposed to fix. As such, a few glaring tools for rule authors have gone unimplemented. These are tools that—because they're so tied into the core system—need a lot of expertise and fine-tuning to work properly.

If you take a look at the diff, you'll find that there are a lot of small edits. Since each one takes a minimal amount of cognitive effort, it is easy for me to slide into a trance. A trance where I am technically moving towards my goal (in this case, creating an expression system that encompasses—but is more powerful than—our existing Pattern system), but I'm not doing so in a way that's truly productive. I would like to cover some of the individual strategies I've found that have helped me speed up my refactoring proces. After all, if the process of refactoring is the process of paying back tech debt, refactoring is incredibly important to maintaining velocity.

I am doing this for two reasons. First, because I believe this is valuable information for any developer. Second, because I want to solidify these ideas in my mind to further improve my refactoring down the line.

Use Your Tools, but Not Too Much

In whatever language you're working with, there are likely specialized tools for refactoring. For Java, I've used IntellJ IDEA. For Harper, I'm using rust-analyzer. These provide neat functions for changing identifiers or moving modules, all while updating relevant references.

These tools are imperfect, however, and often fail to update references in parts of code that require a higher level of semantic understanding.

It's a common pattern for authors to create an instance of a class and assign it to a variable with the same name.

#[derive(Default)]
struct Foo {
    // Implementation details
}

fn main() {
    // The author of this code named the variable after the class
    let foo = Foo::default();
}

In these cases, when someone changes the name of the class, the associated variable name will not get updated.

#[derive(Default)]
struct Bar {
    // Implementation details
}

fn main() {
    // Variable name is not updated by our static analysis tool.
    let foo = Bar::default();
}

When doing a refactor like this, you need to read every changed file to ensure the code still makes as much sense to a human as it does to the compiler.

Plan It Out, but Not Too Much

I can't believe I need to say this, but it's chronically under-discussed. If you're doing a fundamental change to how a system works, it is vital that you read the code first and plan it out. The plan doesn't need to be specific nor need to be complete.

Having a good idea of what the start and end state should look like will save you hours of time. While I am a proponent of discovery coding for new features and code, I find that it is lackluster when it comes to major re-writing of code.

To instruct people to plan out their changes may feel juvinile or basic, but I think it's worth mentioning.

The Joy

Refactoring helps me write new features, faster. Refactoring helps me find bugs in existing code. Refactoring makes it easier for newcomers to join and contribute to a project.

It is one of my favorite pastimes. I enjoy refactoring, but also want to do it well. I hope you feel the same.

The most common type of machine learning out there takes the form of some kind of neural network. Inspired by how our own brains work, these systems act as function approximations. They are great, but they come with a few key pitfalls.

First and foremost, they start out with very little baked-in understanding of the context they live in. This is fine—usually enough data can be provided to bridge the gap. It does mean, however, that they spend an inordinate amount of time learning the fundamentals of their field. This translates to a larger model size and longer inference time (especially at the edge).

Secondly, most neural networks are initialized with randomness, which results in extremely high entropy. High entropy means that these models cannot be compressed easily (if at all).

This disqualifies neural networks from many aspects of Harper's architecture. Harper tries to be fast and small, so it can be shipped and run wherever our users are. Neural networks (especially in the world of natural language processing) are neither fast nor small.

This is why we've taking an alternative approach to machine learning, as evidenced by last week's post on transformation-based learning.

Transformation-Based Learning: A Refresher

Transformation-based learning is remarkably simple. It boils down to just four steps:

• Use a simple, stochastic model t label your data. This can be as simple as tagging each token (or other discrete component) with that variant’s most common tag. It doesn’t need to super accurate, just enough to establish a baseline.
• Identify the errors between the tags in your canonical data and that which produced by your baseline model.
• Using a finite list of human-defined templates, generate candidate rules that transform the output of the baseline model into something else. This is where the term “transformation-based” comes from.
• Apply each of the candidate rules to the baseline model’s output. Check if the result is more accurate than before. If so, save the rule for future use.

These saved candidates become your model.

If you're interested how this could be applied to POS tagging, I've since updated my original post on the subject to better explain the process. I'd recommend taking a look.

Nominal Phrase Chunking

It's often useful, especially when building a grammar checker, to be able to identify the subjects and objects of sentences. Suppose, for example, that we want to insert the missing Oxford comma in a list of fruits: "I like apples, bananas and oranges". In this trivial example, this can be done with POS tagging. If we have more complex subjects, like in the phrase "I like green apples, deliciously pernicious bananas and fresh oranges," POS tagging starts to fall apart. Identifying multi-token subjects is the job of a nominal phrase chunker.

I've been wanting to build a nominal phrase chunker for a while, but haven't had the tools to do so. Now that I have pipeline in place (from last week), it should be relatively straightforward.

For the purposes of this model, we'll be tagging each token with a boolean; it is either a member of a noun phrase, or it is not.

I started by assigning each token to a nominal phrase if a POS tagger marks it as a noun. This is our baseline model. It performs poorly because the resulting nominal phrases do not include determiners or any adjectives.

Similar to our POS tagging model, I used a Universal Dependencies treebank to determine the accuracy of our baseline. After generating candidate rules using the same patch templates as the POS tagging system and running them against the treebank, I have a model with a 90% accuracy.

I feel as though there should be more details to share, but that was pretty much it. I spent a good amount of time optimizing the training code. There's still a lot of work left to do to incorporate it into the rest of Harper. I am also unsatisfied with the model's current accuracy. To get it closer to 100%, I suspect I'll need to do a good amount of data cleaning.

Harper is currently undergoing some pretty radical changes when it comes to its language analysis. These improvements will improve the output of our existing rule engine, in addition to making entirely new corrections possible. This post will cover our existing NLP pipeline, the recent changes and improvements to our machine learning approach, and what will come next.

While AI is a common topic of discussion online, I don’t hear much about actual machine learning. In that light, I hope this post piques someone’s interest.

What is POS Tagging?

POS (Part-of-speech) tagging is the first step of most NLP (Natural Language Processing) pipelines. For any grammar checker worth its salt, POS tagging is essential. Apart from the basic corrections you’re capable of doing with simple string manipulation, most grammar checking directly or indirectly depends on POS tagging. High-quality tagging results in high-quality suggestions.

What is POS tagging? It is the process of identifying which possible definition of a word is being used, based on the surrounding context. For those unfamiliar with the territory, I’m certain an example is the best way to explain.

“I am going to go tan in the sun.”

Here we have a simple, English sentence. In this case, it is clear the word “tan” is being used as verb. The linguists in the audience would point out that it is specifically in the first-person future tense. Consider this similar sentence:

“I am already very tan, so I will stay inside.”

In this sentence, the word “tan” is being used as an adjective. How can we tell?

As intelligent humans, some of whom have been speaking English their entire lives, it is easy for us to determine which words are serving which roles. It’s not as easy for a computer to do the same. From an algorithmic standpoint, there are a number of ways to go about it, each with differing levels of “machine learning” required.

Before this week, Harper primarily took a dictionary-based approach. In short: we ship a “dictionary” of English words to the user’s machine and use hash table lookups to determine the possible roles each word could assume. The authors to our rule engine could then use rudimentary deductive reasoning to narrow the possibilities down. This strategy is remarkably effective and it has scaled to tens of thousands of users with surprisingly few hiccups.

That said, there are edge-cases and systems (which I’ll cover next week when I discuss chunking) which require extreme specificity from POS tags. My mission: improve our POS tagging to increase the confidence of Harper’s output and open the door for more advanced algorithms.

Why Transformation-based Learning?

The literature highlights four underlying machine learning model strategies that seem to work well for POS tagging.

• Hidden Markov Models (probabilistic models that predate the modern deep learning era)
• Maximum Entropy Models (statistical models closely related to logistic regression).
• Transformer-based Models (which use deep neural networks)
• Transformation-based Rule Models (which are based on learned rules)

While I heavily considered using a neural network (either via an HMM or MEM), I discarded the technology for three reasons.

• TRMs are typically more accurate (barely; measured in basis points).
• TRMs are more amenable to fine-tuning.
• TRMs are exceptionally low-latency and can be compressed quite small.

Transformation-based learning is remarkably simple. It boils down to just four steps:

• Use a simple, stochastic model to label your data. This can be as simple as tagging each token (or other discrete component) with that variant’s most common tag. It doesn’t need to super accurate, just enough to establish a baseline.
• Identify the errors between the tags in your canonical data and that which produced by your baseline model.
• Using a finite list of human-defined templates, generate candidate rules that transform the output of the baseline model into something else. This is where the term “transformation-based” comes from.
• Apply each of the candidate rules to the baseline model’s output. Check if the result is more accurate than before. If so, save the rule for future use.

These saved candidates become your model.

POS-Tagging using Transformation-based Learning

Let’s apply these steps to build a POS-tagging system.

For our baseline model, we will just assign each word in our dataset the most common POS tag associated with that word. If the word is “tan”, we’ll assign it’s most common POS tag (verb). It will often be incorrect, but those cases will be handled by our rules.

To identify the baseline model’s errors, we’ll use an off-the-shelf tree-bank from the Universal Dependencies project.

Our rule templates will take the form of these PatchCriteria. By initializing our candidates with any one of these enum variants and initializing the child variables to random values, we can cover a good number of cases.

#[derive(Debug, Clone, Serialize, Deserialize, Hash, PartialEq, Eq)]
pub enum PatchCriteria {
    WordIsTaggedWith {
        /// Which token to inspect.
        relative: isize,
        is_tagged: UPOS,
    },
    AnyWordIsTaggedWith {
        /// The farthest relative index to look
        max_relative: isize,
        is_tagged: UPOS,
    },
    SandwichTaggedWith {
        prev_word_tagged: UPOS,
        post_word_tagged: UPOS,
    },
    WordIs {
        relative: isize,
        word: String,
    },
    /// Not applicable to the Brill Tagger, only the chunker
    NounPhraseAt {
        is_np: bool,
        relative: isize,
    },
    Combined {
        a: Box<PatchCriteria>,
        b: Box<PatchCriteria>,
    },
}

Finally, we’ll apply each of the hundreds of thousands of candidates to our treebank to see if the result of their transformations have a lower error rate than the baseline.

Here are a couple of the candidates we found:

[
  {
    "from": "PRON",
    "to": "SCONJ",
    "criteria": {
      "Combined": {
        "a": {
          "WordIs": {
            "relative": 0,
            "word": "that"
          }
        },
        "b": {
          "WordIsTaggedWith": {
            "relative": -1,
            "is_tagged": "VERB"
          }
        }
      }
    }
  },
  {
    "from": "PART",
    "to": "ADP",
    "criteria": {
      "Combined": {
        "a": {
          "WordIs": {
            "relative": 1,
            "word": "there"
          }
        },
        "b": {
          "AnyWordIsTaggedWith": {
            "max_relative": -4,
            "is_tagged": "NOUN"
          }
        }
      }
    }
  }
]

That’s the whole process! With it, I was able to bring our previous accuracy all the way up to 95% (from 40%) without a meaningful change in linting latency or compiled binary size.

We designed Harper to be the ultimately portable grammar checker, but we're still working on living up to that promise. As the Harper Chrome extension becomes more capable day-by-day, it has one critical weakness: it doesn't support Firefox. Since Chrome has a much larger market share, it isn't a priority at the moment (although we accept contributions).

This is a wonderful example of where the open-source nature of the Harper's core engine really shines. Someone has taken harper.js and put it into their own extension. SpellBolt is a Firefox extension that delivers Harper's fantastic grammar checking straight into your Firefox browser. My initial testing shows that it's a solid extension.

I am delighted to see people making use of the pluggable architecture Harper has adopted, and I can't wait to see what SpellBolt does next.

Harper's Chrome extension continues to come along beautifully. I'm actively working to make it more useful and responsive, slowly crushing bugs that I hear about from our users and contributors. I will not go over them here, since our closed pull requests should speak for themselves. I would, however, like to rapid-fire some of the small improvements and ideas about Quality that I've been brewing up this week.

Quality

I first heard about big-Q Quality from Zen and the Art of Motorcycle Maintenance by Robert Persig. The book's length—whose main subject is the idea of Quality—speaks to its complexity as a topic. It's a great read, and I'd highly recommend it.

One important point the author makes is that Quality is inherently a human idea. It is hard, if not impossible, to define algorithmically or on paper. When one interacts with an object or a system, they come away with a sense of its Quality, without necessarily knowing exactly what led to that sense.

Some argue that in today's world of LLMs and fast-iterations, the real value of a good software engineer derives from their sense of Quality. We know whether a design decision or change to the code is worth exploring. We know how it will affect users. I don't think this argument is relevant to the LLM discussion at all, but I do agree with it in spirit.

The Most Important UI Element

Harper's Chrome extension has one critical UI element that user's should be interacting with more than any other: the suggestion box.

Before	After

When Harper locates a problem in a user's text, it underlines it and waits for them to notice. When they do, they have the option to click it and review Harper's suggestion. This is a flow typical of most spell and grammar-checking programs, so you're likely familiar with it. This process makes the contents and behavior of the suggestion box extremely important for user satisfaction.

After receiving some feedback related to the suggestion box's visual unpleasantness and difficulty to understand intuitively, I've started making some modifications. Nothing drastic—I don't want to confuse existing users. I've focused on making it more compact and use more visual storytelling. Now, when a spelling mistake should be corrected, it shows an icon rather than a full "Add to dictionary" label. I'm already getting positive feedback about the changes.

All-in, it seems like much of the design philosophy of the UI is being pushed further in the direction of Harper's core tenet: to get out of the way.

Why Quality is Important

Quality is important for the Chrome extension (more so than other integrations) because of the Chrome Web Store's front page.

Extensions that get featured on the front page get a significant amount of free advertising. Millions of users view this page monthly. Getting on the page is guarantee of tens of thousands of new users, if not more.

Here's the rub: you can't get on the front page by paying for it or by chance. An extension is placed on the front page if and only if it meets a high bar of value and quality. In other words: you have to earn it. This is the essential reason why I've been working so hard on Quality these past few weeks: I'm trying to get on the front page.

Most of the steps I've taken in this direction are small enough to not be worth mentioning here. If you're curious, feel free to reach out!

As I've accounted and discussed in previous posts, one of the hardest problems Harper faces is that of the great diversity of the internet. There are a great variety text editors on the web, each used in a different context. That's a beautiful thing, but unfortunately our users expect Harper to work in all of these places seamlessly. I suppose you can view this post as a third part in this series where I talk about supporting thousands of websites for potentially millions of users (with zero server costs, I might add).

This journey started in Discord (as many do), with a satisfied user reporting an issue. I'm quite fortunate: they were technical. They're report was detailed and included some initial speculation on what the root cause could be.

While the actual underlying problem was complex and difficult to fix, that is not what this post is about. It is about Harper's strategy for doing end-to-end testing on the many sites we support.

It's relevant because this was a problem that could have been discovered through end-to-end testing. Since we have a diverse set of users already, they found the issue swiftly. That is far from ideal. Our testing suite should catch these problems before the PR is merged.

Why Playwright?

"Playwright enables reliable end-to-end testing for modern web apps." At least—that's what their site claims Playwright can do. I'm not sure if it lives up to this claim of reliability, at least not yet.

The decision to use Playwright over alternative choices came down to a few key points:

• It's quite polished and well supported (Microsoft seems to be the main player).
• While it is more complex to load a Chrome extension in Playwright than Puppeteer, I am also given a lot more control.
• We're already using it for other integrations (but not for end-to-end tests).

The Game Plan

My goal is to build up a comprehensive-enough test suite that I can catch integration problems in foreign text editors before I merge PRs for logic that interacts with them.

Step one was to get Playwright installed and running on my machine, reproducible with npm. Fortunately for me, this was as simple as: pnpm create playwright. Step two was a little more complex: get our extension loaded within the headless browser. I found this could be done by overriding Playwright's default Chrome properties, instructing it to only install the Harper plugin and nothing else:

export const test = base.extend<{
	context: BrowserContext;
	extensionId: string;
}>({
	// biome-ignore lint/correctness/noEmptyPattern: it's by Playwright. Explanation not provided.
	context: async ({}, use) => {
		const pathToExtension = path.join(import.meta.dirname, '../build');
		console.log(`Loading extension from ${pathToExtension}`);
		const context = await chromium.launchPersistentContext('', {
			channel: 'chromium',
			args: [
				`--disable-extensions-except=${pathToExtension}`,
				`--load-extension=${pathToExtension}`,
			],
		});
		await use(context);
		await context.close();
	},
	extensionId: async ({ context }, use) => {
		let [background] = context.serviceWorkers();
		if (!background) background = await context.waitForEvent('serviceworker');

		const extensionId = background.url().split('/')[2];
		await use(extensionId);
	},
});
export const expect = test.expect;

From there, it was pretty trivial to build out assertions and tools for interacting with basic elements for the specific text editor I was interested in (Slate).

The Cool Part

Most text editors on the web advertise themselves in the DOM, usually with a special attribute like data-lexical-editor="true" or data-slate-editor="true". This even happens on world-class sites like LinkedIn or Instagram. I wonder if I can use this for something?

I believe this consistency in production code is intentional. Making our tests easier to write must be a side-effect of making the editor author's tests easier to write.

This is great news for me. With just a few tweaks, I can use the same code to test Harper on Discord, Medium, Notion, Desmos, Asana—you get the point. Since they use just a small set of rich text editors (which come pre-tagged), I can generate automated tests to determine whether Harper works properly on their sites.

The Not-So-Cool Part

The bad news: this process is slow. Each page must be fetched from the network and operated on like a user. We might be able to fix the first problem, but the second is interminable. My initial experiments put the runtime of each test case around thirty seconds. If I'm testing hundreds or thousands of sites, this is a real problem.

For the time being, I'll have these tests be their own workflow in GitHub Actions and only run them when the extension code changes. I don't see people other than myself messing with this code too much, so I am not worried.

What Does Testing Thousands of Sites Look Like?

I'll admit, I'm not quite to the "thousands of sites" territory just yet. The most frustrating part is getting started. Now that I've got a couple of sites under my belt, with the tools ready and able , the next thousand will be a lot easier.

Before I get too deep, I just want to get it out of the way: OpenAI's o3 model is impressive. With its tool use and web search capabilities, it can do a lot more than most offerings out there.

That said, although I've found it to be quite a capable coder, I still don't trust it with anything important. Once or twice, I've instructed it to outline a React component, only to rewrite most of it myself.

I also don't trust its factual accuracy at all. After hallucinating a campground and several entire web APIs, I can't quite believe anything it says. Not that I've trusted any model that came before.

That said, there is one thing it is incredibly useful for: finding canonical documentation for complex subjects.

I've found the greatest success, personally and professionally, when I am working with the most concrete and original source of information available. When working with the web, the most canonical source is the W3C spec. When working with compilers, it's The Dragon Book. When researching the ins and outs of GNU/Linux systems, it's the man page.

This can't be a novel concept. You (the reader) must see things the same way I do. Even LinkedIn seems to agree that base-truth documentation is where we should be getting our information.

I've always wondered: if the W3C spec is the best place to find information about the web, why isn't it the first result on Google? Why, after all these years, is W3Schools still the first result 90% of the time?

I use o3 to find canonical sources of information.

I was recently looking to improve the dynamic range in my camera, but do so before my post-processing step. By improving dynamic range in-camera, I can avoid the pitfalls of certain kinds of compression. I asked o3: "find me the canonical guide for improving dynamic range on my D7100 from the most authoritative source."

I learnt more from the resulting guide (which was hosted on the Nikon website, NOT ChatGPT) than the last three years of shooting, combined.

When our users install Harper, they should expect it to work anywhere they do. Whether they're writing up a blog post in WordPress, leaving a comment on Facebook, or messaging a loved one on WhatsApp, Harper should be there. Harper's core is extremely portable and it can run pretty much anywhere Rust can, so what's the big deal?

Why can't we just run Harper in the browser through a web extension?

Running Locally

There's a single complaint that I hear over and over again from people who use Grammarly or LanguageTool: they are both slow as molasses. The process of writing has slowly evolved to be more complex than it needs to be. With these tools, it writing looks like:

1. Write a sentence.
2. Wait for the grammar checker to run (which takes as many as four seconds).
3. Fix the mistakes you made.
4. Go back to step one.

The whole process reminds me of the copilot pause. This is part of why Harper is better than these other tools: it doesn't stop you from writing at the speed of thought. Our most ardent users tell us this all the time: it feels great to just write, error free.

How do we deliver grammar checking so quickly?

Instead of hosting huge Java codebases in the cloud, we ship our software straight to the user's device. Since there's no network request involved, we're able to put pixels on the screen faster than anyone else. That's not to mention the privacy implications.

Running Harper's engine locally in the browser presented some technical challenges.

I'm quite proud that our JavaScript library can be installed as simply as npm install harper.js. In order for that to work as well as it does, I needed to develop a system for:

1. Compiling our engine to WebAssembly.
2. Shipping that engine to the browser.
3. Instantiating the WebAssembly code.
4. Build out the boilerplate necessary to make it feel native.

Steps one and four were easy. I just slapped #[wasm_bindgen] tags on a Rust library and put on a pot of coffee. Steaming coffee is vital for writing tedious JavaScript.

Steps two and three, however, were a little more difficult. The latest iteration of Google's extension standard, Manifest V3, places some heavy-handed rules on how executable code could be loaded. I won't bore you with the details here. Know that I spent many hours in JavaScript bundler hell.

Running Everywhere

Harper, nascent as it is, has the greatest market opportunity in the browser. Over 3.5 billion people use Chrome on a weekly basis. The plurality of knowledge workers spend most of their waking moments (as crushing as it sounds) in a web browser. Half of the time spent at desks today is spent writing.

In order to address this market segment, we need a Chrome extension. To lint text in the browser, I need a way to:

1. Cleanly read text from input fields.
2. Locate the pixel coordinates of grammatical errors.
3. Render suggestions in popups.
4. Cleanly replace text in input fields when a suggestion is selected.

Reading and Writing Text is Hard

The web may have standards, but there is nothing standard about it. The "standard" way to input text is with a <textarea /> element. Even so, most high-traffic sites implement their own text editors from scratch, using <div contenteditable="true" /> as a base. Each of these cases required special care.

<textarea />

<textarea />s are hard for one reason: it is difficult to get a good understanding of what they look like. I can obtain their content with input.value, but I can't directly infer the pixel coordinates of grammatical errors inside them.

When the Harper Chrome Extension is offered an <input /> or <textarea /> element to analyze, here's what it does.

1. Creates a new <div />.
2. Copies all styles from the provided element onto the <div />.
3. Using position: absolute;, it moves this <div /> directly on top of the provided element.
4. Copies the content of the provided element into the <div />.
5. Uses the Range API to turn the text indices emitted by Harper's engine into pixel coordinates on the <div />.

Using this mirroring strategy works, but has performance implications and additional complexity to handle scrolling within the element.

<div contenteditable="true" />

Since elements in a contenteditable text editor (like Trix, Lexical, Gutenberg, etc.) actually exist in the DOM, I can just use the Range API to get pixel coordinates. The trouble this time comes when I try to write a suggestion back into the editor.

Most documentation I could find suggests that you:

1. Select the content of the element you wish to edit. This can be done using window.getSelection().addRange().
2. Call document.execCommand('insertText', null, "YOUR TEXT").

Much of this documentation acknowledges that document.execCommand is deprecated, but instructs you to use it anyway.

This is bad advice. Do not do this. I spent an embarrassing amount of time trying to get it to work consistently.

The better way to replace text programmatically comes directly from the W3C standard:

1. Manually edit the DOM in the fashion outlined by the suggestion chosen by the user.
2. Fire input events to instruct WYSIWYG editors to synchronize their internal state to the DOM.

I'll admit that this is an oversimplification. Much of the complexity here lays in determining which DOM nodes to edit or fire events on.

Read the Source of Truth

If you take one thing away from this post, it should be this: always read from the source of truth. There is a lot of faulty information out there, especially when it comes to creating complex interactions with opaque systems. If there is a source of truth, read it. It may look intimidating or seem unnecessarily verbose. It is that way for a reason.

In his popular book, Outliers, Malcolm Gladwell made the case that there was one number that predicted your skill level in a field: the number of hours of dedicated practice. In popular culture, this idea has been simplified. Some think that if you just do something for 10,000 hours, then you'll become a world-class expert. This is false.

What does the word "dedicated" mean in the phrase "dedicated" practice? It means applying focused, deliberate thought to it. It means reflecting on your own performance. It means seeking help from masters in the field. Above all, it means achieving mastery is not easy.

I aim to master the skill of writing. In my free time this week, I've been working on my deliberate practice of writing. In doing so, I've come to some conclusions. If you'd like to join me in pursuing mastery of the written word, here's what I think you should do.

Proof it Yourself

Two-time Pulitzer prize-winner David McCullough says "Writing is thinking. To write well is to think clearly. That's why it's so hard."

Because the act of writing is also the act of thinking, the process of revising text clarifies thought. I personally find that my argument when I first touch my keyboard usually stands in stark contrast to when I hit "publish."

Heavy-handed revision tools like ChatGPT actively disrupt this great benefit of the revision process. Sure, they can rewrite tone to be more attractive online, but they also rob you of the opportunity to develop your concept. That's why I turn everything else off and only use Harper: because it gets out of the way of what I am trying to say. You can't reflect on what you've written if you are not the one reflecting.

Read The Work of Great Writers

When I sit down to write after spending time online, I feel deflated. The competition of social media and the churn of the news cycle both lead to poor prose.

When I sit to write after reading Bonnie Garmus or Cal Newport, I feel invigorated. I can imagine a shape for the words on the page which precisely captures the ideas I'm trying to communicate. Reading great literature gives you an intuition for what your own work should look like.

Think of the Hook First

In rapper circles, it's a common point of debate whether to think of the verse first or the hook.

On one hand, the verse is the substance of the song. It contains the message, the detail, and the personal meaning. Music derives much of its complexity from the verse. On the other hand, the hook is what gets people to listen. It is what makes them stop flicking through Spotify and fall into the beat.

If you're writing for an audience, you have a lot of competition for their attention. Spend more time on the introduction than anything else. It should capture your readers, delight their imaginations, and put them in your shoes.

Don't Get Attached

I spent a long time on a post similar to this one. I was quite proud of it, for I had spent hours adding layers of detail and quite a few case studies from my own work. Alas, Gutenberg decided it was not meant to be and chucked it into oblivion.

Although I was initially crestfallen from the loss, I now believe the second round ended up even better. To truly revise your work, you'll inevitably have to delete parts of it. Get comfortable with that.

My mom is a violently intelligent woman. But she lives in the unfortunate reality of not being a software engineer or mathematician. While she is just as excited about ChatGPT (and all the other nascent AI technology) as the rest of us, she isn't aware of the unintelligent pitfalls artificial intelligence has these days.

If your mom is experimenting with ChatGPT, I'd highly recommend sending this list to her. Powerful technology is in their hands, but I've got three things they should know before relying on it.

Advice for Moms

Use Web Search

My mother recently learned how to make cobbler. She adored asked ChatGPT questions about the recipe, and how she could improve it using the ingredients she had on hand. Unfortunately, she was getting subtly wrong information from the bot, so the cobbler turned out "sub-par". This can be solved by using ChatGPT's web search.

When you open up ChatGPT on a phone or on a computer, you'll be met with the familiar chat window. If you ask it questions it will give you an answer. Sometimes that answer will be good, sometimes not.

You can generally get better information if you enable the web search tool.

This will allow ChatGPT to do Google searches on your behalf. It improves the quality of information it gives you, while only adding a second or two to their response.

It Is Not Your Friend

When we see a chat window, our first thought is that there's a person on the other end. Someone with emotions and lives all their own. That is not the case here.

ChatGPT, while an advanced machine learning model, is not a person. It does not replace real human connection. It can't watch your kids when you've got an emergency. It won't show up to the school recital. It doesn't care about you—despite what it says.

There's a lot that goes into a genuine social connection, all of which are missing from ChatGPT.

Use Your Brain

ChatGPT is no substitute for critical thought.

It cannot do most things, even if it claims it can. You are a capable human being—able to accomplish anything you put your mind to. While ChatGPT can be a helpful assistant or brainstorming partner, don't let it steer the ship.

What to Remember

While ChatGPT can be a powerful tool in the right hands, it is important to remember it is just that: a tool. It can be enhanced with the right settings, it is not real human connection, and it can't replace your brain. Use it well and use it wisely.

Elijah! Elijah! When will we have a Chrome extension for Harper?

Although it's usually asked with a bit more panache, this is a question I've gotten a lot. In fact, an author at Lifehacker wrote about it just yesterday.

For the longest time, it was a project on the back-burner. Something I intended to work on, but was always passed up for more important things. It never saw the light of day.

Until now.

What is Harper?

...and why do I need its Chrome extension?

Harper is a grammar checker that respects your privacy. All language processing happens on-device, no matter where you're using it.

For most of Harper's history, it has only been available in one of many text editors as a plugin. With the Chrome extension, you'll be able to get high-quality grammar checking anywhere on the web.

What's the Status?

I just marked the Chrome extension PR as "ready for review". Here's what that means.

If you're willing to get into the technical brambles, you can compile and run it yourself. That will continue to be the case until the Chrome Web Store reviews and approves our submission, which could take a couple of weeks.

When installed, it will be enabled for a curated list of website that I've personally reviewed to confirm work well. If you want to try using the extension on another site, you will be able to adventure into the world of unexpected behavior by hitting the popup menu.

Importantly, it is working exceptionally well on any site using the Gutenberg editor. I will be backporting a lot of this code to our WordPress plugin to improve the experience there.

Will it Support Firefox?

As the PR stands today, the extension does not support Firefox.

I have list of things that should get done whilst waiting for the Chrome Web Store to approve the submission. Firefox support is on that list.

Any Questions?

If you've got questions for me about this, feel free to reach out via Discord or on GitHub.

The goal of meditation is to become more mindful and aware. For me, it is often an attempt to get an appreciation for my surroundings—to see how utterly complex the world is.

Photography does something similar for me. The heft of the camera in my hands reminds me to slow down and take in the view.

My mother is a photographer—a great one at that. She's been in the business for at least as long as I've been alive. She tells me something similar: the act of framing a picture is itself a way to understand your subject better.

English is an area of interest littered with edge cases. In preparation for addressing similar problems in the development of the Harper Chrome extension, I'm spending some time here reflecting on what I've learned by tackling the complex maze of English edge cases.

Harper is not alone, and there's a real chance that you'll have to work on exception-tolerant code. In that case, you will need to understand the why for some of the design decisions you encounter in the wild.

What Do I Mean By "Edge Case"?

An edge-case is a situation (that is often context-dependent) which results in incorrect behavior from a model, program, or theoretical framework. In conversations about the Harper Chrome extension, an "edge case" is where the extension improperly reads from or writes to a site's embedded text editor.

Most sites use <textarea /> or <input /> elements for text editing, but a number of sites (including WordPress, as you know) have complex WYSIWYG editors. Each behaves differently, which can cause problems with the aforementioned read/write loop. The problem: our users expect us to support all major text editors.

Err on the Side of Inaction

In Harper's core algorithm, we err on the side of false-negatives. This decision was derived from an observation made early on in the project's life cycle: people usually blame themselves for their own writing mistakes, unless the error is truly trivial.

All in, we get far more complaints about false-positives than false-negatives. Which is why we err on the side of inaction. If the algorithm thinks it's possible, but not certain an error was made, we suppress the report in case Harper is wrong.

In the Chrome extension, we will be exemplifying this rule by only enabling it on a curated list of domains by default.

As site visits tend to follow a Pareto distribution, we can cover most traffic with just a few items in this list.

Make Tweaks Easy

When an edge case (which is almost always a false-positive) appears in Harper's core algorithm, it's usually in a pretty obvious spot. This is because we associate each lint output with a specific, easy-to-find module the core code.

In most cases, this is not verbose, specialized Rust code. Rather, it is an LLM-friendly DSL that is legible to most with even beginner-level programming experience. This combination of easy-to-read and easy-to-edit makes contributions from the community (regarding edge-cases) commonplace.

I aim to replicate this success in the Chrome extension's read/write capabilities by carefully documenting the architecture and working with third-parties to make the contributing process clearer and easier.

It Is an Art

There is a reason I call this an art. Design decisions in exception-tolerant systems stem from these two simple ideas, but they grow from an intuition developed by their maintainers. I'm not exactly sure how the Chrome extension will grow to handle edge cases. I'll be sure to come back here and detail them when I do.

I consider myself an avid member of the Neovim community. As a whole, they have a great pulse on what a good writing experience feels like. Over time, this community has grown into a real force in the software engineering field.

Linkarzu has been a constant presence in this ecosystem for a while, giving a voice to the hidden maintainers that make Neovim great. I am honored to be counted among the guests he has hosted on his channel.

We talked at length about Harper, open source maintenance, and the tooling we use in our development workflows.

WebAssembly—even after several years of standardization—is still a nascent technology.

I've been working with Rust and WebAssembly for nearly four years now. This post is intended to be a distillate of that experience, formatted for developers who are interested in publishing WebAssembly code to npm. Specifically, these are the footguns I've personally encountered while working on harper.js, a WebAssembly-powered package for grammar checking at the edge. This page should be valuable if you are even speculating on the possibility of using WebAssembly in your codebase.

1. Exposing Only Synchronous Functions

The Harper package exposes one interface that captures all of its WebAssembly interactions: the Linter. This is an object that handles downloading and compiling the Harper WebAssembly module, as well as invoking functions in it. Every function returns a Promise. There are several good reasons for that:

• WebAssembly modules larger than 4 kilobytes must be instantiated asynchronously to avoid blocking the event loop during download or compilation. This is a technical limitation that cannot be avoided with clever logic.
• If most functions are asynchronous, you can centralize computation and caching into a single instance of the WebAssembly module, hiding the complexity of instantiation and making caches easier to build.
• If your problem domain can be computationally intense, it might be prudent to offload jobs onto a web worker, which is easier if every function of your facade is asynchronous.
• It will be easier for both you and your users to expose any important function as asynchronous out of the gate.

2. Assume Rust Can Trivially Do IO

Whatever you do: avoid assuming that Rust libraries (like reqwest or rand) will be able to perform IO without some work on your end. WebAssembly alone is not capable of IO, which means any function in that category will require some amount of JavaScript to work properly.

Rather than leaving that up to the Rust toolchain to figure out, save yourself the headache and inject the necessary JavaScript functions directly into the WebAssembly module by passing them through wasm_bindgen.

3. Just Inline the WebAssembly Module

Obsidian plugins, for example, must be composed of exactly one JavaScript file, which means everything must be inlined. In the interest of keeping the bundle size small, it's much easier for the package developer (of harper.js) to set up inlining than the plugin developer.

On the other side of the spectrum, Chrome's Manifest V3 disallows WebAssembly from being loaded inline.

If you plan for your package to be consumed by a variety of applications, know that it will also be consumed by a variety of bundlers. Both bundlers and applications are pretty inconsistent with their inlining and tree-shaking behavior. To avoid problems, you should provide two versions of your package:

• One where your WebAssembly module is already inlined.
• One where it isn't.

Wrapping it Up

Harper’s problem domain is not your problem domain. We have to integrate with a variety of unique applications, which means we must keep our system flexible. That may not be the case for you, which may mean these footguns do not apply. If you have any questions about any other problems the Harper project might have faced, let me know.

In a world containing TikTok, YouTube and a vast array of other apps that profit off human attention, it takes effort to place our focus on things that truly benefit our minds.

At least, that is what I would I say on LinkedIn as an explanation for why I read. In reality: I simply enjoy it.

Each month I go back and summarize what I've learned from the books I've read. In doing so, I've found that I get both deeper insights and greater joy from the literature.

Wait! What's Up With February?

While I only finished one book in February, I started reading several others that I ended up finishing in March. It only felt right to combine the two months into a single post.

The Dinner by Herman Koch

Someone convinced me to read this book based on its simple premise: the entire novel is an account of a single dinner. I was skeptical. It is a big book. How could they possibly fit that much plot into a single dinner?

The answer: Koch drip-feeds you information about the eclectic characters in attendance. They gain an additional dimension that you couldn't achieve in another, more action-packed book.

I won't spoil anything here, but the ending was a complete surprise to me. I did not see that coming.

Gold by Isaac Asimov

I remember reading Gold for the first time on a camping trip to the Great Sand Dunes National Park. I was captivated by the dynamic short-stories contained within. At the time I believed them to be pure science fiction.

Reading Gold for the second time revealed how wrong I was. More than one of the fictional ideas Asimov presents in the collection of short stories has since come to reality.

My Grandma Asked Me to Tell You She's Sorry by Fredrik Backman

I'll admit: this novel was challenging to finish.

Pros: the main character (an almost-eight year old girl) is expertly conveyed in the writing. Backman makes clever use of run-on sentences to convey frustration, which is a literary device I've never seen before. Further, the character has an unusual degree of insight into everyday aspects of life. I found myself appreciating my every day more placing myself in her shoes.

Cons: it takes a long time for the insights to start coming. An excessive degree of unimportant information is thrown at you right out of the gate. I had difficulty maintaining my focus until the second half of the book (when stuff starts heating up).

Lessons in Chemistry by Bonnie Garmus

Garmus knocked it out of the park for her first novel. It follows Elizabeth Zott, who, in all respects is the person I want to be when I grow up.

Lessons in Chemistry is a testament to the value of subtraction. The original manuscript of this book was about twice as long as the final published version. The author was forced to condense it down after getting rejected from several publishers. It effort shows.

I cannot capture the novel's elegance of prose, creativity in fiction, or importance in morality here. You need to read this book.

People (myself included) love to make cascading lists of tasks. Actions, each of which are needed to improve a product, release a feature, or manage people. These lists tend to explode in size, as more is asked of you.

There's one problem: I have only eight hours in a working day. With so little time, how can I be expected to both maintain existing projects (bug fixes, etc.) while consistently delivering new functionality and maintaining a fast iterative loop?

Insight

I've had two key insights that have led to some tremendous personal productivity gains.

First: I've observed that the difficulty of my tasks tends to follow a Pareto distribution. A small number of tasks are shockingly difficult to solve, while a much larger set boil down to communication.

Second: the morning is my most productive time of day. For some reason, I'm able to tackle problems with an otherwise unusual mental clarity. I suspect it's something to do with the natural human circadian rhythm or digestion.

Leveraging the Pattern

Motivated by these two observations, I've developed a habit: I dedicate each morning to its own difficult problem. This is a single task that is chosen specifically to stretch my limits. I call this my daily one hard thing.

Previous examples:

• Developing the first version of harper-ls which is now used by thousands of developers daily.
• Building the first 80% of the Harper Obsidian plugin, which is also widely used.
• Grokking how Jetpack's Write Brief with AI feature works (before I joined Automattic).
• Compiling Harper to WebAssembly and embedding it somewhere no one has put a grammar checker before: inside the web browser.

Think About It

I implore you: think about how you work. How is your energy affected by the things you do in the day-to-day? If you enjoy your eight hours, maybe the rest will be better too.

As evidenced by my previous posts on LLM-Assisted Fuzzing, I've been dedicating a lot of my mental bandwidth to maintaining a low false-positive rate while we improve Harper's rule coverage. Part of that is through fuzzing and dogfooding, some can be through statistics, but the first lines of defense will continue to be unit and integration testing. This past week particularly, I've been reading up on how other big linting programs approach this problem.

1. Test Features, Not Code

I often ask myself: am I spending more time thinking or talking about the thing, or am I spending more time doing the thing? I've personally seen how projects fall into decline because their leaders are more interested in planning than doing.

In the context of software testing, this mantra is transformed into "test features, not code." To my eye, good code is flexible and self-explanatory. Tests that hook deeply into application or library internals make code less flexible and harder to read.

I especially like Alex Kladov's heuristic for this: the neural network test.

"Can you re-use the test suite if your entire software is replaced with an opaque neural network?" - Alex Kladov

It's not a question of whether a neural network would pass the test suite, only whether the test suite could work for it. If the answer is no, the tests are likely testing code, not features.

2. Performance

The speed at which you can build and run tests (unit, static, integration, etc.) is a force-multiplier for everything else. You can validate ideas sooner, run CI faster, and get contributors on-boarded in less time.

Our goal to be fast at runtime dovetails quite nicely into this, so it's something Harper already does quite well. Moving forward, we need to make sure that we don't rely on any kind of IO in our tests, since that continues to be the slowest part of most Harper integrations.

3. Good Assertions

We can simplify programs like Harper down into a single function which consumes text and returns a list of observed problems.

fn harper(text: String) -> Vec<Lint>{
    // Implementation details...
}

Most testing we are interested can be done with assertion functions that declare what qualities the output should have with a specific input.

For example, we have a function called assert_suggestion_result, which runs a grammatically incorrect string through Harper, applies the first resulting suggestion and checks whether the edited string matches a given value.

/// An example of a test that uses assert_suggestion_result
#[test]
fn catches_less_then() {
    assert_suggestion_result(
        "I eat less then you.",
        ThenThan::default(),
        "I eat less than you.",
    );
}

It's also vital that these assertions show good, readable error messages when they fail. Each time I've improved their logs, I get unprompted positive feedback from contributors.

Moving forward, I'd like to create a more diverse array assertions like this, as well as better-document their use. A lot of the current back-and-forth for rule contributions is related this.

Wait! I Disagree

I hope someone does. Good test suites are something I'm continuing to learn how to build. I understand that a lot of what I've said here doesn't apply to other kinds of applications or codebases. If there's nuance I'm not covering here, let me know!

Preface: this post was actually written on February 24th, despite being published today.

I'd like to do some reflection on how we can improve Harper's output. It is fundamentally a process problem. For context, let us go through a contrived example.

1. Somebody (usually me) discovers an error in their writing that Harper did not catch. This is a concrete string like "I have a a care in the world".
2. I look for other forms of the error to see if I can generalize it to more cases. Here, we see an indefinite article repeating itself. We could just look for the exact string "a a" and call it day, but we can cover more cases by looking for . In fact, we can generalize it even further. In this case, writers want to avoid repetitions of any word that is not a homograph.
3. Once the code is written that defines the rule, I push it out, and wait to hear from users if any false-positives arise.

This is a flawed system, particularly that last step. I've found something a bit better.

I've been experimenting more with LLM tooling, particularly Ollama. With it, I've set up a useful system for identifying false-positives before pushing changes out to users. I like to call it "LLM Assisted Fuzzing".

LLM-Assisted Fuzzing

Fuzzing is a common practice wherein engineers rapidly and continuously push random data through their system to deterministically check if it is behaving correctly (i.e. not crashing). If a particular batch of random data causes an issue, it is elevated to an engineer.

Harper as a program stands in a somewhat unique position: it analyzes natural language to identify grammatical errors. LLMs exist as a direct complement to it: they generate "natural" language. Do you see where I am going with this?

Let's be clear: language models do not generate grammatically correct text even a good amount of the time. As a result, their output is useful as a proxy for our user's written work. If we can implement a feature that reliably corrects (without false-positives) an LLM's output, we can reasonably expect it to work for our user's text.

My Initial Attempt at LLM-Assisted Fuzzing

I started working on this Friday morning, and finally have what I would call a "first draft" of an LLM-assisted fuzzing system for identifying false-positives. Here's how it works:

I start a local Ollama server, preloaded with several models (which I'll expand on more below).

I run a bash script that repeatedly feeds the same prompt into the models using a command-line client for Ollama. This gives me a large dataset of responses to the prompt. I'm not looking for the semantic content of the response. I'm interested in extensive coverage of whichever linguistic domain I've defined in the prompt. I can run this as long as I like before going on to the next step.

Once I have a large dataset, I run a separate script that runs Harper over each response. If it finds an error, the script opens the response in the Harper editor for further inspection.

The biggest issue with this workflow right now: regardless of my prompts, the LLMs continue to emit actual grammatical errors. This means that when I go through this process, I get at least as many true-positives as false-positives. It also does nothing for false-negatives.

Tweaks

There are a couple parameters here that I'm going to continue to tweak as the workflow becomes more mature. So far, I've been using deepseek-r1:32b and mistral-small. DeepSeek tends to use more colorful language than Mistral, which runs much faster on my machine.

I've also been iterating on the prompt as I go. Here's the latest version:

Please write a long, drawn-out essay on a topic of your choosing.
Write out the essay multiple times before giving me your final draft.
Your final draft should be grammatically perfect, clearly demonstrating that you put time and effort into your submission.

I repeat: I do not want to see anything other than your final drafts.
Do not give me any intermediate work.

Furthermore, do not use any characters or words other than English. 
I do not want Chinese or anything else.

I have all the time in the world. I'm perfectly fine with waiting a little longer if it means you give me something you are proud of.

When I move on to tackling false-negatives with this, I'll be able to prompt the model to give me specific kinds of errors, which makes it significantly more useful than trying to track down existing work.

Harper is a grammar checker for developers. Its roots are in code editors like Neovim, Helix, Zed, and you guessed it—Visual Studio Code. I don't think a lot of my followers know this, so I thought I'd give it a little shout-out here.

Since Cursor and Windsurf are forks of Visual Studio Code, Harper is available for both via the Visual Studio Marketplace. Give us a good rating over there if you install the plugin and end up liking it!

How Does It Work?

Harper works similarly to a number of other extensions you may have installed. Like Pylance or ESLint, it runs in the background as a language server. Each time a modification is made to your document, it reads your comments for grammatical mistakes and typos and displays them as errors or warnings (it's configurable).

Does It Support My Programming Language?

The Harper language server, thanks to contributions from the community, supports a pretty wide range of programming and markup languages. We've only recently added support for PHP, so if that's your thing, know you're in somewhat uncharted territory. If you find issues, let us know.

Preface: This post is specifically about Pull Requests for Harper. Read the contributor guidelines for a project before opening a PR.

I get it. Opening a pull request is an intimidating proposition.

It is non-trivial to put your blood, sweat, and (let's be honest) tears into code and put that onto the internet for the world to see. I remember my first pull-request: it wasn't pretty either.

But I want to highlight a couple reasons why I want potential contributors to open their pull requests as early as possible.

Drafts Reduce Duplicate Work

When a contributor to a project is assessing the viability or usefulness of some work they're preparing to do, most look at existing Pull Requests first. Even if you're still working on the patch, it's a great idea to open a "draft" PR so no one starts work that could go to waste.

Further, if you have a draft open whose CI is failing, a maintainer may see it and fix whatever the problem may be. I find myself frequently browsing Harper's Pull Requests looking for failing builds, since I'm often the best equipped to find the issue.

It Helps With Debugging

I try to make myself available to contributors in case they have any questions regarding the architecture of Harper. I'm also game to help debug their code, partly because I am of the opinion that debugging is a skill which is best learned by example.

But it is hard for me to debug code if I don't have it. If a draft PR has been opened, I can usually see the problem without even cloning the code, since our CI is so comprehensive.

Conclusion

So please don't wait; open that PR. I am thrilled each time I see a name I don't recognize on a GitHub notification.

I've been experimenting with using LLMs locally for generating datasets to test Harper against. I might write a blog post about the technique (which I am grandiosely calling "LLM-assisted fuzzing"), but I'm going to make you wait.

I've written a little tool called ofc that lets you insert Ollama into your bash scripts. I think it's pretty neat, since it (very easily) lets you do some pretty cool things.

For example, you can swap out the system prompt, so if you want to compare behavior across prompts, you can just toss it in a loop:

#!/bin/bash

subreddits=("r/vscode" "r/neovim" "r/wallstreetbets")

# Loop over each item in the list
for subreddit in "${subreddits[@]}"; do
  echo "++++++++ BEGIN $subreddit ++++++++"
  ofc --system-prompt "Assume the persona of a commenter of $subreddit" "What is your opinion on pepperjack cheese?"
  cat
done

Or, you can instruct a model to prompt itself:

ofc --system-prompt "$(ofc "Write a prompt for a large language model that makes it think harder. ")" "What is a while loop?"

Installation

ofc is installable from either crates.io or its repository.

cargo install ofc --locked

# Or...
cargo install --git https://github.com/elijah-potter/ofc --locked

When we started work on harper.js, our goal was simple. We wanted it to take less than 24 hours for a developer to embed high-quality grammar checking in their app.

A friend of mine, Lukas Werner recently set a record for this. In just over 15 minutes, he was able to embed Harper into Raycast, a command palate for MacOS.

This is fantastic news. It's always great to see when your goals are having a real impact on someone's ability.

I hope you take the time to look at the project on GitHub.

In other news, we have some other small projects using Harper:

• Lockbook
• Stubby CMS

I only got to read two full books this month. The first was a short novel by Matt Haig with the title The Midnight Library.

It reads like and feels like a therapy session. I quite enjoyed it, but I'm not sure it is for everyone.

The second book (and the one I got the most value from) was Working in Public: the Making and Maintenance of Open Source Software by Nadia Eghbal. Until now, I've largely organized Harper in ways inspired by projects like Vite and Rust that I interact with on a daily basis.

Eghbal gives a more holistic view than mine, written for an external observer. This book was especially interesting due to the comprehensive coverage of "open" projects with no code at all. She seems to believe the OSS software management problem is one of social complexity, not technical complexity.

Furthermore, Nadia makes the case that synchronous communication, while often a burden on maintainer attention, is usually necessary.

I have found that the first couple hours of the day are my most productive. I try to move tedious tasks to later so I can focus on solving "hard problems" that need my full attention.

In the past, this has been hard to do for one reason: GitHub notifications. They pull me in and consume hours of my time. Responding to and working with OSS contributors is important to me, but that can happen later in the day.

So I wrote a little ViolentMonkey script to hide the notifications icon during these early morning hours.

// ==UserScript==
// @name        Hide Notification Indicator
// @namespace   Violentmonkey Scripts
// @match       https://github.com/*
// @grant       none
// @version     1.0
// @author      Elijah Potter
// @description 1/29/2025, 8:16:11 AM
// ==/UserScript==

function hideEm() {
  const matches = document.getElementsByClassName(
    "AppHeader-button--hasIndicator",
  );
  for (const element of matches) {
    element.remove();
  }
}

new MutationObserver(hideEm).observe(document, {
  childList: true,
  subtree: true,
});

My grandmother is a wonderful woman. She never leaves the house without a least a little bit of purple. Maybe a keychain or a purse. Sometimes, she wouldn't leave the house without a lot of purple, head to toe. She also never forgets a smile and a good attitude.

She's a wise woman. With more than 70 years and five children under her belt, she's seen some stuff. All that experience comes with a lot of advice. The three steps to an apology are some of the best bits of advice she's ever given me.

Step One: "I am sorry"

Admit you've done something wrong. Recognize your fault and imagine what would have happened if you had done the right thing.

Step Two: "It will not happen again"

If you're actually sorry, you'll want to avoid replicating the same mistake at a later date. Figure out what you need to do to make sure you this is a one-time error.

Step Three: "What can I do to make it better?"

Attempt repair. In a fantasy world, you could go back in time and stop yourself from doing whatever it is you did in the first place. But we don't live in a fantasy world, so you have to find a way to make it right, as soon as possible.

Earlier this year (I don't remember the exact day or month) I decided, screw it, I'm going to try using a flip phone. For years I've wondered what it was like before the information revolution, but as a Zoomer, I've never had the opportunity.

I got the cheapest Verizon-compatible flip phone Amazon had to offer and I committed myself to using it exclusively for at least a week. A week without the entirety of human knowledge in my pocket. A week of using my voice to communicate, rather than a touch-screen keyboard.

Beside two factor authentication for GitHub, that first week was a breeze. I found myself socializing more, since it turns out that most people in my social circle are just a phone call and a 10-minute walk away. I also found myself reading better quality literature. Rather than resorting to HackerNews to ease my boredom, I began devouring full books at a frightening pace. That first week turned into two, then four, then twelve. All the while, I was proving to myself that I didn't need a smartphone.

For a number of reasons, I'm back to using a smartphone. But this experience has taught me some important lessons that I don't think can be taught any other way.

Organizing social encounters over text just doesn't work.

Some people don't check their phone too often. Calling them make it clear that their participation is important to you. This may sound trivial or obvious to some of the older members of my audience, but to pretty much every Zoomer I've talked to about this, it is a revelation.

Quiet is sacred, and should be protected.

I see loads of people who walk around every day with their headphones in, barely paying attention to the music blaring out of them. I've done this myself. It closes you off the world. From the tapping of water off an icicle. From the shaky but warm greeting from a colleague. From the chortling roar of an engine.

But most of all, it closes you off from your own mind. The best thinking and imagining happens in the quiet, when your brain has nothing more stimulating to do.

Overall, using a flip phone was a very quieting experience. It helped the world feel larger and more magical.

As the maintainer of a reasonably popular open source project written in Rust, I find myself cloning PRs and swapping between branches dozens of times a day. In doing so, I often blow away Rust's compilation cache either via git clean -xf . or by changing dependency versions.

Having a build cache that is separate from the project is a huge win for compile times, which is why I use sccache.

For those unaware, sccache is a build tool for C, C++ and Rust projects that stands in front of the compiler. If a given file's hash is the same across calls to the compiler, sccache reuses the previous result. This cache is compiler–and project agnostic. In my case, it sits in $HOME/.scccache, but you can set it up with AWS S3, Redis or a myriad of other options.

If a dependency is cached from a compile in one project, it is available for a build in another.

In short: you need sccache if you are frequently swapping between branches with similar (but not identical) dependencies or expect to frequently delete Rust's disk cache.

Why You Don't Need sccache

If you are working in a single branch on a project whose dependencies remain somewhat stable, sccache might actually increase your compile time. Since sccache has to hit the disk (or even the network) regardless of whether it is a cache hit, a little bit of latency is added. If you expect most compiles to be cache hits or most compiles to be cache misses, sccache is no more useful than the built-in caching cargo provides.

In that case you should not use sccache.

Wrap-Up

Regardless of whether you install sccache on your machine, it is a neat little technology. I would recommend the curious reader take a look for themselves. You might be inspired.

This year I've been reading a lot more. Is there a particular reason? Other than simply admiring those that do, no.

I have found some gems of literature that I would like to share.

Social Intelligence by Daniel Goleman

Social Intelligence is truly a treasure. I heard of Daniel Goleman's work first from an episode of Talks at Google and later from psychologists who raved about his previous book, Emotional Intelligence.

It gives a deep dive into the subconscious and lightning-fast systems the underlay every social interaction. As I read it, I found myself analyzing my own behavior, as well as the behavior of others. As someone who has been overthinking my own social life from a young age, Daniel's advice to let my brain go to its default mode was relieving.

If you're the type to feel anxious in social situations, this book is perfect for you.

Tomorrow and Tomorrow and Tomorrow by Gabrielle Zevin

Tomorrow and Tomorrow and Tomorrow is probably one of the best fiction books I've read. Period.

At first glance, it is just the story of the careers of two game developers and their producer. But if you dig a little bit deeper, it reveals itself to be the story of how the relationships between people who work together can be incredibly deep and complex. Specifically, how the passion to create something can both bond and break people.

I don't think it is possible for me to accurately describe the emotional rollercoaster this book took me on. Which is why you should read it!

Why We Sleep by Matthew Walker

I was curious why some people in my life sleep very little and other sleep quite a bit, both with varying levels of general wakefulness. So, I ran a search: "Why do we sleep?"

To my surprise, I discovered someone wrote a book with almost that exact title. Since I've read it, I've increased the duration of my sleep by nearly two hours. This practice has had an outsize impact on my health and happiness. I could not recommend it more.

I've written about this book extensively, so I won't repeat myself further.

That's It!

I could wax poetic about these books and how they've changed me—but I won't. In reality, I read them because I enjoyed reading them. In an age where people read less than ever, I implore you: try picking up a book, you might just like it.

I adore a good iframe. They're so elegant as a web component.

Just expose an endpoint, say https://writewithharper.com/editor, set it up to accept some query parameters and get it hosted. Now you can access this web-component from any page that has a loose enough Content Security Policy. For me, that means my school assignments and other assorted documentation.

It also means that I can avoid setting up a complex build system for MDX, while still being able to include interactive components.

The example from earlier:

<iframe
  src="https://writewithharper.com/editor?initialText=See, we can now embed the the Harper editor%0Arght into this document!%0A%0AIt's a little too easy."
  width="100%"
  height="400px"
  style="border-radius:10px;border:0px"
></iframe>

One major caveat though: when we pass our arguments to the component through the query URL, this gets sent to the component's server as well. I certainly trust the Harper website's server, since I maintain it and the code is open source, but that isn't always the case.

You should also probably avoid doing this too much. Most browsers spawn a whole new process for every iframe, so if you want things to stay snappy it is best to limit yourself to just one (maybe two) per page.

Someone recently asked me where the name Harper came from.

When I first sat down to start work on Harper, I had one goal in mind. I wanted to create a grammar checker that wouldn't impinge on the artistic freedom of authors. Part of that meant that I did not want literary devices to be flagged as grammatical errors. Harper should speed up the writing process rather than slow it down as Grammarly tends to.

To embody this original goal, I wanted to make the project a namesake of an author known for their literary devices and prose. I also had three other (practical) criterion:

1. It could not be occupied on crates.io.
2. It could not be occupied on GitHub.
3. It should not be more than two syllables.

I asked around for some ideas and someone suggested I name the project after Harper Lee. It fit the bill, so I went with it.

I am not one who enjoys complexity. I am also someone who likes to make their own tools.

As a student, I write a lot. That includes notes, papers, and documentation for my wide variety of projects.

A lot of classes request to submit assignments as either PDF, HTML or .docx. For a while, I submitted PDF's. I had a whole orchestrated setup.

I would write everything in Neovim, save it, and KNAP would render it using Pandoc. Finally, it would be rendered to my screen using Sioyek.

This worked fine, I guess, but it was far from perfect.

1. It was slow. Each edit I made in Markdown could take as many as 10 seconds to show up in Sioyek.
2. It wasn't interesting. While I made modifications to my Pandoc settings, my PDF's still looked like every other pdflatex document ever made.
3. I couldn't make my documents interactive if I wanted to.

Markdown was designed to be turned into HTML, I reasoned. So why not just do that?

That's why I created Tatum. It does one thing, really well. Point it at a Markdown file, and it will run a tiny web server to render the resulting HTML to. If the file changes, a WebSocket connection tells the browser to refresh.

Tatum renders in milliseconds and creates beautiful pages with $\KaTeX$, Simple.css and highlight.js. I can embed interactive HTML, CSS and JavaScript elements directly into my Markdown to get the interactivity and aesthetics I desire.

Once I'm done working, I just run tatum render <file.md> and I get a single file (images and all) that I can submit for my assignments.

Tatum isn't for you to use. Feel free to poke around at how it works, or even fork it and make your own modifications. It fits my use case perfectly.

The Beginning

Although it was several months ago, I remember it like it was yesterday. It was what I would call a dusty Colorado Friday. The sky was overcast and the trees depressingly barren. The air smelled of cow farts, which was typical when the winds changed, blowing in cold air from the beef farms in the nearby town of Greeley. In my mind, this signaled snow.

I walked into the Green Center, the building in housing the opening ceremony of BlasterHacks, the hackathon run by the Colorado School of Mines. I sign in at the entrance and walk down into the auditorium.

As I greeted my teammates, Grant Lemons, Lukas Werner and Byron Sharman, I noticed the nervous smiles on their faces. You know the kind, the smile with the inner eyebrows slightly raised, shoulders tense. We were excited, but we also knew we had 36 solid hours of programming ahead of us.

After collectively downing several whole pizzas and listening to the rules of the competition, we made our way over to Labriola Innovation Complex, the hours-old manufacturing building the competition would take place in.

The Plan

We brainstormed some ideas under the overcast sky:

• A Bennett Foddy-inspired battle-royale game where all the controls were vim-keybindings.
• A health insurance app that estimates the cost of your care at all local hospitals.
• An app that encouraged you to stand up and move every thirty minutes.

It was that last one that we all got excited about. We imagined it to be social: at a random interval (which would average to thirty minutes), the app would simultaneously send out a notification to every user. Should the user choose to participate in the StandUp moment, the app would guide them all through the same short workout activity. Finally, the user had the choice to upload "evidence" of the workout, which could be a short caption about what they were doing, or a photo.

The idea of StandUp (as we would go on to name our project) was inspired simultaneously by BeReal, recent research in the detrimental health effects of remote work, and the use of social liability for comparatively more extreme workouts (Strava).

Of course, most of this had yet to be nailed down yet, for we were just entering the Labriola Innovation Complex. I remember asking myself if I would leave before the competition was over. Although I was tired from the days classes and homework, I felt a drive to dedicate my whole self to this project.

The Board

We sat down and created a new git repository. That's when I started thinking about how we were going to organize our work. I proposed we setup a KanBan board, a project management technique I had picked up at Archytas Automation.

If you've never heard of--or used a KanBan board, here's the gist. There are three buckets:

1. To Do.
2. Doing.
3. Done.

The second bucket is divided up by person, so in our case, it was really four separate buckets. When someone needed something new to work on, they would go to the "To Do" bucket and select a new task (which in our case, was a sticky note). They would place the task into their corresponding "Doing" bucket. Once the task was done, they would move the sticky note into the "Done" bucket and repeat.

As a gag, we added an additional (superfluous) bucket: IDGAF. If a task was unanimously deemed no longer important or relevant to the project, that's where we would put it.

So, we got to work. We had decided to use Firebase, SvelteKit and Go. I started setting up Firebase, Grant started researching the notification service, Lukas started working on a circular status page and Byron started researching the Camera API.

As I look back at this moment today, I realize something important about all of these tasks that ended up benefiting us in the long term: they were all independent. None of these original tasks relied on any other to get done. We could work in parallel. Down the road, this would change, but the fact that we were parallel at the beginning set us up for success.

The Snow and the Crash

We kept working through the night, downing energy drinks and laughing as we worked. We got the important services and UI element up and running. Around midnight, as my nose had predicted several hours earlier, it started snowing.

As the snowflakes grew larger and ever more fluffy, I felt the teams energy begin to dip. We were encountering roadblocks, one-by-one, as our motivation waned and the threat of sleep became more prominent.

Then, someone in the main atrium (who, I don't remember) pronounced, "we are going outside." We ran down the halls like children spreading the announcement, and soon enough, there were dozens of tired computer science students throwing snowballs in their jeans the t-shirts.

As I threw (and received) a number of snowballs, I realized something important: we were making an app about taking care of yourself physically, but had neglected to do so ourselves. As important as the project may be, it won't get done when everyone is tired, muscles cramped, and unhappy. To keep the momentum going, breaks are required.

As we filed back inside to warm our hands under tap water, I swore to make sure the rest of the team would take care of themselves--and each other.

The Conclusion

The rest of the hackathon was uneventful. We sat at our laptops, cranking out line after line of code. We took breaks and slept when we were blocked by other tasks.

Although it was our first hackathon, we decided to run our project in the advanced track, normally reserved for hackathon veterans. It paid off, because we won second-place!

Look at us: haggard but satisfied.

There are a number of grand challenges my generation faces. Some threaten the way we live, like the housing crisis. Others are difficult to solve, like space travel. More still both threaten the way we live and are difficult to tackle, like climate change and pollution.

But of all the discussion of the problems of the modern world, one is often overlooked: how do we live happier, more meaningful lives.

I've found that there are two main potential answers which involve the way we sleep and the way we work. They are related to the ongoing brain damage of millions of Americans, social media and lighting. By attempting to solve this problem (that of happiness and productivity), all other grand challenges become easier to solve.

Unlike other grand challenges, like space travel, whose solutions lie in the hands of relatively few scientists and administrators, this problem of how to live well is the responsibility of both individuals and businesses.

Step 1: Sleep

As a high school student, I had an adversarial relationship with sleep. The way I saw it, every minute asleep was a minute of homework not done, a minute of friendship not cherished. A minute of life lost.

I went many nights completely forgoing it entirely, believing that it was merely a preview of death. In hindsight, I think it reflects well on my work ethic, but poorly on my work technique. The ancient Greeks seemed to agree with me. According to them, god of sleep was Hypnos, twin brother of death.

I couldn't have been more wrong, for sleep is incredibly important. In fact, I may have received mild brain damage for my decisions. Sleep is not mere lack of activity. In the book Why We Sleep by Matthew Walker, he discusses why sleep is critical for living a life containing depth of work and depth of being.

There are two types of sleep, and each are equally important.

NREM (Non-Rapid Eye Movement) Sleep

NREM sleep is critical in the integration and solidification of memory. This is the portion of sleep where your eyes are stationary, thus the name. In fact, NREM sleep seems to be the only time the brain actively works to retain memory. Since NREM sleep is more prevalent in the earlier hours of the night, going to bed two hours hours later than usual results in a 70% reduction in retention of factual information the next day.

REM Sleep

REM sleep is pivotal in creative thinking and emotional processing.

Walker models ideas as a knowledge graph. During wakefulness, traversal across this graph for the purpose of problem solving is somewhat isolated. In other words, people who are awake are only thinking of solutions or making connections that are conceptually similar to the existing solutions and ideas. During REM sleep, however, the brain makes finds longer-distances relationships, from much further-apart sections of the graph.

PTSD patients always have poor quality sleep from to nightmares. It isn't just a correlation, it is literally required in the diagnosis. Walker claims this is because the brain is attempting to process the negative emotions associated with trauma, but, since the emotions are so strong, wakes itself up. One of the main goals with cognitive behavioral therapy for PTSD patients is get the victim through a full night of sleep. By giving the brain the opportunity to properly process the negative experience (in REM sleep), they are able to dramatically speed up recovery.

Main takeaway: sleep more and you will live a longer, healthier, more impactful life.

The need for sleep varies across your lifetime less than many people think. The most desperate need is during infancy, with babies and toddlers requiring as much as 18 hours of sleep per day. As you get older, decreases until leveling off at eight and half hours of time in bed. No, that is not time asleep it is time in bed. It is a myth that seniors require less sleep. In truth, they require just as much sleep as in middle age, but have greater difficulty achieving it. This is primarily due to the correlated difficulty breathing, and secondarily due to the earlier bed-time seniors require.

There are a number of traits of the modern world that have resulting in the WHO diagnosis of sleep deprivation as a pandemic. More than 28% of Americans are chronically sleep deprived.

The Danger of Sleep Deprivation

Chronic sleep deprivation can result in brain damage. While you are awake, the glymphatic system (which is similar in function, different in implementation to the lymphatic system) is inactive. As a result, there is a gradual buildup of garbage in the brain, blocking neurotransmitters and generally disrupting the ability for neurons to communicate with each other. One protein that builds up: beta amyloid, the same protein that causes Alzheimer's disease. When you sleep, the glymph system starts up and cleans up the garbage wakefulness accumulated. However, if not given enough time to do its work (by sleeping), the glymphatic system is not able to do its job entirely. This is why your likelihood of getting Alzheimers increases the less you sleep.

Sleep deprivation also causes increased concentrations of cannabinoids. Just like Marijuana, these increases appetite for unhealthy foods. Additionally, sleep is an energy intensive process. The body burns more calories at night during sleep than during harmful wakefulness. All in all, there is a very strong connection between sleep deprivation and obesity.

Matthew Walker highlights one especially critical time when physical and mental health (especially memory retention) are important: school. The primary goal of school is to learn, to which NREM sleep is critical (due to its role in memory retention).

Starting in middle school, children's circadian rhythms start to shift later. In other words, teenagers naturally fall asleep and wake up later than other age groups. Walker attributes this to the need for teens to become more independent. Evolution pushes them to operate by themselves by forcing their parents to go to bed earlier.

Schools operate under the circadian rhythms of parents and teachers. 7 o'clock in the morning is the ideal time for them, but not for the students. But we find that students don't go to bed any earlier, even though they are forced to awake at the same time as their parents. The end result: millions of middle and high school students are currently experiencing chronic sleep deprivation, just because school starts too early.

The eating and physical habits developed during one's school years serve as the template for the remainder of their lives. If one spends their developing years under sleep deprivation (which is the case for millions of students in the U.S.) they are learning how to live their life under the influence of cannabinoids, all the while getting gradual brain damage from the buildup of beta amyloid.

Sleeping Pills

As an aside, I feel the need to address sleeping pills. Do not take them.

There are currently no medications on the market that induce real sleep. There are medications that slow and stop brain function, like Ambien. They do not provide real sleep, and as such do not result in memory integration, emotional processing or any of the other benefits of sleep. In fact, studies with Ambien show that induced "sleep" actually results in a net loss of memory, compared with not sleeping at all. Not only does Ambien stop natural memory integration, it actively causes forgetfulness.

Phones and Blue LEDs

The part of the brain that runs your circadian rhythm is called the superchiasmatic nucleus. This is an extremely small cluster of neurons that sits on the intersection of your two optic nerves. It watches for signals of blue light. If is any is detected (since the only significant, natural blue is from the sky) it can safely assume it is day and will slowly adjust itself under that assumption.

Phones and other sources of blue light mess with this process. If your eyes detected any at night, you are guaranteed to sleep worse.

There are other neurological effects at play. Most activities that take place on a phone activate the part of the brain called the amygdala, which we most commonly associate with the fight or flight response. When activated, your body prepares itself for survival. It slows or halts digestion, dilates the pupils and increases metabolism. In other words, prepares your body for action, not rest.

The issue: one of the most commons staples of modern office spaces are the rows upon rows of florescent lights. All people working in buildings outfitted with these lights may be experiencing worst quality and quantity of sleep as a result, especially those working later or longer hours.

In the search for producing deep work and living a full life, getting enough sleep is paramount. It is the single most effective move one can make, since it makes all other actions more effective.

However, as I previously pointed out, success in the new economy relies particularly on deep work, which requires deep focus.

Step 2: Deep Focus

At the beginning, I mentioned there are two problems restricting the ability for people to live happy, effective lives. The first was related to sleep, the second was related to deep focus.

Before I can explain how to achieve deep focus (according to Cal Newport), I first need to explain why it is so critical.

The Information Age

We like to separate history into ages. The age of the dinosaurs was followed by the ice age which was followed by the human age (the Anthropocene). Granted, that's an oversimplification, but it highlights the point: we delineate swaths of time with the most important and unique variable at play.

The industrial age began with the widespread use of the steam engine. With steam engines came factories, and with factories came factory workers.

We live in a new age, identified by the integration and advancement of computer and communication technology. We call it the information age. With the information age comes information or knowledge workers.

This is the subject of Cal Newport's fantastic book, Deep Work: Rules for Focused Success in a Distracted World.

Knowledge Work

Knowledge work is defined by the need for a depth and breadth of knowledge, and a high ability to manipulate it. Programming, writing, accounting, and teaching are all examples of knowledge work. They require enormous amounts of knowledge and the value the provide is accomplished through thought.

Cal Newport makes the case that the globalization of knowledge (via the internet) has initiated an economic shift that will benefit just a few knowledge workers, the most skilled superlinearly relative to the rest.

Before the internet, any given company had a limited pool of potential workers: those in the geographic vicinity. This meant, if a company wanted to hire the best programmer for a job, they would have to make do with what was available.

Today, however, if a company wants to hire the best programmer, they have the entire world's pool. Knowledge work doesn't typically have a strict on-site requirement. Companies also have much more to gain from getting the best than getting pretty good, so they're willing to pay much more. The end result: the people at the top of their field are in higher demand now than ever before.

Cal Newport identifies two key traits in these "superstars" at the top of their field. First, "the ability to quickly master hard things." In other words, learn complex and applicable things.

Second, "the ability to produce at an elite level, in terms of both quality and speed."

All of this is to say: to be successful in the changing economy, people need to become better at deep, meaningful work. Since the most valuable resources are the result of deep thought, it makes sense that the highest achievers are those who practice deep focus.

In Deep Work, Cal Newport highlights a number of strategies for achieving true, deep focus. However, only one is broadly applicable for most people. The others require an often unreasonable amount time set aside, something that most employers do not allow, unfortunately.

Rhythmic Deep Work Scheduling

This is most effective strategy for achieving deep focus for most people. There are others Newport highlights in his book, but this is the most important. Particularly, most people have daily obligations that halt them from deeply focusing for more than 22 hours. This could be due to your job, family, or school.

Either way, a number of people have centered on a seemingly simple solution: make a habit. Instead of forcing yourself to make a decision to move into deep work, regularly schedule it.

The brain has a limited number of deep concentration hours per day (I'm not sure why, but I suspect it is related previously described buildup of proteins in the brain). Use them up front, when you are least likely to be distracted: in the morning.

This is something I've adopted. I schedule all my classes in the afternoon and most social encounters naturally happen in the evening. This leaves the entire morning dedicated to deep focus. I put my phone out of sight and work, undistracted, from 5:45 AM to 11:00 AM.

Since implementing this habit, I've found that I'm able to complete the majority of my work twice as fast, leaving more time for other pursuits, like Harper.

The Open Office Plan

The open office plan it an example of a general trend in many modern companies. Managers believe the largest barrier to productivity is a lack of communication. In theory, knowledge workers will be able to complete work faster if they are able to share knowledge faster.

For example, a junior accountant needs to perform an operation in Excel to tabulate a company's expenses for this year. This process, with the knowledge they currently have, will take three hours to perform. They have to write the formulas, import in the data, and present it in a graph.

A senior accountant, however, if given the same job, would know that there is a single button in Excel that does all the same operations in five minutes.

The friction at play, is that the senior accountant has more knowledge of Excel capabilities than the junior. If they could communicate this knowledge, the junior's work could be completed much faster. This is why managers believe that increasing communication increases productivity.

However, most attempts to increase productivity have a negative effect on the ability for knowledge workers to focus, which results in a net decrease of productivity in the team.

For example: the open office plan. Sure, there may be instances where a senior accountant sees that a junior needs help on something, but if any two people are talking elsewhere in the room, their noise disrupts the focus of every single worker present.

Hub and Spoke Model

There is one tried and true method of increasing deep focus without sacrificing communication: changing the way our buildings are designed.

The Bell Labs Holmdel Complex is a fantastic example of this. At its peak, it was filled with some of the greatest minds of a generation. Ten Nobel Prizes, five Turing Awards, and twenty-two IEEE Medals of Honor were awarded for work done in this building

The Holmdel Complex was designed like with a Hub and Spokes. Each worker had their own private office, so they could work completely uninterrupted. You can think of each office as a spoke on a wheel. However, if they needed to go anywhere else in the building, they had to enter the hub: a large hallway that connected all rooms in the building. This meant that chance encounters with other workers were commonplace, but only when a worker wasn't actively trying focus. They retained the benefits of both deep work and communication.

These chance encounters gave rise to unexpected applications of each individual researcher's work. The people working on Unix discovered they could help the information theory researchers and vice versa.

Conclusion

Let us return to great challenge I'm trying to highlight: the question of how to live happier and more effectively. The answers are surprisingly simple.

We need to schedule our schools and workplaces to account for the needs of sleep for our workers and students. Students that remember more material become more effective problem solvers later on, which makes all other problems easier to solve.

The people who are attempting to solve other great issues, like unequal access to education, climate change, and pollution will be able to work more effectively given the opportunity to enter deep focus.

So please, sleep well, and don't be distracted by the things that don't matter.

Works Cited

[1] M. Walker, Why we sleep: Unlocking the Power of Sleep and Dreams. Simon and Schuster, 2017.

[2] C. Newport, Deep work: Rules for Focused Success in a Distracted World. Hachette UK, 2016.

For the last month, I've been spending a lot of time replacing one key component of my writing and programming environment: my grammar checker.

Up until now, I've been using the eponymous LanguageTool via ltex-ls and nvim-lspconfig. Don't get me wrong, these tools are really good, and I would recommend them to anyone and everyone. However, they come with a few key annoyances.

LanguageTool Grievances

Performance

LanguageTool is slow. I'm not exactly sure why. Every time I would run LanguageTool over my Markdown or $\LaTeX$ documents (which are reasonably sized), I would have to wait several seconds before even the rudimentary spell-checking would show up.

Additionally, I would find ltex-ls regularly becoming the most memory-hungry application on my laptop, often exceeding 4 gigabytes.

After hours of scouring their code base, I have come to no better explanation than that it is written in Java. There are a couple questionable algorithmic decisions in there as well.

Download Size

As I said: LanguageTool is really quite good. However, to get everything it can offer, you need to not only install a Java Runtime Environment (>150 MB on my system), the actual .jar file (>300 MB), but you also need to download a 16 GB n-gram dataset.

Grammarly Grievances

"But Elijah," I hear you say, "just use Grammarly!"

No. I'm not going to drop $12 a month for something even slower and worse. Not to mention how they are likely going to use my work to train their large language models. Grammarly is a great product, just not for me.

The Algorithm

Now that I've thoroughly explained my reasoning for implementing a new grammar checker (one that I'm calling Harper), I'd like to recount my first, admittedly naive, attempt at spellchecking.

The first idea we need to get a grip on is Levenshtein edit distance. In essence, edit distance is the least number of single-character edits (insertions, deletions or replacements) necessary to turn one word into another. For example, the edit distance between "cat" and "bat" is one; the only edit involves replacing the "c" with a "b".

Similarly, the edit distance between "kitten" and "sitting" is three: remove the "g", replace the second "i" with an "e" and replace the "s" with a "k". For this naive spellchecking, we aren't too concerned with the exact edits (atomic errors) that occur in a given misspelling, only the magnitude of the error.

From a high level view here's how the algorithm is going to work:

1. Determine whether a given word is misspelled. If not, exit.
2. Calculate the Levenshtein edit distance between the misspelled word and all valid English words.
3. Pick the three words with the shortest edit distance and present them to the user as alternative spelling options.

Step 1.

To determine whether a given word is misspelled, we will need a list of all the valid words in the English language. Turns out, this isn't too easy. For today, we will just use a subset of the English language with this short list:

into
the
a
cat
tree
jumped

To check if a given word is within the list, we can place the list into a hash set, and grab it's contents.

let words: Vec<String> = vec!["into", "the", "a", "cat", "tree", "jumped"]
    .into_iter()
    .map(|s| s.to_string())
    .collect();

let word_set: HashSet<String> = words.iter().cloned().collect();

let word = "thw";
let word_chars: Vec<_> = word.chars().collect();

if word_set.contains(word) {
    println!("It is a valid English word!");
    return;
}

println!("Are you sure you meant to spell \"{}\" that way?", word);

The Wagner-Fischer Algorithm

Now that we know our word is actually misspelled, we can move on to finding the correct spelling. We need to find the edit distance between the misspelled word and all the words in our set.

To do this, we will be using the Wagner-Fischer algorithm.

// Computes the Levenstein edit distance between two patterns.
// This is accomplished via the Wagner-Fischer algorithm
fn edit_distance(source: &[char], target: &[char]) -> u8 {
    let m = source.len();
    let n = target.len();

    // Create an m-by-n matrix.
    let mut d = create_empty_matrix(m + 1, n + 1);

    // Since we know we can transform each word into the other by replacing
    // successive characters (or deleting them), we can fill the first column and
    // row with values from 0..m and 0..n, respectively.
    for i in 0..=m {
        d[i][0] = i as u8;
    }

    for i in 0..=n {
        d[0][i] = i as u8;
    }

    for j in 1..=n {
        for i in 1..=m {
            // The total edit distance of two given letter indices i and j, one from each word
            // will be the sum of the edit distances of prior combinations + whether the characters
            // at the two indices are equal.

            let cost = if source[i - 1] == target[j - 1] { 0 } else { 1 };
            d[i][j] = (d[i - 1][j] + 1)
                .min(d[i][j - 1] + 1)
                .min(d[i - 1][j - 1] + cost);
        }
    }

    // After all possible edits have been explored and minimized
    // the resulting minimum edit distance will be in the final item in the matrix.
    d[m][n]
}

// Create an empty matrix of size [m, n]
fn create_empty_matrix(m: usize, n: usize) -> Vec<Vec<u8>> {
    let mut d = Vec::with_capacity(m);

    for _ in 0..m {
        d.push(vec![0u8; n]);
    }

    d
}

This works pretty well. There are a number of optimizations we could apply to this function alone. I'll leave that as a problem for the reader, since they aren't particularly relevant to the meat of the larger algorithm.

Steps 2 + 3

Now that we can determine the edit distance between two words, we can perform a brute-force search. In this short example, we're going to use sort_by_key to do this, since our data set is so small. If we were working with a larger dictionary (say, the entire English language), there would be a number of things we would need to do to reduce time and memory consumption.

let mut suggestions: Vec<(String, u8)> = words
    .into_iter()
    .filter_map(|possible_word| {
        let possible_chars: Vec<_> = possible_word.chars().collect();

        let dist = edit_distance(word_chars.as_slice(), &possible_chars);

        if dist <= 2 {
            Some((possible_word, dist))
        } else {
            None
        }
    })
    .collect();

suggestions.sort_by_key(|(_, d)| *d);

println!("Possible alternatives: ");

suggestions.iter().for_each(|(s, _)| println!("- {}", s));

If we run the whole program again, we get an output something like:

Are you sure you meant to spell "thw" that way?
Possible alternatives:
- the

That looks pretty good to me!

If you would like to look at the whole program, and maybe try out your own inputs, go right on ahead.

Heads Up: This article is a republished (with some tweaks on spelling, grammar and layout) version of an article I wrote in my senior year of high school for my Linear Algebra class. As such, the publish date is not quite correct.

Computers are fantastic at processing, producing, and consuming data. But I've often found that the most difficult part of the pipeline is representing the data to a user. This can often take the form of bar charts or scatter plots, but there are situations where they just don't fit the bill.

3D graphics enable developers to create interactive programs that appear most similar to the natural world. By presenting a three-dimensional space, the barriers for entry drop. I wanted to learn more about how this worked.

Inspiration

For a long time, I've been told that the most prevalent application of linear algebra was computer graphics. Before I even began my study on linear algebra, I knew I wanted to get into software rendering.

One of the big roadblocks was the amount of technical know-how I thought it required. You see, most 3D programs do all the number-crunching on the specially designed graphics processing unit that is readily available on most modern computers. From my previous attempts to use GPUs, I knew setting up the pipeline is quite involved. If I went that route again, I know I would likely spend most of my time dealing with vendor-specific APIs.

Since I wanted to focus on the math, I postponed the project. That is, until it occurred to me that I could simply not use the GPU. I know it might sound obvious, but it felt so freeing at the time.

What Is a Software Render Engine?

A render engine is a piece of software that takes a set of triangles in space and projects them onto a 2D grid that can be displayed on a computer screen.

A software render engine is, as it may sound, a render engine that does all computation in software. No specialized hardware is utilized at all.

Demo

Before I get into how it works, I want to give you the chance to try it out yourself. I've created a very simple scene to demonstrate.

Explanation

Note

In this article, I intend only to talk about the math related to the problem. If you are interesting in the nitty-gritty how lines and shapes get drawn to the screen, I suggest you read up on Bresenham's line algorithm, Xiaolin Wu's line algorithm, and Drawing Filled Triangles

Projection

With a perspective camera, projection happens from the world, towards the "sensor," which is a defined point. For the sake of argument, let's say that point is at the origin and the camera is facing the positive $z$ axis.

We want all other points to be projected onto a plane $d$ distance away. If you are a photographer, $d$ will be the focal length of the camera.

Orthographic Projection

In the case of orthographic projection, this is easy. Because we have placed the camera on the origin, facing the positive $z$ axis, we can draw the coordinates of any given point directly to the screen. The only consideration necessary pertains to the points behind the camera, which we can skip by checking the sign of the $z$ component.

Homogeneous Coordinates

This is where homogeneous coordinates come in. When working in euclidean space, we represent a given vector or coordinate using three components:

When we are working in projective space, we can represent any given vector or coordinate using four components.

We can convert between these formats interchangeably. To convert a homogenous coordinate to euclidean, divide all other components by $w$:

We can transform any euclidean coordinate to a homogenous coordinate by setting $w = 1$

Just like we can perform various transformations on euclidean coordinates, we can perform similar ones on homogenous coordinates. The major difference: instead of requiring conventional additions or subtractions, homogenous coordinates can be translated via matrix multiplication.

For example, let's say we have a point at the origin, and we want to perform both a translation and rotation. If we were using euclidean coordinates, we would have to translate it via addition, then rotate it separately.

In homogenous coordinates, we can do it with a single matrix operation by preparing the matrix ahead of time:

By condensing the entire transformation into a single matrix, we are able to save a ton of computing time.

Perspective Projection

The essential idea of perspective projection is simple: we want points further from the camera to appear closer to the $z$ axis the further away they are. Remember that $d$ is the surface we are projecting onto. This is possible with homogenous coordinates with the following matrix:

Assuming you have an understanding of matrix multiplication, it should be apparent why this works. When the $w$ component of the matrix is being computed, the $z$ component will be divided by $d$. The result then becomes a divisor of $w$, which affects all components of the resulting vector due to the nature of homogenous coordinates. In short: $w \leftarrow w * (-z / d)$

Color

Now that we've established exactly how to project points in space onto the screen, we need to start coloring in triangles. As I said before, I am not going to go into the algorithms that do this. I want to discuss how to determine the color to fill in.

We could just choose one solid color. As you can see from the demo (by pressing R), this doesn't lead to a particularly impressive or visually pleasing result. I want an additional way to convey depth.

Given the three points that make up a triangle $p_1$, $p_2$, and $p_3$, we can find its normal vector (the vector perpendicular to it's surface), $\vec{n}$ fairly easily.

Note: the vertical bars around a vector $||\vec{v}||$ signify getting the vector's length.

Now that we have the triangles normal, we can fill it in more brightly depending on how directly it is facing the camera.

The resulting shading is the default in the demo.

Alternatively, we can also simply color based on the distance from the camera. The resulting image is called a depth map.

Sorting

When the program is supplied a mesh, the faces are not in any specific order. If we were to just draw each face in the order it arrives, nothing would make sense.

To solve this, we simple sort each face by it's distance, then draw the furthest faces first.

If you want to see what it would look like, go to the demo and press O to toggle face sorting.

Optimization

There are countless ways to optimize a renderer like this. They all involve work-avoidance. The one I want to discuss is often referred to as backface culling.

In most situations, there is no need to see the inside of a mesh. This allows us to avoid a lot of work for very little effort. By checking the alignment of the point-to-face vector with a face's normal, we can check if a given face is facing toward us or not.

In the demo, you can toggle backface culling with B.

Conclusion

When I initially designed this project, I hoped it would allow me to apply some of the more advanced linear algebra concepts that I've learnt in the second trimester. In this regard, it did not live up to my expectations.

While I was allowed to explore some concepts, like orthogonality, it was not quite satisfactory.

It was not for naught, though. I learned a lot about the fields of math and computers integrate together, as well as how to more effectively convert mathematical concepts into a working prototype. I want to continue doing projects like this, and cannot wait to re-take Linear Algebra when I go to college.