Combining Human-in-the-loop and Loop Engineering in Memolog's AI Development

When developing Memolog, I do not start by defining a detailed specification and completion criteria, then ask GitHub Copilot (Copilot) to implement them exactly.

Instead, I begin with a conversation: “This feature might be useful,” or “I want to make this experience a little better.” I ask Copilot to create something that feels about 80% of the way there, then I try it myself. If it feels wrong, I ask for changes. If it differs from what I imagined but works better, I adopt it.

I used to think of this entire process as Loop Engineering. But when I compared it with Addy Osmani’s explanation of Loop Engineering and the Japanese article “From Introduction to Practice: Loop Engineering,” I realized that I had conflated the conversational loop between people and AI with the system that lets AI operate autonomously.

This article reframes Memolog’s approach as a double loop: an outer Human-in-the-loop discovery loop and an inner Loop Engineering implementation and validation loop.

Table of Contents

• Key Takeaways
• Distinguishing the Three Terms
• Memolog’s Double Loop
• Discovering the Answer Outside, Converging on It Inside
• Why I Do Not Define 100% Up Front
• Six Loops at Memolog
• Why One-on-One Is My Default
• Blog Writing Has Loops Too
• Setting Observable Checkpoints
• Boundaries That Keep the Loops Safe
• Summary

Key Takeaways

Memolog’s development process does not rely on just one kind of loop.

• The outer loop is Human-in-the-loop: People and AI talk, examine prototypes, and discover what to build and how to evaluate it
• The inner loop is Loop Engineering: AI autonomously repeats implementation, testing, fixes, and retesting to reach an adopted goal
• People decide at critical boundaries: AI does not make decisions about security, cost, data deletion, or release on its own

This is the basic flow at Memolog:

People and AI discuss a loosely defined idea
  ↓
AI turns it into something about 80% of the way there
  ↓
Review it and try it firsthand
  ↓
A person decides whether to adopt, revise, or set it aside
  ↓
AI repeats implementation and validation until it reaches the adopted goal
  ↓
A person reviews and releases it, then feeds observations into the next improvement

I do not lock down every completion criterion at the outset. I define non-negotiables such as security, data protection, and preserving existing functionality in advance. For the UI, copy, interaction design, and presentation, I leave room to decide after seeing a prototype.

In this model, Human-in-the-loop and Loop Engineering are not competing approaches. People define the purpose and retain responsibility, while AI autonomously performs verifiable work inside those boundaries.

Distinguishing the Three Terms

For this discussion, I distinguish Human-in-the-loop, Human-on-the-loop, and Loop Engineering as follows. I am not claiming that they share a single universal standard; this is simply a framework for describing how I work at Memolog.

I do not describe Memolog’s overall development process as Human-on-the-loop because I am not merely a supervisor who intervenes only when something goes wrong. I participate in the loop by examining prototypes, changing my mind, putting my preferences into words, and deciding what to build next. During the inner implementation loop, however, I supervise the results without issuing step-by-step instructions, which is closer to Human-on-the-loop.

The official Microsoft Agent Framework documentation also uses Human-in-the-loop to describe a pattern in which an agent requests human approval before taking an action, then proceeds after the person approves or rejects it.

Addy Osmani describes Loop Engineering as moving away from prompting an agent step by step and instead designing the system that runs the agent. After receiving a goal, AI iterates until it meets the completion criteria.

Under that definition, repeatedly conversing with Copilot is not Loop Engineering by itself. At Memolog, I treat conversational exploration as Human-in-the-loop and the autonomous implementation and validation system within it as Loop Engineering.

Memolog’s Double Loop

Different participants drive the outer and inner loops.

Human preferences and a sense that something feels off matter in the outer loop. Machine-observable conditions such as builds, tests, lint, Playwright, and API responses matter in the inner loop.

The goal is not to automate everything. It is to explore work with people while the right answer is still unknown, then hand repeatable work to AI and automated systems once the answer has been chosen.

Discovering the Answer Outside, Converging on It Inside

The Outer Discovery Loop

This loop runs while I am still figuring out what I want.

Conversation → Prototype → Try it → Articulate what feels off and what I prefer → Prototype again

Success is not measured only by how closely the result matches my initial request. “This is different from what I expected, but it is better” is also a successful outcome. The result gives me something concrete that can reshape my thinking.

This is Human-in-the-loop. I am not there only to approve the final result. I use the deliverable to think and decide what direction to take next.

The Inner Implementation and Validation Loop

Once the outer loop establishes a direction, Copilot handles implementation and validation.

Implement → Test → Read the cause of the failure → Fix → Retest

I do not need to provide a new instruction after every test failure. AI reads the result, identifies the cause, fixes it, and runs the test again. I also define observable stop conditions in advance, such as all checks passing, reaching a retry limit, or repeating the same failure.

This is Loop Engineering at Memolog. Rather than locking down everything while I am still exploring, I keep discovery flexible in the outer loop, then harden only the necessary parts in the inner loop after I adopt a direction.

Why I Do Not Define 100% Up Front

I am not good at writing detailed completion criteria before implementation, and seeing a working result often changes my mind.

That is why I am comfortable starting with something that is about 80% of the way there.

This does not mean leaving everything vague. I divide conditions into two groups.

For example, it is difficult to define a “natural and easy-to-use modal” precisely before building one. It is still possible to verify that the modal opens and closes, renders correctly on mobile, and lets someone continue interacting with the page after closing it.

I first have Copilot build something that works, then try it myself and describe my preferences or points of friction. Once I adopt an interaction, I preserve it in a Playwright test. That gives me room to explore without giving up regression protection.

Six Loops at Memolog

Looking back at how I work, I see several connected loops in Memolog.

These loops do not always move in one direction. A problem found during validation sends the work back to implementation. A new idea prompted by the production site sends it back to discovery.

In an earlier article, “How GitHub Copilot and I Divided the Work of Building Memolog,” I described how Copilot and I went back and forth through research, design, implementation, testing, production verification, and recordkeeping. This double-loop model reframes that division of responsibilities by identifying where people participate and where AI operates autonomously.

Why One-on-One Is My Default

One-on-one work and Loop Engineering are not in conflict. I use one-on-one conversations for the outer discovery loop, then let Copilot run only the inner loop autonomously after we settle on a goal. Loop Engineering does not require multiple AI agents, either. What matters is not the number of agents but whether the system lets AI iterate until it meets the completion criteria.

Memolog is a relatively small repository maintained by one person. My usual workflow is a one-on-one conversation with a single Copilot session.

• It is easier to preserve the flow of the conversation and recent decisions
• It is easier to communicate how a UI feels or what nuance I want an article to convey
• Even when a change spans the web app, API, Azure, and CI, I can follow it in one context
• Responsibility for the final decision is less likely to become ambiguous

In a small repository, otherwise separate tasks often converge on the same page, shared component, or workflow. Running multiple agents in parallel creates integration work: transferring context, resolving conflicting decisions, comparing diffs, filtering duplicate changes, and coordinating merge order. At Memolog, that cost often outweighs the benefit of parallel execution, which is another reason I default to one-on-one work.

I use an independent subagent for English translation because I cannot confidently judge its quality on my own.

Reviews of English drafts have also highlighted six points that help preserve my voice while reducing translation-like phrasing.

• Avoid treating organize as a catch-all verb
• Avoid repeatedly starting sentences with So
• Turn nominalizations back into verbs where practical
• Remove unnecessary can carried over from Japanese “be able to” constructions
• Split a long sentence in two at a meaningful boundary
• Reorder cause and effect to sound natural in English instead of preserving Japanese word order

These are review perspectives, not mechanical replacement rules. The goal is to preserve my claims and voice while giving the prose a structure that English readers can follow more easily.

I explain this workflow in more detail in “How Memolog Translates Japanese into Natural English.”

Most work stays one-on-one between me and Copilot. I add a subagent only when I need an independent perspective. The subagent is not the final decision-maker; its role is to provide input that would be difficult to obtain from me and the main Copilot session alone.

Blog Writing Has Loops Too

Human-in-the-loop exploration is not limited to code. I did not begin this article with the entire structure already decided.

I started by discussing how I work with AI in chat and working through a series of questions and answers. That process helped me put the following ideas into words.

• I would rather judge a working result than lock down every completion criterion at the outset
• If the result differs from what I imagined but is better, I want to accept it
• I want to see a prototype that is roughly 80% of the way there as soon as possible
• I want one-on-one work to remain the default
• I want to delegate English translation to an independent subagent because I cannot judge its quality myself
• I want checkpoints to inform the next decision, not merely serve as quality checks

Article writing also divides into outer and inner loops.

I do not begin with a finished answer in my head and ask AI to transcribe it. Dialogue and drafts also help me discover what I want to say.

Setting Observable Checkpoints

Checkpoints serve different purposes in the outer and inner loops. In the outer loop, I ask, What does a person need to see to make the next decision? In the inner loop, I decide, What conditions tell AI to continue iterating, and what conditions tell it to stop?

When I start a task, Copilot and I consider four questions together.

1. What can I inspect along the way to correct the direction?
2. If something breaks, what signal will reveal it?
3. Which parts can automated checks verify repeatedly?
4. Which parts should I try myself and judge at the end?

I often use checkpoints like these at Memolog.

Here, automated checks include builds, tests, lint, Playwright, CI, telemetry, and other systems that return a result without requiring a person to repeat the same steps every time.

An independent subagent serves a different purpose. At Memolog, subagents mainly edit and review English translations whose naturalness I cannot assess confidently. Their feedback informs my decision; I decide whether to apply it and publish the article.

Observations in the outer loop do not produce only a pass or fail.

• Adopt the result because it matches what I expected
• Revise it because it does not
• Adopt it because it is better, even though it differs from what I expected
• Gather another option or more observations because I do not yet have enough information to decide

Each outcome is a useful checkpoint because it determines the next action.

Boundaries That Keep the Loops Safe

Faster iteration does not mean giving AI every decision and permission. I keep the following boundaries at Memolog.

• Create a GitHub Issue before changing the repository or Azure
• Keep human control over decisions involving security, cost, data deletion, and release
• Start with targeted tests after implementation, then expand to the full test suite as needed
• Verify GitHub Actions and the production state, then record the results in the Issue
• Return to the assumptions or design if the same fix repeatedly fails
• Run independent review and synchronization checks before publishing the Japanese and English versions

An Issue is not only a place to lock down a complete specification at the beginning. It also records what I tried, where my thinking changed, and what I verified before adopting a result.

Loop Engineering autonomy does not mean allowing the system to run without limits. It is essential to design where the system stops automatically and where it hands control back to a person.

Summary

Memolog’s AI development process uses a double loop: Human-in-the-loop and Loop Engineering.

• In the outer loop, I work one-on-one with AI to build a prototype that is roughly 80% of the way there and discover what I want
• In the inner loop, AI repeats implementation, testing, and fixes to reach the adopted goal
• Outer-loop checkpoints help people decide what to do next
• Inner-loop checkpoints tell AI whether to continue, finish, or stop
• People retain responsibility for security, cost, data deletion, release, and other critical decisions
• I add an independent subagent in areas where I cannot judge quality on my own
• Post-release results feed back into the next outer discovery loop

Even when I cannot define the right answer in advance, Human-in-the-loop lets me discover it by examining deliverables. Once I decide on the answer, Loop Engineering lets AI iterate on implementation and validation.

People do not need to perform every step themselves, nor should AI be entrusted with everything. People own the answer and the responsibility; AI owns the iterations that move the work toward that answer. That is the double-loop model that fits Memolog today.