agent-03 · What AI agents change about research work — Lee Hung-yi-inspired teaching style

All right, everyone, let's begin with one question: if an AI agent can draft an analysis in an hour, does that mean it can replace the researcher?

The tempting answer is yes: AI writes papers, humans go home. Very efficient. Also very wrong.

In one sentence: agents make first passes cheap, but they do not make truth cheap; the bottleneck moves from producing output to verifying evidence, choosing feasible ideas, and deciding what matters.

Here is our roadmap. First, we follow AI from tool to coworker to bounded worker. Then we split research into its actual stages and watch the naïve “automate the whole thing” story fail. Finally, we locate the parts where human judgment becomes more—not less—important.

Style note: this is a Lee Hung-yi-inspired teaching-style rewrite. It was not written, reviewed, or endorsed by Lee Hung-yi.

Builder takeaway: agents do not replace research as one blob. They lower the cost of drafts, code, reviews, and analysis, which makes verification, problem selection, and feasibility judgment more important.

From tool to coworker to worker

Start with the role, because the role tells us what kind of failure to expect. A tool waits for one command; a worker carries a bounded task through several steps.

The lecture starts with a useful progression.

At first, AI is a tool. You give one command, it does one thing.

Then AI becomes a collaborator. You discuss, iterate, and co-produce.

Now agents are starting to look like workers inside bounded tasks. Give them a goal, a repository, some tools, maybe a paper draft, and they can run for a while.

That last step changes the shape of work. It is one thing to ask a model to summarize a paper. It is another thing to ask an agent to inspect a dataset, modify code, rerun an experiment, write up the result, and check whether the output matches the original question.

The second version is not just generation. It is a workflow.

The 100x research assistant example

Now try the naïve economic argument: if the first analysis is 100 times cheaper, research must be 100 times easier. The missing term is checking cost. Cheap output is not the same thing as cheap truth.

One example in the lecture involves Claude Code writing a paper-like analysis using an existing paper's method and new data. The result was not perfect. It made at least one data mistake. But the cost and time comparison was striking: roughly an hour and a small amount of money versus a human research assistant spending far longer.

The lesson is not that the AI result is automatically better.

The lesson is that the first draft of many research tasks is becoming much cheaper.

This shifts the economic question. Instead of comparing "AI alone" against "human alone," the relevant workflow may be:

AI generates first pass
human checks errors
AI revises
human decides if it matters

If the checking cost is low enough, this beats doing everything manually. If the checking cost is high, or if errors are subtle and dangerous, it may not.

That distinction matters. Agents do not eliminate verification. They make verification more central.

Research is not one task

Asking whether AI can “do research” is like asking whether a machine can “run a restaurant.” Which part—taking orders, cooking, buying ingredients, or deciding the menu? We have to open the box.

The lecture breaks research into pieces, implicitly and explicitly:

• finding literature
• understanding prior work
• proposing ideas
• designing experiments
• writing code
• running models
• analyzing data
• writing the paper
• reviewing the paper
• teaching the result

AI is not equally good at all of these.

It is already useful for literature triage, code edits, drafts, and analysis. Autoresearch loops show agents changing training scripts, running experiments, looking at results, and trying again. That is real leverage.

But idea generation is harder.

The lecture discusses work where LLM-generated research ideas scored well on novelty, excitement, and overall impression, but worse on feasibility. That is exactly the failure mode I would expect. Language models are very good at combining concepts into something that sounds new. The real test is whether the idea survives contact with implementation.

A beautiful research idea that cannot be run, measured, or falsified is mostly decoration.

The feasibility bottleneck

Here the naïve idea generator meets reality. A proposal can sound novel while having no dataset, no meaningful metric, or no detectable effect. Fluency produces a pitch; feasibility determines whether the pitch has bones.

This is probably the most important part of the lecture.

AI can generate many ideas. Some will look novel. Some will even be genuinely useful. But feasibility is where human taste still matters.

Can this experiment actually be run?

Is the dataset available?

Will the metric answer the question?

Is the expected effect large enough to detect?

Is this a real problem or just a clever recombination of terms?

These questions require domain judgment, not just fluency.

The lecture's follow-up example, where AI ideas looked worse after being implemented into short papers, is a good warning. Idea quality cannot be judged only at the pitch stage. Execution reveals whether the idea has bones.

AI reviewers are coming, but that is not the real issue

The naïve debate asks, “Was this review written by AI?” The more useful systems question is, “Did the review find a real problem, explain it concretely, and leave someone accountable for the judgment?”

The lecture then moves to AI review. AAAI 2026 is mentioned as an example where AI reviewers and AI meta-reviewers become part of the process.

This will make people angry, but the lecture's position is more nuanced than "AI review bad."

Bad review is bad.

If a human reviewer pastes a lazy AI-generated review that misunderstands the paper, that is bad because it is useless. If an AI reviewer points out a real flaw, gives a concrete suggestion, and helps the author improve the work, that is useful.

The question is not whether the text came from AI. The question is whether the review does the job.

That said, the system needs accountability. A review process cannot become a chain of agents producing confident artifacts that nobody owns. Someone needs to decide whether the criticism is valid.

Deadline-aware review is an underrated idea

An evaluator without a clock can be correct and still be useless. Advice has to fit the time remaining in the human workflow.

One of my favorite practical examples is the lab agent that reviews student papers with awareness of the deadline.

If the deadline is far away, it can suggest more experiments.

If the deadline is tomorrow, "run three new ablations" is not helpful. The agent should focus on clarity, framing, related work, and changes that can actually be made before submission.

This is a small example, but it captures what good agent design looks like. The agent is not just evaluating in the abstract. It is evaluating inside a human workflow.

A technically correct suggestion can be useless if it ignores time.

AI Agent for Science and the partial loop

Once proposing, experimenting, writing, reviewing, and revising are connected, the diagram looks like a closed science loop. But do not let a closed arrow fool you: every edge can have a different reliability.

The lecture also discusses AI Agent for Science, an experimental setup where AI is a major contributor and AI review is part of the process.

This points toward a possible research loop:

AI proposes
AI experiments
AI writes
AI reviews
AI revises

It is tempting to call this a closed loop of scientific progress.

But the details matter. Accepted work still appeared to involve more human input in the early stages: idea selection and experiment design. AI was stronger in data analysis and writing.

That division feels right. Later-stage tasks have clearer local feedback. Early-stage research taste is harder to specify.

The human job moves around

Whenever automation makes one stage cheap, scarcity moves elsewhere. Here, drafts become abundant and judgment becomes scarce.

The strongest version of the lecture is not that humans become irrelevant. It is that human work moves.

If AI makes drafts, code edits, and routine analysis cheap, then human value shifts toward:

• choosing problems
• setting research taste
• designing experiments
• checking validity
• deciding what is worth publishing
• accepting responsibility for errors

This is not necessarily comforting. Some jobs are built around doing the pieces that AI is making cheap. But it is more accurate than saying "researchers are replaced."

Research was never just writing papers. At its best, research is choosing good questions and forcing reality to answer them.

What I would change in my own workflow

The practical strategy sounds paradoxical: use agents more, trust their outputs less. There is no contradiction if agents produce candidates while humans own verification and choice.

After this lecture, I would use agents more aggressively for first passes, but I would not trust them more.

That sounds contradictory. It is not.

I would let an agent:

• summarize a literature cluster
• draft experiment code
• create a baseline implementation
• generate tables and plots
• write a first related-work section
• review a paper for obvious gaps
• run a checklist before submission

But I would keep human control over:

• what question matters
• whether the experiment tests the claim
• whether the result is real
• whether the paper should exist

The point is to spend less human time turning the crank and more human time deciding which crank is worth turning.

The main lesson

Let us put the whole lesson back into one picture: automation does not flatten the research pipeline. It changes the price of each stage by a different amount.

AI agents are not a single shockwave hitting all research work equally. They are a set of tools that make some steps dramatically cheaper and leave other steps exposed as the real bottlenecks.

The cheap parts will expand. We will produce more drafts, more experiments, more reviews, more plots, more candidate ideas.

That makes taste more important, not less.

When output becomes cheap, judgment becomes the scarce resource.

Concept inventory

The main research-work concepts covered in the lecture:

• the role shift from AI as tool, to collaborator, to bounded autonomous worker
• Claude Code-style research assistance for data analysis and paper drafting
• the 100x research assistant idea: cheaper first passes plus human verification
• human verification as the bottleneck after AI generation becomes cheap
• autoresearch loops that modify training scripts, run experiments, observe results, and try again
• AI-generated research ideas with high novelty or excitement but lower feasibility
• the gap between surface novelty and executable research design
• AI reviewers and AI meta-reviewers in conference workflows
• review quality as the key issue, not whether the reviewer used AI
• deadline-aware review that changes advice based on remaining time
• AI Agent for Science-style loops with AI authorship and AI review
• the four research stages: idea, experiment design, data analysis, and paper writing
• AI being stronger in later stages and weaker in problem selection or experiment design
• the shift of human value toward taste, feasibility judgment, quality control, and responsibility

Sources and references

Primary source watched for this post:

• AI Agent (3/3): AI Agent 對於工作帶來的衝擊 - 以學術研究為例