Home
Yoshinari Takayama · Research Overview

A shared language for people and machines.

People describe what they want in sentences. Machines run on code. My research lives in the space between the two — in a language precise enough for a robot to act on, close enough to ordinary speech that the person beside it still recognises what they asked for, and exact enough that a computer can check, every instant, whether the promise is being kept.

01 — The space between

Ordinary sentences carry more than they say.

"Pass me the wrench when I reach out, keep clear while I'm bent over the machine, and don't take longer than a couple of seconds."

Any human colleague understands this immediately. A machine cannot — not because the request is complicated, but because almost every word in it hides a decision.

clear how far is clear? 40 cm measured to the torso a couple of seconds how long, exactly? within 2.0 s of the hand appearing while during? or until it ends? from stoop to stand the whole interval someone has to decide these. the only question is whether it is written down.
Three words, three hidden decisions. Writing code answers them silently, inside thresholds and timers only the programmer can read. The person who made the request can no longer check what the machine understood — and the machine can no longer explain itself in the terms it was asked in.

What is missing is a middle language — one both sides can hold.

02 — The middle language

Four words, and a clock.

Signal Temporal Logic — STL — is a small language for saying how things should behave over time. It talks about quantities that change continuously, which is what the physical world is made of: distances, speeds, temperatures, forces, currents. And it has almost no vocabulary at all.

Everything is built from these

always  ·  eventually  ·  until  ·  and a time window attached to any of them. That is the language.

ALWAYS at every moment keep a safe distance — the whole time EVENTUALLY at least once the task gets done UNTIL one thing, then another move slowly … the person steps back WITHIN a deadline, every time hand extended tool handed over — before the clock runs out
The whole vocabulary. Almost every instruction people give a machine — a rule, a habit, a preference, a deadline — is some combination of these four, and nothing more.

The point is not the notation. It is that once a sentence is written this way, its hidden decisions are out in the open, and if the person disagrees with a number they can say so — before the machine moves, rather than after.

03 — Not a verdict, a distance

The machine can say how close it came.

Most ways of checking a rule return yes or no. This language returns a number: how much room was left. Comfortably fine, barely fine, and just missed are three different answers, and a system that can tell them apart is one that can be improved, watched, and explained.

distance time the line you asked for the closest it ever came clear of the line all the way through — and here is exactly how much room was left
A number, not a verdict. The same quantity does three jobs: something to aim at while planning, something to watch while running, and something to show afterwards to whoever asks what happened.
04 — Both directions

Words into statements, and behaviour back into words.

Until recently this language had a serious drawback: somebody had to write the statements, and that somebody had to be a specialist. That is changing. Language models turn what a person says into a candidate statement; vision-language models look at a scene and report what is actually there. The specialist's job shifts from writing the formulas to making sure they mean what the person intended.

The return direction matters just as much and gets discussed far less. Because the machine holds its requirements as explicit sentences rather than buried code, it can report back in the terms it was given: I slowed down because the clearance rule was getting tight. Not a log file, not a confidence score — the requirement the person themselves stated.

person the requirement written so both can read it machine how close are we? margins, moment by moment says what matters acts on it is measured against it explains itself, in the same words the scene changed — revise what we agreed
A loop, not a command. The requirement is the shared object: the person states it, the machine acts on it and is measured against it, and what the machine sees can send the statement back for revision.
05 — What counts as a signal

Anything a sensor reports, and anything a system logs.

The language needs one thing from the world: quantities that vary over time. That is a low bar, and it is why the same sentences work whether the numbers come from a lidar, an event camera, a wristband, a fleet database or an agent's own execution log.

lidar range event camera wearable motion, pose quantities over time distance to the person activity, as it changes speed, load, temperature… the requirement always > 40 cm hand over within 2 s slow until they step back
Sensors differ; signals do not. Lidar ranges, event streams, wearable motion, fleet telemetry and software logs all reduce to the same thing — numbers with timestamps — which is exactly what the language is written about.

One detail matters for anyone building on small devices. These sentences do not have to be checked on a fixed clock. A requirement can only get closer to breaking when something actually changes, so a monitor can sleep through the quiet and wake on the events that matter — which is the natural way to run on a battery, and the same rhythm that asynchronous sensors and event-driven hardware already work in.

FIXED RATE check, check, check… ON CHANGE wake only when it matters same guarantee, a fraction of the work
Quiet is cheap. Because a requirement can only be threatened by change, checking can follow the events rather than the clock — a good match for event cameras, asynchronous sensing and low-power hardware.
06 — Where the same sentence lands

One statement, very different machines.

The reason to care about a middle language is that it is portable. A requirement about distance and timing does not know whether it is being kept by an arm, a wheeled robot, a piece of software, or a whole fleet — which means the same tooling, the same monitoring and the same explanations carry across communities that rarely share code.

one statement a robot beside a person stay 40 cm clear while I work, hand it over within two seconds an embodied agent look before reaching, and never grasp while the gate is open a software agent never send before I approve, and answer within ten minutes a fleet, or a city no stop unserved for 15 minutes, no junction blocked twice in a row
Portable by construction. Human-robot collaboration, embodied policies, software agents and city-scale services are different engineering worlds — but their requirements are the same four words with different numbers in them.
Why this matters now

Learned models are extraordinarily good at proposing what to do next, and have no way whatsoever of promising anything about it. Capable proposals, no promises: that is the situation the language was built for.

What people actually want

Rarely perfection. Usually: knowing what the machine will not do, and being told when it is getting close. That is a communication problem as much as a technical one.

07 — When it must not break

The same sentence becomes a monitor, and sometimes a guarantee.

So far this has been about understanding each other. The same writing does harder work when it needs to. Once a requirement is written down precisely, a small program can watch it continuously — and for the few rules that must never break, the system can be built so that they cannot be broken at all, whatever the model on top proposes.

what the model proposes learned, opaque, improving does it keep the statement? checked against the margin carry it out with room to spare hold, adjust, or ask and say which rule it was the model may be replaced tomorrow — the statement, and the check, stay put
Guarding the interface, not the model. Nobody can certify a learned component from the inside. What can be built is a boundary around it that holds regardless of what it proposes — so the model can be fine-tuned, upgraded or swapped without reopening the question.

There is one more subtlety, and it is the heart of my doctoral work. Decisions are made every so often; the world moves continuously in between. A promise checked only at the moments of decision says nothing about the gap between them — and the gap is where real accidents live.

DECIDES every so often nobody checked in here MOVES the whole time the line you asked for closest approach — between two decisions
The gap between decisions. Making the two layers of a system — the one that decides and the one that keeps moving — agree about what happens in between is what turns a promise on paper into a promise in the world.
Monitoring, always

Cheap, continuous, and useful even when nothing is guaranteed: a live reading of which requirement is under pressure, and by how much.

Guarantees, selectively

Expensive, and worth it for the short list of things that must never happen. Knowing which list is which is half the engineering.

08 — What I work on

Four threads, one goal.

All of this depends on the language actually being usable: fast enough to compute with, expressive enough for real rules, sound when a system has several layers, and honest when the world turns out to differ from what was agreed.

Make it fast

Rich requirements used to be too slow to compute with. Restructuring the problem around the shape of the logic makes it practical to plan against a full set of requirements inside a real control loop, in real time.

Make it real

Operating rules written in prose by the engineers who run national infrastructure — limits, tolerances, exceptions, habits — carried into this language and run on the actual system, with a live reading of which rule is under pressure.

Make it sound

Real systems are layered: something decides what to do next, something else executes continuously underneath. I work on stating what each layer owes the other, so a promise made at the top survives all the way down to the hardware.

Make it honest

A requirement is a belief about the world, and beliefs go stale. Using vision-language models to compare what the machine sees against what it currently holds itself to — and to revise the statement when the two disagree.

Say it clearly. Act on it. Show your working.