Like many modern small software development teams, we've introduced agentic teammates across nearly every stage of the software development lifecycle (SDLC). As a remote-first team, we nevertheless still rely on daily and weekly human-to-human synchronous videoconferencing rituals for efficient decision making, troubleshooting and prioritizing, and consider these encounters opportunities to bond and build trusted connections with each other. In fact, designing videoconferencing software that makes people who are not co-located feel like they are is what we're all about. That is why we use our own product, Quad, for these rituals.
Quad is a browser-based videoconferencing venue laid out as a room full of customizable tables. You sit at one, you hear the people at your table, the murmur of tables nearby, and wander wherever a conversation pulls you. You can use the tables as a shared surface to look at documents, share pictures and links, organize ideas, and play games together. We built it for gatherings where mingling is the point, and it is the culmination of years of work on how people actually talk online, and what natural conversational flow should feel like.
Until recently, these team rituals in Quad were the one place our agents weren't. Or rather, we'd be their voice when they had information or relevant insights. I wouldn’t say that arrangement was broken, but at some point, we stopped wanting to say "my agent said" to each other and wondered how it would feel to have them take a seat at the table and a voice in the conversation. We had spent all that time building the perfect space to foster natural conversations, without fatigue, and our new agentic guests were poised to make it even better. Then we seated agents in it. It ruined everything.
One agent at the table is tolerable, but somewhere between the second agent and the sixth, the same setup turns into a mess, and it took us a while to accept that no amount of prompting any single agent was going to fix it. Six of them can individually behave and still collide. The problem lives at the table, not with any one participant.
One of our synchronous rituals is the user acceptance test (UAT). Before a build ships, the team gathers in Quad and walks through testing the release candidate together. It's the kind of ritual that underscores the challenge of making a videoconferencing session feel like a real-world jam session, because each human is simultaneously running end-to-end testing and chiming in, often talking over each other and asking questions. This causes audio collisions and ducking with typical videoconferencing solutions like Zoom, creating a stilted conversation dynamic with participants cautiously taking turns. Quad has been designed from the ground up to fix this.
In bringing agents to the party, we hoped we’d be able to maintain our UAT team’s natural flow: speaking freely as a group, watching the telemetry, filing bugs as people found them, and riffing on solutions. Our hopes were immediately dashed. The agents responded too slowly, and only once the conversation had moved past them. They volunteered things nobody asked for. Their responses were tedious and verbose.
The logs revealed supporting evidence. The Quad client each of us runs in the browser watches its table for sustained human speech, 1.5 seconds of it, and any client can ask the server to silence the agents; the server only needs to hear it once, and there's a 5-second cooldown before a client can ask again.
In one 65-minute session that request fired 61 times, and the spacing between requests kept coming out at 6.5 seconds (1.5 + 5). Whenever the agents were talking, the Quad clients representing each human sent requests to silence the agents as fast as the software allowed. Roughly two seconds of unwelcome audio, from the moment a human starts speaking to the moment the agent stops, feels like a painful eternity; we know from years of fine-tuning that two humans who collide in conversation reliably sort it out within a few hundred milliseconds.
When I describe these problems, people usually point me at turn detection, and the work there is real. LiveKit ships a model that decides when a human has finished speaking, and it does that well. Frameworks like LiveKit and Pipecat will join an agent to a live videoconferencing session as a participant, so nothing stops you from seating five agents in one call today. What these frameworks call a room, we call a table: one shared audio channel and the seats around it.
While turn detection is each agent privately answering the question "is this person done talking," our problem turned out to be a different question: with six potential agentic speakers at the table, which of them should speak at all, and who gets to take that away? Each agent is making its decision alone, with no view of what the others are about to do, so the decision has to move somewhere that can see the whole table.
It turns out "floor control" is an old idea. Our engineers point out there's a two-decade-old standard for it in human conferencing, and the agent world is starting to circle it too, in specs and papers. What we haven't come across is a version of it running in production with real people in a session, or in our case a table, which is why we thought our experience was worth writing up.
In Quad the table already is a server, so our team gave the server the floor.
Each agent sits at the table as a participant, with a seat and a voice of its own. We bring in the agents via MCP, some built on Claude, others on Codex. Anything that can take a seat gets a voice and follows the same rules. When people talk over an agent, our clients notice and the server takes the floor back: any queued agent speech gets purged, and the agent stops speaking mid-sentence. Then a 90-second quiet window starts: agents keep working, and they'll still answer if you address one directly, but they don't speak up on their own.
Every time an agent could have spoken, the server also records the reason it did or didn't. That ledger has 23 distinct reasons in it so far. Some were obvious from the start, like "the humans have the floor" or "another agent is already answering." Others were only added after we watched real sessions, like a cap on agents responding to other agents, which exists because two agents will politely take turns speaking forever if nothing stops them. This was only funny once.
The floor is all or nothing per table within the Quad space, so you can silence every agent at once, but you can't turn down just one, or mute the chatty agent while keeping the useful agent.
Brevity is still a prompt rather than a rule, and nothing caps how long an agent can hold the channel once it starts talking. The version we actually want would put the headline in voice and the detail in text, and we haven't built it yet.
Quad has several other designs that subtly convey engagement or the lack of it. Participants at a table can tell when you are "looking" at them, or when you are tabbed away looking at something else. We use emojis. We can layer these onto what agents are already tracking to help them read the social cues and tone of the conversation, and the signals could run the other direction too: we haven't yet given agents the ability to raise their hands, and I want to see what happens when they can.
Only one agent can speak at a time, which is the right default for now, but turn-taking is what makes videoconferencing fatiguing, and we've invested years in spatial audio and high dynamic range so people can talk over each other without distortion. What if one agent wants to affirm what another is saying with a simple "yep"? The same audio engine can re-create the sensation of someone whispering to you while the group talks around you, which may be useful when an agent wants to tell you something without disturbing the table or posting to a text channel.
And as the measurements above show, we need to reduce the time from talk-over to silence so that it's closer to how quickly humans do it.
Every team that hopes to have its agentic teammates participate in group voice conversations is going to end up in this room sooner or later. Some already are: in the last month, one of the busiest agent-workspace products gave the agents in its huddles a voice, and its humans a way to cut them off. The frameworks already make it easy to seat five agents at a table, but as far as we can tell nothing yet makes the resulting conversation one you'd choose to have twice. The open problems above are harder than the parts we've solved. We have one running answer and a pile of measurements.
If you're building anything where more than one agent shares a live audio channel, write me: ryan@highfidelity.io.