Brian Douglas (Paper Compute) on Who Watches the Agents? - Episode 5

[00:00:00] Welcome to The Task at Hand

[00:00:00] Jakob Heuser: Welcome everybody to The Task at Hand.

I'm Jakob Heuser, CTO co-founder at Taskless, and here on The Task at Hand, we talk with the builders, makers, and the things they put into the world and why it's so cool that they're making what they do. My guest today is Brian Douglas, the co-founder of Paper Compute, where he and his CTO, John McBride, are building open source infrastructure for the AI agents that run in production. Their first two products go straight to the trust problem, and we'll talk more about those down the road here. Tapes is a zero instrumentation layer. It records everything an agent does at the network level, no code changes. its complement is Stereos, a hardened Linux OS that sandboxes agents and keeps them from going off into the weeds. Brian was GitHub's first developer advocate. He later led its developer advocacy org. He was a third engineer at Netlify, founded OpenSauced, the open source contributor analytics project that's now part of the Linux Foundation, he's a prolific podcaster. And today, he's on the other side of the microphone. welcome to The Task at Hand

[00:01:03] Brian Douglas: thank you. How's it going?

[00:01:06] Jakob Heuser: It's going well. I mean, as far as I can tell, none of my agents are breaking, so we're, we're off to a good start today

[00:01:12] Brian Douglas: Yeah, good, good times. Yeah, I just, just had Codex put a PR up and I hit merge as you were while you were finishing the intro, so here we are

[00:01:20] Jakob Heuser: Technology is magical.

[00:01:22] Brian Douglas: Yeah

[00:01:22] Jakob Heuser: I

[00:01:22] How dev environments evolved

[00:01:22] Jakob Heuser: guess let's just cut right to it. This is not the dev environments we thought we had back then. Like, if I, if I were to go back to myself five years ago and I would say, "Jacob, the way we write code is gonna look nothing like what you're doing today. There might still be a GitHub, there might still be Git, there might still be pull requests, but nothing about how any of that works is gonna look the same," I wouldn't have believed myself. And you've kind of got a front row seat for this, Brian, because the work you're doing is deep within the idea of agents performing tasks, specifically at, like, Paper Compute

[00:02:00] Brian Douglas: Yeah, 100%. And like, yeah, the dev environment's definitely not what we thought we'd be getting. But I also, you mentioned I worked at GitHub. So GitHub, we had this thing called, Codespaces. And Codespaces was like a successor to all these other in-browser editor tools. And, I remember when I first learned how to code, like about, like 2013, like, I forgot the name of these, like these tools, but like Code Sandbox was like one of these tools that came out years later, or Code Anywhere.

which Code Anywhere is actually the predi- the, the, the, I guess, successor to that product is, Daytona. so it's like a roundabout way to get to that point. So I guess what I'm getting at is like it made a lot of sense to get browsers to basically own the coding infrastructure and like get those into EC2 instances.

but that was ex- that was kind of like clunky and expensive and not quite as like what you thought you would wanted. And then you had like this dev container experience, which Code Sandbox did, and VS Code did. Or sorry, Codespaces really, and VS Code. Code Sandbox also had this. but now we have this place where it didn't make a lot of se- Like, if you have a MacBook, like a M4 or M3, whatever, like it's good enough.

Like it's, it's more than good enough. Like you can get everything you need. So then we started offloading stuff to LLMs, and then the LLMs weren't as accessible like in your local dev environment. So now we have this cloud experience. So like the, you can kind of see how we got here, but if you like, you weren't paying attention, you're like, "I don't know why I'm no longer like, like going to my Z cell and like changing things anymore."

Now it's like Claude's job. Like where did, how did we get here? But we got here slowly.

[00:03:25] Jakob Heuser: We got here slowly. And at some point, we, we... I'm not saying we stopped coding, but I think our role as engineers changed.

[00:03:31] Brian Douglas: Yeah

[00:03:32] Jakob Heuser: no longer about the code I write, it's about the systems I build and about how the pieces fit together. And then the second piece of sort of looking at what exactly my agents are doing.

[00:03:45] Brian Douglas: Yeah

[00:03:46] Jakob Heuser: I kinda wanna talk about Tapes, 'cause that was the first like, the first thing from Paper Compute that got a lot of traction.

[00:03:53] Brian Douglas: Yeah

[00:03:54] Jakob Heuser: And for those that don't, aren't familiar with Tapes, I'd love for you to fill in our listeners, talk about what is Tapes, and sort of what was that aha moment? Like, when did you sit there and go, "Oh my God, a network proxy for my agent needs to exist," which is a really esoteric

[00:04:09] Brian Douglas: Yeah, I

[00:04:09] Origins of Tapes

[00:04:10] Brian Douglas: mean, I don't know if I came to that conclusion like that. Networks proxy, like let's make this happen. but what I, I will say is that I, last, last fall, Ralph Loops became a thing. So it was like this, like a, like series of blog posts and live streams from, from Geoff Huntley. and I was like, "Oh, this is cool.

Let me try it out a weekend and like see what happens." And like I thought it was like really cool 'cause I'd have like all these Vybe Coded projects where I'd go automate a bunch of like maintenance to it, like clean up a bunch of TypeScript or, clean up a bunch of stuff that like I had Vybe Coded through like Claude Code or through Continue at the time I was working on, at that company.

so I got it to basically work, but then I was like, okay, well I use like Opus for this, like obviously it's gotta work. Like what happens if I put a Ralph Loop like on Haiku? Like could I also get the same job done if I had like really structured work for it? so yes, it could get done, but then I was like, okay, well it got done, but like What if I did Haiku and Opus on the same problem, and I just recorded the tokens, like input outputs, and then like calculated the cost of tokens?

so I did that, which is like basically what I-- And it says-- when I was googling for like solutions, the network proxy became the thing. It was like, oh, let me just like inject in front of api.anthropic.com, and I'll just capture data and then drop it into like a SQLite data. I think, actually, I think I went with SQLite originally, and just drop it in there and just like look at it later.

and pretty easy to accomplish, and I think there was a couple other tools. Like there was a designer at Notion that had built something very similar, but it was like had a UI in front of it. And I just wanted super command line based, make it simple, make it something that like anybody can pick up and just run.

but it was like, there was like... It was kind of clunky. I wrote it in Go, and I say I wrote it. I, I vibed it with Go, in Go with, Claude, and it solved the problem. And then my co-founder, who was my co-host of my podcast at the time, Open Source Ready, still, we still run this podcast. I pitched him.

I was like: Hey, I got this idea, and like I don't think it's like perfect, but like this is what it is. Here's like a private repo. Like can you take a look at this? and then he took a look. He's like: Yeah, this, I would actually approach it this way, and like maybe I'd clean this part up. I was like: Oh, cool.

Thanks for the feedback. so then I went the next weekend and fixed it, and like it, it was even better, where I could just like run it and then have like decisions based on like the same problem run with all different models within Anthropic, like let's go. so that's kind of how we just started. It was just more of like the proxy part was a, it was kind of like dumb luck.

there is... Like the other concept of like Claude Code and Codex, they all have session data that they record already. But even at that time in November, October, it was pretty light. Like Claude had just started putting session data, and, and they're like started collecting the data, 'cause previously it didn't.

Codex, Codex, the current form of Codex has only been around since January, so like it wasn't even an option. but then there were like all these other tools. So at Continue we had .continue/sessions. That same pattern everyone did. Cursor has sessions as well. so just knowing that the session data existed, wasn't good enough, but like the session data actually was missing a lot of information because I started capturing tool calls, started cap- capturing like skill usage, started capturing like more information, which I think a lot of these tools ha- are now capturing a bit more.

but yeah, that's, kind of the inception of like how we got here and it was, I ended up open sourcing it. actually, we didn't open source it. It was still a private thing until, we actually went and got funding, incidentally. I was actually just doing a demo with a friend, and they said they would invest, and I was like, "I guess we're gonna make this thing a company."

[00:07:33] Jakob Heuser: And so it became a thing.

[00:07:35] Brian Douglas: Yeah

[00:07:35] Jakob Heuser: that's actually kinda cool, the network boundary as a, as a layer, ultimately, we've pulled apart these calls ourselves, and, like, it's just JSON over the wire. But in there is a rich corpus of information which is different than what ends up in everybody's session log formats,

[00:07:52] Brian Douglas: and this...

[00:07:53] Jakob Heuser: because the network becomes a sort of source of truth

[00:07:55] Brian Douglas: Yeah, and I even called this out in like our, our-- There's like a one-liner on the tapes.dev docs page, which is like high fidelity traces. and I think I'm alluding to the like tapes being, you know, how you, how you listen to music. but it really comes down to like it is, it's a bit more high fidelity than just like doing the generic OTel that comes off these things, which is like a good place to start.

Like you could rsync all the

[00:08:14] Network as source of truth

[00:08:17] Brian Douglas: session data up to S3. You could o- like do a OTel, drain, get it up into your Grafana or wherever you, you like to look at OTel. but I think what it would come down to is like there was a lot of questions I wanted to ask about this data, and it just wasn't available. so like we do have OTel now out of these tools.

We do have session data that now is pretty much a standard for all these tools. but there's still so much more you could do with it. So yeah, we're kind of just pushing the, the envelope on it.

[00:08:38] Jakob Heuser: And there, and there's a lot of other stuff. Like just because it's not in session doesn't mean it didn't go over the wire. And

[00:08:43] Brian Douglas: Yeah

[00:08:44] Jakob Heuser: helpful to see the raw tool response that says, "I want you to call these tools the, because I've told you these things."

[00:08:50] Brian Douglas: Yeah

[00:08:52] Jakob Heuser: it does end up in the session,

[00:08:53] Brian Douglas: Well, I was gonna say like,

[00:08:54] Jakob Heuser: up in that nice consistent package

[00:08:56] Brian Douglas: other thing is like, this, this is around the time where sub-agents really took off for Claude Code. Like, it was like an idea and like it was slowly trying to figure itself out, 'cause like there was a point where you actually would have to define su-sub-agents.

if anybody used Claude Code like last summer, you'd actually have to like basically say, "Hey, Claude, I need a sub-agent for reviews. I need a sub-agent for deployments. I need a sub-agent for like cleaning up the code in this way." And then you could define it in a markdown file, so like it'd be .claude/subagents.

and then it got to the point where sub-agents was automatic. Like you just say what you need and it spins up the sub-agent, and now if you do a plan inside of Claude, and Codex as well, it'll spin up a sub-agent. But at the time, the sub-agents were just black boxes. so you couldn't actually see what was happening, like in, the ins and outs of like what the decision was made.

nowadays you can hit Control Zero and it expands it and you can see. Or now you can actually tab through, and like click in and look at the sessions. But that didn't exist before. so at the time that we were discovering this, we were like, "Hey, these are black boxes." Because... And the reason why the network layer is because you could still capture the sessions, the ins and outs, if

[00:09:59] Sub-agents and black boxes

[00:09:59] Brian Douglas: you were capturing on the network layer.

So you have full control and full access to everything that goes across between the API and your machine. so that's also the reason why we also went that way instead of just scraping sessions that, again, didn't exist in October, November. it's, it's weird 'cause things are moving fast where like a lot of the stuff that we have that we're like telling a story around in the open source, like it exists today in a lot of different flavors.

but I think where we sit is like we kind of wanna build like a standard in how you approach these problems.

[00:10:24] Jakob Heuser: No, that makes sense. And I mean, individual companies, they work together when their incentives align, but there's no guarantees that Claude Codex and any inference service that you talk to, or even if you use like OpenCode's, code sessions, no guarantees everybody's sessions are going to stay the same

[00:10:41] Brian Douglas: Yeah.

[00:10:41] Jakob Heuser: length of time

[00:10:42] Brian Douglas: Now, and like even An- Anthropic's Claude code has changed multiple times in the last couple months. and like we've been a-adjusting with how we've been capturing data based on their, their changes. They definitely slowed down since like the government, like, Middle East, like, sh- shooting missile stuff.

Their PR nightmare, they've kind of slowed down like the accelerant, and then obviously Fable has slowed them down a little bit. So, from what I've heard from the team is like they've been focused on resiliency. so that and also probably, I don't know, like viability of a, a product in the market and that they need to sell models and stuff like that.

So, I, I do appreciate them taking a, a, a breath for a moment so that we, we can catch up and have feature parity.

[00:11:19] Jakob Heuser: It, it definitely is a, like, e- they're racing too. And like I think on the OSS side, because Taskless also has an open source component, we are constantly, I'd say two to three steps behind where the frontier models are

[00:11:32] Brian Douglas: Yeah

[00:11:33] Jakob Heuser: they are racing so far ahead, and they're also putting a lot of things out there that may or may not stick.

[00:11:38] Brian Douglas: Yeah

[00:11:40] Jakob Heuser: I use MCP as a classic example. When MCP came out, everyone was like, "It's the greatest thing ever." And then it was terrible.

[00:11:45] Brian Douglas: Yeah

[00:11:46] Jakob Heuser: now it's finally in like a normal spot

[00:11:48] Brian Douglas: Yeah, and I think it's in a normal spot. I think there's gonna be an announcement next week about MCP 2.0, like its new protocol. You also see like even Anthropic and Claude Code, like they don't go MCP forward anymore. They're ca- they're called plugins. and then the, the desktop app, they're connectors, and I think even with Codex they're called connectors.

So I think they, they needed like a, an abstraction to like combat against like this, the how convoluted MCP was. Granted, like MCP was so early and so much adoption t- like really early, that the structure of like whether you have a server-side rendered events or ser- SSE events, or like a local MCP or like a, a remote MCP, like there was just too many flavors and it wasn't very clear like which one was the right

[00:12:29] MCP evolution and bloat

[00:12:29] Brian Douglas: one you should use.

and I think there's some like decisions that have been made since then that make it a lot clearer, but I think a lot of people have been burnt out on just like MCP in general.

[00:12:37] Jakob Heuser: Yeah, it, it tried to do everything, and it was... I-- There was a lot of stuff around discovery and things like that that were really good, and I, I saw the, the two-spec and, like, there's just good pieces carrying

[00:12:48] Brian Douglas: Yeah

[00:12:50] Jakob Heuser: but then they also have to get over the hurdle of everyone hears MCP, and it carries all the baggage of almost over-hyped and over-implemented V1.

[00:12:59] Brian Douglas: Yeah

[00:13:00] Jakob Heuser: I, I don't know how else to describe that. Like, everything got built around that hadn't even really been finalized 'cause everybody was so excited and hungry for it

[00:13:10] Brian Douglas: Yeah, yeah. 100%. And I think as we saw the agents, like, really take steam, like earlier this year, like it just like prope- like accelerated that. So I think we had like, we had the sort of, harness engineering, like it really took off like last summer. We, we saw Cursor ship a CLI, we saw all the other tab completion startups also s- ship CLIs and build what we now know as harnesses.

but then MCP was just right there. So I think where Anthropic was like, trailblazing for everybody and showing off like what you could do with it, you saw, see the adoption. But I think the MCP thing is interesting 'cause about the enterprise adoption, security, and like all the stuff that wasn't ready for primetime, I think we now are ready for.

But the industry's already moved the scales and like you use MCP for like... if you're gonna have a startup around MCPs, you're probably gonna have like a bespoke situation that's gonna be like the best onboarding experience is through the MCP. otherwise, I think we're, we've got MCP bloat right now.

[00:14:06] Jakob Heuser: Taskless actually moved over to a CLI specifically because we, we were big fans of OpenSpec by FusionAI

[00:14:12] Brian Douglas: Okay

[00:14:13] Jakob Heuser: And what really shook us up was this idea that your CLI could hand the agent instructions.

[00:14:20] Brian Douglas: Yeah

[00:14:21] Jakob Heuser: Suddenly, the process of versioning got really easy.

[00:14:24] Brian Douglas: Nice

[00:14:25] Jakob Heuser: And kind of in that space, as we kind of segue through here, you've talked about sort of the three hard problems that production agents have, and it's not the prompts or the models.

Like, we can get prompts in very easily. you said it was isolation, observability, and memory

[00:14:41] Brian Douglas: Yeah

[00:14:41] Jakob Heuser: of the three really hard problems for production agents, I'm really curious about why those three and why people keep underestimating them.

[00:14:51] Brian Douglas: Well

[00:14:51] Jakob Heuser: memory in particular because I feel like I al- every time I try to explain it, it's like, "It's just a markdown file," and I, I know it's not. But, like, my inner

[00:14:59] Three hard problems for agents

[00:14:59] Jakob Heuser: engineer wants to just very much reduce memory to something much, much simpler.

[00:15:04] Brian Douglas: I, I, I-- So I, I think you're probably citing a blog post I wrote, like, in February or March. so the world has changed a lot since. so, like, I think the isolation part with sandboxes, like, we definitely have... I think the last thing I saw was, like, 135 different sandbox providers and competitors. Open, Open Sandbox, I think, is the company that, like, keeps a, a table of all the sandbox providers and, like, they're all to compare and contrast.

so I think the isolation thing, I think we're, we're on board. Like, there's some validation people have, like, figured it out. And, like, I talk to more and more enterprises that already have sandbox as a primitive. So all engineers have to spin up a sandbox, which is great 'cause you can close your laptop, you can move to the bus and commute back up to San Francisco from South Bay or whatever it is.

So, like, that, that primitive's, like, very clear. the memory thing's interesting because, I, I, I, I... You were at, the meetup on Monday where I gave the demo, at Work OS about dreams, and I think we're, we're still underutilizing, like, all the data. And this is, like, part of what we've figured out with tapes, enabled, like, the Paper Compute company to say, like, the session data is really valuable.

And, like, until you start looking at it and you start mining it for things, like, you're not gonna really understand, like, how valuable this is. And, like, the majority of us are basically, like, just spinning up a new session, and, like, the last session is just, like, garbage to us. Like, we just moved away from that.

but just like any branch or PR or commit, like, is also valuable intent and understanding, for, like, future engineering. Like, every dead session is, like, super valuable as well. So, like, I love spinning up features that never see the light of day and never go to PR. and what's cool about this is, like, inside of my Paper console, like, I can capture that.

and then, like, that could be a memory of, like, we have a designer/product person who's, working with us now. and she can go to all the old sessions and be like, "Hey, what did Brian work on, like, the last three months that never saw the light of day and never got a doc? Like, how can I sort of mine that for, like, the, the next parts of the roadmap?"

And, that is, like, extremely valuable that, like, once we tell that story, it's, like, very clear. but yeah. And also the security part. Like, obviously, like, you could just drop an entire sales doc, like, a customer evidence into Claude Code. It's like, where does it go? I don't know. like, there's a lot of opportunity to, like, okay, maybe we sanitize some of the session data that goes through and, like, still make it okay to, like, drop secrets in there.

But, like, the, the data being in its rawest form, like, if you leverage Fable, Fable is using definitely the data you send the first 30 days, to help train Fable for security reasons and et cetera, so. Anthropic's, like, I--

[00:17:30] Mining session data for memory

[00:17:30] Brian Douglas: They're, they're an amazing company. Like, they have a lot of amazing feats and stuff like that.

But, like, there is a world where you're like, like, I don't know if I want this to be shared." And you get that little pop-up on Claude Code. It's like, "Hey, do you wanna share your data with, Anthropic?" And it's like- Like what's the, what's the value of me sharing it now versus tomorrow versus like last week?

Like, and one thing I'll also call out about that, which again, I, I do believe in Anthropic, hoping for the best for them, but they had a, a, a bit of a bug where if you said no to sharing data, you would actually exceed your rate limits, but also your token usage really quickly. And it was because the prompt caching, they would-- If you didn't say yes to sharing data, they would take your prompt caching to like five minutes and put it up to an hour.

so like you, yeah, you just weren't caching, prompts. I don't-- I'm, I'm not sure what the correlation there, and they never explained it, but they did, Tharic did have like a tweet about like apologizing to the community and like they, we'd figured it out. but it, it was basically that

[00:18:24] Jakob Heuser: It, it's amazing how much goodwill those things erase too, because when the agents do things unexpected, or your harness in this

[00:18:31] Brian Douglas: Yeah

[00:18:31] Jakob Heuser: like it erodes trust, and you only have so much of it

[00:18:34] Brian Douglas: Yeah, 100%. So yeah, it's, But I think, trust is earned and drops and, like, it's, like, lost in buckets. This, this AI space though, like, like, OpenAI's had their, their time in the sun, and I think they're kinda coming back into the sunlight and, like, h- having a good time with Codex and stuff like that.

But I think we have short-term memory, 'cause, like, if you keep giving us free compute and, like, gifting us, like, new features, I think we have a bit of, like, grace. And I think even the, the, the question about, like, even, Sorry, we're going super detailed, but, like, Groq ha- had a bit of an exposure period where they were, like, full sail putting GitHub repos, not from GitHub, but from your local environment, so .env file and secrets and everything, like full kit and caboodle up into Groq.

so they, like, they walked that back, but that's also like, "Hey, what actually is going on here?" But then, like, new model, shiny new thing. Like, "Let's forget about that and move on to the next thing."

[00:19:28] Jakob Heuser: I'm convinced that new models are basically the Neosporin of AIs. Like, "Oh, your boo-boos heal up

[00:19:33] Brian Douglas: you're not too far off. Yeah

[00:19:35] Jakob Heuser: Especially because you, I'm waiting for the au- like the, the Claude skill that just does the agent rotation. Like now that like Cloudflare has their MCP and you have the CLI for it, just be like, "Claude, rotate my keys." I'm sure that will work fine. I'm sure it'll be-- I'm sure it'll go perfectly

[00:19:50] Brian Douglas: Yeah. And then Cloud's like, "Hey, your new key is this." And it's like this saved in memory

[00:19:54] Jakob Heuser: All, all I'll need is something that can audit what it's actually doing. Hey,

[00:19:59] Trust, security, and data sharing

[00:20:05] Jakob Heuser: wait a minute. There's a guy I'm talking to that's actually caring a lot about what goes over the wire. you mentioned trust, and I wanna get your take on this. Like what does trust actually mean for an agent? you built Stereos, which is a sandboxing environment to like sort of con- constrain the agent's worst impulses. And you've got Tapes, which is like observability and ultimately tangential to that is auditing. because ultimately we don't trust agents. Like, I mean, it's gonna generate code and then it's going to execute code and yeah, I think we're, I think we're probably right to not trust that. But then what does it mean to eventually trust it?

Like what does trust refer to in that context?

[00:20:47] Brian Douglas: Yeah. Yeah, it's, yeah. Trust, I mean, we are definitely in a space that, like I just did a code review inside of Codex this morning. I was like, "Man, I haven't done this in forever." but like the trust is like we, we kind of have an anticipation of like, hey, I've got, I've got test. I like, I have best practices that are hopefully codified into the skills.

Like if I go spin up like an experience or like ship a feature, can I trust this thing that like it gets to completion? But then trust is like, hey, if I drop in my entire Notion doc of every sales call I've ever had, like can I trust this thing doesn't go off the, off the rails? And like, most of the time, no.

Because like I, I definitely get to that point where, I've like prototyped a bunch of random stuff that Claude thinks, specifically Claude Desktop, but thinks like this is like real life features that are like in production that has users. and there was like, with like, I forget which one it was, but like back in the, the Opus 4.4, 4.5 experience, like it would actually hallucinate and be like, "Oh yeah, let's like, let's id- identify like possible customers."

And I actually had this, so when I was like possible customers that could leverage this product, and like it would spin up a list. and it'd be like, "Oh yeah," it, like it would know like these was all fake. But because that's like committed into like its memory, it's like, oh, okay, well if we,

[00:22:00] When agents hallucinate

[00:22:00] Brian Douglas: we've like prototyped these are the customers that we can have, and it eventually gets to the point where it's like, "Oh, and we have like big government contracts, and blah, blah, blah."

And like, it just like starts committing that you've like done this already, even though like, "Hey, no, we just like made this up together." Like we were just like, we were shaping up a hypothetical. so same thing with like features. Like I, I've like id- identified features that I wanted to build, that are not in product, that are not in our roadmap, but they're like what we could do if we had like raised more money or like if we had a bunch of chips, to run on or hardware to run on.

But it like just will bring that in and just like, almost like a, like a, like, like a frenemy, I guess. Like a person who like knows your deepest, darkest secrets and will bring them up and be like, "Hey, remember when you said this dumb thing?" Like, "Let me inject this into your like, your blog post," which I don't generate blog posts from Claude anymore, because like you just don't know, you don't know what you're gonna get out of it.

So like it's better to like, I'll get an outline from it, but like I, I won't go full, generate anything from it.

[00:23:00] Jakob Heuser: Yeah, it's, it seems to pull from a lot more information than it's supposed to, and that's, it's actually kind of dangerous, especially if it's... This, we saw this when we built Taskless. Taskless adds like determinism to AI, AI agents. Like, if we can give you a process exit one and stop an AI from doing something that's a really dumb idea, we're gonna script it, we're gonna build it, we're gonna run it. One of the things that we noticed time and time again was as code bases evolved, Claude and Codex in particular would look at existing code when implementing future patterns. It, it wouldn't go look for a best practice, even if you told it like, w- for example, React migrations are my personal favorite because there's so many fundamental changes version to version. And you'll say, "These are the old ways of doing it. It's done this way. Reference the docs." And it will immediately go back to its code base and just replicate the patterns that are either deprecated or broken. It'll go into Git history, and it will find examples of how it used to do it, and will keep doing it that way. And at some point, there has to be systems that tell it no. And I think there's this really cool complement where a tool like Taskless is the enforcer, but it can only enforce what you know. And a tool like Tapes is finally that knowledge. I wanna know if you're going into Git and looking places you're not supposed to.

I wanna know you went to solve this da- this migration problem. you even go to the docs?

[00:24:31] Brian Douglas: Yeah.

[00:24:31] Jakob Heuser: you do the thing I asked you to do?

[00:24:33] Brian Douglas: Yeah, and it's a really good question to ask the agent as well. And this is like where we have production traces of things that I've done. Like when it's been happening, it's, it's happened, anomalies, failures, et cetera. Like, we can codify that in a way that, like, can produce memory and, like, today our memory is just like we just have a database.

we're actually looking for partners for the memory layer 'cause, like, there's some folks who are doing, like, a better job on that. but at the end of the day, it's like, like there are things that I've done, and I, I explained like the sort of like going back in the, the look back machine of your, your sessions and identifying like features or roadmap you could, like, you could do.

It's also, like, finding out what needs to be documented, like what's been broken, what could be a task or what could be a skill. so yeah, it's just like, again, like the, the, the thing I tell to all of our pro-respective customers is like, Claude Code stores your sessions on your machine for 30 days. After 30 days, those are deleted.

So, like, if you're not saving those sessions, you're missing on a lot of really good context and opportunity to, like, build best practices into, like, how you engineer. mind you, also our product is not engineering-only focused, so because we went to network layer and we're not hooking in the Claude Code directly, we're agnostic enough that, like, if you're using Claude Desktop or Claude Cowork or, like, all these other tools that have inference on it, like we can capture the, that, those sessions as well.

And the cool thing about these, they're all just tokens. They're all just prompts and tool calls and skills. Like, all the data that flies through back and forth on the network layer is the same. It's the harness is what's the different part.

[00:26:00] Jakob Heuser: I, I like that idea about, you mentioned like after 30 days the session's gone, it's gone forever. something that I just realized was that you've got this data sitting inside of tapes, and now you've got the ability to almost run remote inference on these sessions. Like what, what-- Almost like dreaming, but across any platform.

Like, what were the mistakes we made as an agent?

[00:26:23] Brian Douglas: Yeah

[00:26:24] Jakob Heuser: What can be, what can be codified? What can I then export to other, other engineers so that their agents don't make the same mistakes? And there's this really cool one engineer's success, they figured it out, suddenly becomes every engineer's success

[00:26:39] Brian Douglas: Yeah.

[00:26:39] Jakob Heuser: it can just propagate through the org,

[00:26:41] Brian Douglas: Well, it's,

[00:26:42] Jakob Heuser: actually really

[00:26:43] Brian Douglas: it's single player to multiplayer. Like, I don't know if you play music, but like you can play guitar at home or play drums and like you can sound amazing. But like as soon as you bring in other players with you, like, and you're starting to play in an orchestra, it's actually so much better.

It's the same thing if like single agent versus like a, a span of agents, like same deal. But like with those agents, they're all just kind of doing their own thing in isolations, and they're like looking for context engineering and markdown files to be completed for them to catch up. Like when you run in parallel, there are things happening across the wire at the same time in isolation that you could have essentially what the, the, the

So the original picture was actually called Stenographer. It wasn't called Tapes. ironically, I did call it Tapes, but then I was thinking bigger, and it was like courtroom reporter types on a k- a keyboard. and then like you, like, you ask the reporter about, "Hey, read back that thing," or, "Tell me about this other thing."

And like all that knowledge gets stored somewhere. But like the, the actual idea was intranet for agents. So like if you have an underlying intranet where agents can tap in and be like, "Hey, I've actually got this problem. Let me see if anybody else has seen or anything has seen it before," if you're only using agents as like the, the, operator here.

but

[00:27:49] Jakob Heuser: the serendipity of the office. Like, you go by somebody's desk and they just happen to be working on a problem that makes your life

[00:27:54] Brian Douglas: Yeah

[00:27:55] Jakob Heuser: If every agent is talking, suddenly an agent can learn like, "Hey, the

[00:27:59] Single player to multiplayer agents

[00:27:59] Jakob Heuser: person three desks over is actually about to make the problem I'm trying to solve go away. Let me actually surface that so the humans can actually get on the same page about it

[00:28:10] Brian Douglas: Yeah. And, and this is, what's cool about like our, our multiplayer service that we, we-- So we have like tapes and we've been ta- I, I've also mentioned in passing paper, paper, the paper console is like what you wrap everything up into. And I can see live now that like my, one of my engineers is rewriting a bunch of the, open source stuff on tapes.

we've got this whole plan, which I don't know when this podcast is out, but like, in the coming weeks we're gonna have the concept of cassettes. And cassettes are basically things you can take home with you, you can like install, into your, your situation. So these will be like plug-ins, plug-in architecture within tapes.

so he's working on this and I could be like, "Hey," in Slack be like, "Hey, how are you doing with that thing?" Or like jump in his PRs and be like, "Hey, curious when you're doing this next." Or I could just be like, let me just like drop into his sessions, like check out one of his, his sessions or cassettes and be like, "Hey, let me just chat with the session real quick," and just ask a bunch of questions I'd be asking in Slack instead.

And like right there in real time, I've got all the questions answered. And everything I can't answer, I go to Slack. But like you're now well more informed and prepared to like go have that DM conversation or like go find that person in the office. nothing's worse than like getting a DM from somebody who's like standing right next to you in the office.

Like you DM them and you go walk and be like, "Hey, did you see that DM?" now we're at a place where we could just like, "Hey, let me just go check into your sessions to see if I can find the answers before I come like bother you."

[00:29:28] Jakob Heuser: That's actually, like, super cool because, I mean, it's-- one, it's just documents. It, it turns out that agents are really, really good at summarizing documents for you.

[00:29:37] Brian Douglas: Yeah

[00:29:38] Jakob Heuser: And then the second piece is actually allows you to, it actually allows you to then take those learnings or even jump in or even make corrections.

Like, you can start from a, a point, a point in space in the tape and be like, "Hey, given this, start here. How would-- What are some other ways we could have done this?" There's almost rewind, replay. There's all these things that become

[00:29:59] Durable sessions and dreaming

[00:29:59] Jakob Heuser: possible once you have that sort of durable record of what was done

[00:30:03] Brian Douglas: Yeah. Yeah, 100%. Yeah, so it's like, I've been talking with Temporal about this, about like durable sessions and like how we would like, how we like replay sessions and also like how can you replay sessions in a way that... So we, we, we alluded at the fact of dreams. So I give this talk on Monday, gave a demo about like recording sessions and then like categorizing things for observations and how do you apply that to memory.

but then like once you've done the base layer of like memory and then like skills, cool, like you can move forward. Teams like context engineering or knowledge transfer has happened. But like what if you just wanted to like, just like "Doctor Strange and the Infinite Madness of the Universe," like just wanted to like operate through and be like, "Okay, well, like with this is how it happened, but like what if we like did a parallel universe of like these are the situations and like this is like what we wanna see an outcome.

And like let's change the model here, but then also let's change the effort to high here. but then also let's do this." Like there's a lot of ways you can like kind of re-spin up a session and test. And with the advent of sandboxes, like it becomes economically like approachable to say, "Hey, let's spin up 100 of these."

and what I've been testing right now, which I'll do a lot more on this weekend, is like I've been doing 3D animations on the command line. And what I found really quickly is Anthropic models are not good at it. Anthropic knows 2D pretty good. and you can do some ASCII stuff with 2D and like you've got the very, very, like you...

If you've seen one ASCII art from Claude, you've seen them all. so then what I found with like, what is it, Sol, 5.6 from OpenAI, really good for animations. But then when you start getting like very specific about, "Hey, I need it rounded here," or like, "This is my color palette," then it's like, it kind of falls off.

so then there's like, Gemini 3, which I've only used 2.7, but 3, what I've heard, understands design and front end much better. so now I can like set up three different parallel worlds and be like, "Hey," but if I wanna set up like 60 of them because I've gone to like different levels of high, like how much effort and reasoning will be involved.

Now I'm like, "Cool, 60 of these versions, let me just spin them all up and then choose the best one." And then from that, I've got a benchmark to say, "If I do the 3D animation built in Ratatui in Rust, this is the model, this is the effort, this is how we do it moving forward." So it's like not just having a skill, but also having like a prescribed, like this model, this effort, this reasoning.

Go.

[00:32:16] Jakob Heuser: And I, and I think tapes actually touches on something that most, the frontier models aren't going to do, which is not only rewind, now replay on a different model.

[00:32:25] Brian Douglas: Yeah

[00:32:26] Jakob Heuser: from this point forward, replay it and see if we get a different outcome. It's, it's almost like the best of in, like the best of in design taken to a incredible extreme.

Like, we don't just have to test what does it look like on Sol, what does a test look like on Fable, what does it look like on Kimi? It's what does it look like on all three at like seven different levels of effort? And you can get that all from the same session replay forking at the same point and just see what all of the possibilities could have been

[00:32:56] Brian Douglas: Yeah, yeah. And there's like certain things where you're just like, man, I kinda wanna try this

[00:32:59] Testing models in parallel

[00:33:01] Brian Douglas: out a few times. 'cause like you get like, if you go through like a Claude planning session or use Superhuman, and it's kind of crazy, like I felt like really out of this, like a year ago I was like, man, I don't think I've really caught up on this agent space.

Like it was like Cursor had their agents, Quad Code had been around for a few months, and I felt very behind. so like I just started doubling down into like, let me just go like go work with a company that's building this. Let me go double down and like start building as much stuff as I can. And the, the benefit of working in a company building this is you have free compute through them.

so that was like, that was a deal. I was like, "Oh, let me work for you guys. I'll build some stories. I get free access to the pro- product," win-win. they also paid me, so that was also another win-win. but what I'm getting at is like, man, like I have like basically my workflow where I could try a bunch of stuff.

Like I've got my, Ghostty terminal with the multiplexer happening with like, if you, if you're familiar with c- conductor, I've got conductor at home. Like I've-- can quickly spin up a bunch of terminal em- emulators. I've actually fallen off on using Vim and stuff like that because now I'm at the point where if I have to open up Vim, it's for something like hyper-focused that I have to like slice and dice and like get to the root of the problem.

But even when I used to do that for things, like I used to do that for front end, like animations and stuff like that, now I'm like, just this morning I was in Codex, and Codex has this cool like N Codex, or I guess it's ChatGPT Codex or whatever it's called, but it's got this in-browser server we could spin up an environment like, like your local host.

and you can annotate directly on the local host, like live, which is frankly like paper design. That's a feature they have out. There's a couple other tools that do this already, but I just don't know if I'm gonna use that or just use Cursor, which has auditing built in, and then the code review's built into Cursor.

So now, yeah, there's just a lot of like, okay, we've got the tools, but okay, how do we pass the knowledge a- across? Like I can have a great time. I've got my own workflow, but how do I show you my workflow? And I think that's the thing that we need to unlock.

[00:34:55] Jakob Heuser: Yeah. That, that actually kind of leads into my, one of my last questions here as we kind of look forward. you've had a front row seat, GitHub for Actions, Discussions, Copilot. build open source. You've been much through a whole lot of tech companies. You've seen a lot of stuff. seen a lot of stuff, Bryan.

[00:35:15] Brian Douglas: Yeah

[00:35:16] Jakob Heuser: where do you feel the agent ecosystem is actually heading versus where the hype says it is? Like, what's gonna be the durable stuff that's gonna matter down the road?

[00:35:25] Brian Douglas: Yeah. I, I think what the, the hype is that you're gonna have a software factory and you're gonna basically pay for all of it. I think we're in a, we're very much, if you look at Vercel's Eve, if you look at Cloudflare's, like, agent framework, like, I think that's been commoditized. So now you have specialization of, like, well, this agent does this and this, this agent does that.

But then you have these, like Foundational model providers like so Codex has a really, really good framework for-- or at least the videos I've seen that can edit video. so now I'm like, "Oh, okay, well, like I guess I'll try this thing," and not this other random tool that's other thing inside of Descript or whatever 'cause like the model provider is now training the thing that Descript will eventually have.

So I think what we're gonna have is like we're gonna have a commoditization of a lot of these tools around the models. but I think what's gonna be actually be real sticky is the knowledge. Okay, I figured this out, like this is my prescribed way to get this done. This prescribed way I can hand to an agent or OpenCall or Hermes or whatever.

like that is what always g- where the money's gonna be at. It's like, how do you codify this is our workflow, and this is how you get things done. And I think it's gonna be, beyond coding as well. So like obviously we need to like think through engineers, but I think we've saturated the engineering market.

so now the next thing is like, cool, how do you build, how do you build skills for like my legal proceedings or like setting up like, whatever these documents. Like I, I, I cannot go through... Like our, our company's on Rippling, and Rippling has an insane amount of documentation 'cause they've got so much stuff that's covered.

But what that means is like I have to search for everything and like the very specific bespoke things. So like, like I had to

[00:36:59] Hype vs durable reality

[00:37:10] Brian Douglas: do like Washington state tax exemption something 'cause we have an employee up there, and I was like, "I don't think I have the time or bandwidth to think through all this. So let me just get a quad co-work to go through the documentation f- on my behalf, and I'll come back after making a coffee and like it's done."

so now we're like thinking what else can we, what else can we expand beyond coding? And I think that's just things that we have to start thinking about, which I think unfortunately like it's like it is a question around like where the jobs are and like who's at risk and, I think everyone's at risk.

And I think what it really comes down to is like you're only as good as the tools ahead of you. So like if you were doing horse carriage rides through New York City back in like nineteen oh one and like literally 10 years later, the Model T drops and you're like, "Well, there goes my job." Like, it's still a bespoke world where you're like, "Hey, horse carriage ride, inside of Central Park is good 'cause you can't drive on the sidewalk in Central Park, so like that's my niche."

but like the cars are gonna live, the cars are gonna move forward. And like I drive an automatic, I don't change my oil. Like this is a lot of things that I did when I was like, you know, 20 years old versus now I'm like almost 40 like that I don't do, but I, I don't mind the convenience. Like a- again, like I think we're, we're just gonna s- have to watch, tech evolve in front of our faces.

And, I was looking at the Railway CEO was talking about like code's no longer the moat, and the moat is actually how your company achieves their path achieving the certain goal. And if you're not doing that, like you have a rude awakening. And I think what it comes down to is like, what's the goal?

Like, are we trying to s- like, are we trying to reduce the amount of time for people to like leverage these tools and be really good at it? If that's the goal, like even more specific and like that's what you should be focused on and less about like pristine Go code and pristine, you know, software. Well, I think the systems make sense, like pristine systems help 'cause agents can navigate to that, but some things are probably not as important as they used to be, like pair programming.

Like I don't know if that, that's a necessity anymore.

[00:38:55] Jakob Heuser: It's gonna come down to, like, just really good primitives. Like, can we build on some really good fundamentals?

[00:39:01] Brian Douglas: Yeah

[00:39:02] Jakob Heuser: there's an entire layer. It used to be, the UI layer was very disposable.

[00:39:06] Brian Douglas: Yeah

[00:39:07] Jakob Heuser: hold any state, and I think it's actually moved down a layer as well. It's no longer just the UI layer. It's now the UI layer and the service layer. And there's still good fundamentals you need to have, like how do we actually operate? And these aren't code problems anymore. These are company, like, human problems. How do we set goals? How are we measuring goals? What are the right goals for what we're trying to achieve? And those aren't tech problems anymore. Those are kind of universal problems

[00:39:35] Brian Douglas: Yeah. Yeah. What's even interesting is like I've been-- s- for whatever reason, 'cause people know I used to work at GitHub, my friends and family have been approaching me based on their vibe coding apps and be like, "Hey, can you look at my GitHub?" My random repo. it's not random. Like they've thought through some problems.

They've built tools through like, like Replit and stuff like that, and Claude Code. But now they're just like, "I don't know what I don't know." Like, so my, my take is like I go through their repo. Usu-

[00:39:59] Beyond coding and TDD

[00:40:03] Brian Douglas: usually it's a TypeScript repo 'cause that's usually what gets generated. and then I just look, "Hey, do you have tests?

Do you have linting? Do you have like a deploy strategy? Are you leveraging GitHub CI and actions?" And it's like some of the most basic stuff that we would like take for granted, like for anybody who knows how to use TypeScript apps and like use the npm ecosystem. and I'm just going through like the basics and like, it's like night and day on how they're able to like now accelerate the rest of the stuff they're doing.

'Cause then I, I, I set up Husky with them. so that way every agent will get blocked on committing, if it, like the s- the tests don't pass, the lint don't pass. Like if those things don't fix themselves, which I, I was joking that pair programming is like, it, it's, it's not a thing that's necessary anymore.

But what actually is necessary is the extreme programming stuff where like test-driven development is actually saving the bacon for all the folks that I'm like working with, like all my friends. Because like if the agent has to do tests and it has to pass linting and it has to like really think through like test coverage, the quality of what the output is so much better, because most of the folks who are driving and prompting have no idea.

they wouldn't even know what left is right, and like if they have a failure or things that are broken live in production, they have no way to trace that and no way to see how they got there. So the one benefit is like they have, if I can get them on tapes, at least they have the logs of the sessions to know how they got to this point and where the failure happened.

But if not even doing that, like they're basically like SLO.

[00:41:20] Jakob Heuser: Yeah. There's, there's like that you just mentioned both pieces of those dualities. There's tapes, which is how do we get there?

[00:41:25] Brian Douglas: Yeah

[00:41:26] Jakob Heuser: a t- this entire layer with, with Husky, with pre-checks, all these things that say, "Hey, agent, you need to be self-healing. If you bump into an electric fence and get shocked, you need to fix yourself before you continue on."

[00:41:38] Brian Douglas: Yeah

[00:41:39] Jakob Heuser: Freaking awesome. So I know obviously papercompute.com, Concepts Telemetry for Agents, tapes.dev, the open source project that's out there, so that people can actually ask, "Why did my agent end up where it did?" And briandouglas.me for your personal site. Where else on the internet can they find you? I know you're bdougie pretty much anywhere.

we're gonna put all that in the show notes, but where else can they find

[00:41:59] Brian Douglas: Yeah, I think the easiest is b.dougie.dev. So,

[00:42:02] Jakob Heuser: Okay

[00:42:03] Brian Douglas: so b.period. or whatever, you know what I'm saying. b.dougie.dev, that is like my centralized social experiment. so obviously my BlueSky is there, my LinkedIn's there. Like whatever your flavor, I, I do answer DMs, so like hit me up, ask me questions, and if you want a demo, please find me.

I can show you a demo on tapes and show you some value pretty quickly

[00:42:26] Jakob Heuser: Awesome. Everybody, Brian Douglas with Tapes.dev, the paper compute company, making it a lot easier for agents to figure, to figure out what the heck agents are actually doing. Brian, thank you for being on the show. I'm Jakob Heuser, CTO at Taskless. This has been The Task at Hand. Looking forward to continuing to talking to more builders and makers as we do stuff. See you next time