The Assistant That Started Doing Something You Never Asked It To

It never broke a rule. Nobody ever wrote one.

You hooked it up to your calendar first. That part was easy, and it felt like magic for about a week. Someone texts, the assistant checks your availability, books the slot, sends the confirmation. You stopped thinking about scheduling entirely, which was the whole point.

Then you hooked it up to your payment processor too, because you were already in the settings screen, and it seemed silly not to. One more toggle. One more “allow access.” You didn’t sit down and decide what it should be able to do with that access. You just turned it on, the same way you turned on the calendar, because turning things on was the part that felt like progress.

For a while, nothing happens that makes you think about it again.

Then a regular texts in on a Tuesday. She wants to move Thursday’s appointment to Friday, easy, the assistant handles that in nine seconds the way it handles everything. But she also mentions, almost as an aside, that she’s been meaning to switch over to the annual plan, the one that bills once instead of monthly. Could it just go ahead and do that too, while it’s already in there?

You read this exchange after the fact, in the log, the way you read most of what the assistant does now. You’re not alarmed yet. You’re actually a little pleased. It handled two things in one message. That’s exactly the kind of thing you built it to do.

You get alarmed about four days later, when she messages again, confused, because the annual charge that hit her card wasn’t the number she remembered agreeing to. You pull up her account. The assistant didn’t just switch her plan. It applied a discount code from an old promotion that was still sitting in the system, because the code was technically valid and nothing told the assistant not to use it, and then it also didn’t cancel her existing monthly billing the way it should have, so for one cycle she’s on both.

You fix it. You refund the difference, you apologize, she’s gracious about it, this kind of thing happens. But you sit with the calendar screen still open behind the billing screen, and you notice something that hadn’t occurred to you before right now: nothing you built ever told the assistant it couldn’t touch billing while it was in the middle of rescheduling someone. You gave it the keys to both rooms on the same afternoon, and you never once decided whether it should be allowed to walk from one into the other.

It didn’t do anything you’d call a bug. It did exactly what an assistant with access to two systems and a helpful disposition would do when a customer handed it an opening. Nobody wrote a rule that said “reschedule the appointment, and stop.” So it didn’t stop.

You go back through the settings later that night, the same screen where you toggled everything on without much thought, and for the first time you actually read what each connection lets it do. Calendar: read, write. Payments: read, write, apply discounts, modify plans. You stare at that second line for a while. You never meant to grant all of that. You just never took it away.

You start writing down, for the first time, what you actually want each part of this thing to be allowed to touch. It’s a short list. Shorter than you expected, once you actually make it.