One sentence, one table, sixty minutes, then the login is gone

Incidents are when production data access is most needed and least controlled. A settlement batch is late, merchants are waiting, and the engineer on call needs to look at the invoices table now. In most organisations the answer is a shared read-only database account whose password lives in a vault, a wiki page or someone’s shell history, and which nobody has rotated since it was created. It works. It also means that anyone who has ever been on call can still read payment data, and the auditor’s question “who read this table, and why?” has no good answer.

This post follows one request in the demo organisation, Northwind Labs, from the sentence an SRE types to a PostgreSQL login that exists for sixty minutes, can read one table, and is removed by software running inside Northwind’s own network. Alongside it we look at a riskier neighbour, a request to change an invoice, to show where the same machinery slows down on purpose.

The business problem: shared accounts and long memories

Production databases that hold payment or personal data are where a leaked credential costs the most. The exposure does not come from the one engineer who needs access tonight. It comes from the accumulated access of everyone who needed it on some night in the past three years and still has it. Shared accounts make that worse in two ways: they cannot be attributed to a person, and they cannot be removed from one person without breaking everyone else.

The operating cost is real too. When access goes through a ticket queue, the incident waits for a database administrator to read the ticket, create a login, and remember to drop it afterwards. The usual outcome is that the step is skipped, or the login is created “for the week” and forgotten. Auditors testing PCI DSS or SOC 2 access controls then sample those logins and find that approval and removal are not evidenced.

Step 1: Priya asks in one sentence

Priya is an SRE and is on call for the payments team. Incident INC-4412, “Settlement batch delayed for EU merchants”, is open at severity 2. Priya wants to see whether invoice statuses for the affected merchants are stuck. In the console, in Slack or with the command-line tool Priya types:

insightkeys request "read the invoices table in the payments database for INC-4412, 1 hour"

The intake agent drafts the request. It matches “payments database” to payments-db (Postgres), a real PostgreSQL database in the catalogue, rated critical, holding payment and personal data and owned by payments.lead@. It picks the Read entitlement, takes one hour as the duration and INC-4412 as the ticket, and uses the sentence as the reason. It only ever picks resources from the catalogue; it never invents one. Priya sees the draft and confirms it. This becomes request AR-0008.

Step 2: the scoper narrows, the risk agent scores

The Read entitlement on payments-db has a ceiling of SELECT on *.*: every table in every schema. That ceiling is what the data owner agreed anyone might ever be given for reading. The least-privilege scoper narrows inside it. The sentence named the invoices table, so the permission becomes SELECT on billing.invoices and nothing else. The scoper runs at autonomy level L3, “act and tell”: it applies the narrowing, records it on the request timeline, and the change can be pulled back by a person.

The risk agent then scores the narrowed request. Every point is explained:

Read access adds nothing. Northwind’s routing thresholds are the defaults: auto-approve at 30 or below where the flow allows it, step-up at 55, two approvers at 75. The first flow that matches is #10, “Read access: automatic when the risk is low”, with a four-hour maximum. A score of 30 is within it, so the request is approved automatically. Notice what earned the approval: not Priya’s seniority, but the narrow scope, the short window, the open incident and the fact that Priya is the person on call. Ask for every table for eight hours on a quiet Sunday and the same flow would send it to the owner.

Step 3: the broker creates a login in Northwind’s network

InsightKeys’ servers do not hold a credential that can create database logins. The broker, a small service Northwind runs next to its databases, does. It holds a PostgreSQL role with CREATEROLE, connects out over HTTPS to collect instructions, and accepts only instructions signed with Northwind’s organisation key. Before acting, it checks its own local policy file, broker.yaml, which Northwind owns: is this integration allowed, is the duration within the longest grant it will create, is the broker paused.

AR-0008 from sentence to removal. The granting credential stays with the broker inside the customer network.

In effect, the broker runs the equivalent of the following against payments-db. The role name in the demo is ik_165ab58803c0; the password is random and never chosen by a person:

CREATE ROLE ik_165ab58803c0 LOGIN PASSWORD '…' VALID UNTIL '<one hour from now>';
GRANT USAGE ON SCHEMA billing TO ik_165ab58803c0;
GRANT SELECT ON billing.invoices TO ik_165ab58803c0;

The password is sealed at rest, shown to Priya once on the grant page, then wiped. If it is lost, Priya asks again; there is no “show password” button a day later, because there is nothing left to show. VALID UNTIL is the database’s own expiry on the password, a second lock behind the broker’s clock.

The grant page is also the record. It shows who holds the access and why, when it started and ends, the permission, what the broker created, and every signed broker instruction with its status. An admin can revoke it from the same page with one click; so can Priya when the work is done.

The neighbour: fixing one invoice is a different request

The next day Sam, an engineer on the platform team, needs to correct one invoice status. Sam asks for “write access to fix one invoice status in the payments database for 30 minutes, OPS-219”. This is AR-0004. The Write entitlement’s ceiling is DELETE, INSERT, SELECT and UPDATE on *.*; the scoper narrows it to INSERT, SELECT and UPDATE on billing.invoices. No DELETE, no other tables.

The score is 62, High: production +30, critical +20, write +12, sensitive data +8, 30 minutes −3, first time +5, narrow scope −10. Sam is not on call for payments and OPS-219 is not an open incident, so neither credit applies. At 55 or more, step-up is required: Sam confirms it is Sam with an authenticator app before anyone is asked to approve. Then flow #20, “Write access: the owner approves”, sends it to payments.lead@, who owns the resource.

How Northwind’s flows route database requests. Thresholds are the defaults and are editable in Settings.

The approver sees the sentence, the ticket, the score with every factor, and the exact permission: three verbs on one table for thirty minutes. That is a decision a resource owner can make in a minute from Slack or the console. It is very different from “please add Sam to payments_rw”, which nobody can evaluate.

Step 4: expiry, with the network or without it

When the hour is up, the broker revokes on its own clock. It keeps a local ledger of every grant it made, so it does not need the control plane to remind it. It terminates any sessions the role has open, so a long-running query or an idle psql window does not outlive the grant, and drops the role. Then it reports the grant as expired. The control plane sends its own revoke as well; re-delivery is idempotent, so the second one is harmless.

This matters for two failure cases a security reviewer will ask about. If the InsightKeys service is unreachable, expiry still happens. If the control plane were compromised and stopped sending revokes, expiry would still happen. And even if the broker itself were down at the moment of expiry, the password stops working at VALID UNTIL; the broker cleans up the role when it returns. Revocation is never refused, even when the broker’s pause switch is on.

If Priya needs more time, the grant is not extended. Priya makes a new, smaller request, which is scored again with what is true now. If the incident has closed, the incident credit is gone and the answer may be different. That is intended.

What changes, and what to be careful about

For the SRE, the hour starts within moments of asking instead of after a ticket is read. For the data owner, the only requests that reach them are the ones that change data or reach further than the flow allows, and each arrives already narrowed and explained. For the auditor, every access to payments-db has a person, a reason, a ticket, an approval (automatic, with the rule that allowed it, or a named approver), a start and an end. Grants and reviews can be exported to InsightAssure as evidence for access-control requirements.

Keep reading