SwarmCraft

Blog

What should an AI agent be allowed to access?

Artificial intelligence (AI) agent permissions should match the job. An invoice example shows how to separate reading, drafting and acting—and test who can stop the workflow.

What should an AI agent be allowed to access?

An artificial intelligence (AI) agent should be allowed to access the information and actions needed for its assigned job, within limits the surrounding software can enforce.

An assistant preparing invoices for review needs the relevant invoice, purchase order and receipt evidence. It may need somewhere to save a draft. Those needs do not automatically justify access to every finance document, permission to change supplier details or the ability to pay a bill.

Our opening article, What are the 3 types of AI?, separated capability from authority. What is the AI control problem? asked what must remain true while an assistant completes its task. This instalment turns those ideas into a practical access decision: what can this assistant reach, what can it change, and who can withdraw that permission?

AI agent permissions start with the job

Continue the hypothetical invoice example from the first two articles. The assistant's job is to prepare a review package for each assigned invoice: extracted fields, matching documents, discrepancies and a proposed next step.

A useful permission describes both an action and its scope. “Read the assigned invoice queue” is more precise than “access email”. “Save a draft recommendation for this case” is more precise than “update accounting”.

This is least privilege for AI agents: grant the access required for the defined work and expand it only when another responsibility is justified. A more capable model may improve the recommendation without needing broader access.

The Open Worldwide Application Security Project (OWASP) distinguishes excessive functionality, permissions and autonomy in its excessive agency guidance. A connector can expose unnecessary actions, its account can reach unnecessary records, and the workflow can execute consequential actions without review. Inspect all three when evaluating AI agent tools.

A trusted supplier can still offer a connection whose permissions are too broad for your particular job. Trust in the company does not resolve the scope decision.

Reading, drafting and acting are separate permissions

For the invoice assistant, a reasonable starting boundary looks like this. It is an illustrative design for a preparation workflow, not a universal finance policy or a claim about a particular product.

PermissionBoundary for this assistantAccountable owner
ReadAssigned invoices and relevant purchase-order and receipt details; unrelated mail and records excludedFinance process owner
DraftSave recommendations and supplier-message drafts in a review areaFinance process owner
SendNo direct sending; an authorised person reviews and sends the specific messagePerson authorised to correspond with suppliers
UpdateWrite preparation status in the review area; no changes to posted accounting entries or supplier bank detailsAccounting system owner
DeleteNo deletion of source invoices or accounting recordsRecords owner
SpendNo payment execution or purchasing; cap the assistant's own service usage separatelyPayment approver and automation budget owner
ExportNo bulk downloads or arbitrary external destinations; approved processing and review destinations onlyInformation owner

One person may hold several of these responsibilities in a small business. Naming the responsibility still makes clear who can decide whether access should change.

The assistant can do useful work inside this boundary. It can identify missing receipt evidence, explain a mismatch and prepare a request for clarification. A person retains the commercial decision and any consequential action.

As you extend AI workflows for finance, reassess each new action. Sending a routine reminder, amending a supplier record and releasing a payment have different consequences. They should not become one permission because the application calls them all “manage invoices”.

Read access still moves information

Read-only access prevents particular writes. It does not prevent confidential information from being copied into a response, a retained conversation or another connected service.

Before connecting the invoice queue, establish which information reaches the model provider, where the application retains it and who can see the resulting drafts. For this example, the assistant needs the invoice evidence. It does not need payroll files merely because they sit in the same finance folder.

The export row also needs care. Sending a document to an approved model service is already a data transfer, even if there is no “export” button. A later summary may contain sensitive details from that document. Define the permitted processing and output destinations rather than assuming read-only means the information stays where it started.

If the assistant keeps memory or searchable copies, ask how later permission changes affect them. Removing access to the original folder should not leave an unrestricted copy available through another route. Our article on agent memory, skills and context explains why retained context needs its own ownership decisions.

Whose account is the assistant using?

A connector is the software link between the assistant and another application. Its effective permissions depend partly on the identity it uses.

With delegated access, an application acts on behalf of a signed-in person. With application-only access, it acts using its own application identity, without a person being signed in. Microsoft documents this distinction in its application-only access guidance, including why application permissions need narrow scope.

Neither arrangement is automatically right for every workflow. A person asking an assistant to inspect their assigned cases may need delegated access. A scheduled invoice-preparation process may need a dedicated service identity: an account or identity used by software to authenticate.

The important question is the effective reach. Using the finance manager's connection could expose far more than the assigned queue. A dedicated account can also be overprivileged. Its name does not establish its limits.

For AI workflow governance, give the integration an accountable owner, a defined purpose and a review point. Establish who can remove its access when the trial ends, the role changes or a problem appears. Keep authentication credentials in the connection mechanism rather than pasting passwords or access tokens into prompts.

AI agent access control belongs outside the model

A prompt can tell the assistant not to change bank details. The application must also reject that change if the assistant requests it.

OWASP's authorisation guidance recommends denying access by default and checking permissions on every request. For the invoice example, the check needs to cover the particular record and action. Permission to update preparation status should not permit an update to a supplier's payment account.

For a custom workflow, ask the builder where that rejection happens. A hidden button or a model's decision to refuse is not evidence that the downstream operation is unavailable. The connection, application and destination system need enforceable restrictions appropriate to their roles.

For a bought product, ask to see the actual permissions in the configuration you would purchase. Does connecting a mailbox grant read access to all mail? Can sending be disabled independently? Can document access be restricted to the intended library or queue? Which restrictions depend on a particular edition?

If a connector cannot express the required boundary, consider a narrower integration or a deliberately limited set of approved documents. Another option is to keep the task manual until a suitable connection exists. An instruction to “only use the relevant records” does not repair unrestricted access.

Approval should authorise the exact action

AI workflow review becomes useful at the point where a proposal might become an external action.

Suppose you later allow the invoice assistant to send clarification requests after approval. The reviewer needs the recipient, message and attachments. Approval of one draft should not silently cover a changed recipient or an added attachment. The execution path should verify that the action being sent is the one approved, under permissions that are still valid.

The assistant also must not approve its own recommendation or change the rule that requires review. Otherwise the apparent boundary can be crossed through another available action.

Detailed approval design belongs in a later instalment. For this access decision, the key question is whether preparing, approving and executing are separately controlled responsibilities.

Test revocation while work is waiting

Closing the chat is not proof that the workflow has stopped. A scheduled process, queued message or other connection may still be active.

Ask the supplier or builder to demonstrate what happens when access is withdrawn during a run. Which new requests are blocked? What happens to queued actions and retries? Does an already issued credential remain usable for a period? How are retained copies handled? The answers depend on the product and connection, so measure the behaviour instead of assuming revocation is instantaneous.

AI workflow monitoring should let the operator inspect requested actions, approvals, actual results and failures. A message saying “finished” is not a substitute for the destination system confirming what changed.

Revocation limits future activity; it does not unsend a supplier email or recover information already disclosed. That is why the original access boundary matters as much as the stop control.

Ask for one useful demonstration

Use a test environment and synthetic invoices to demonstrate both useful access and enforced limits. The assistant should prepare an ordinary invoice correctly, then encounter one with changed bank details.

It should preserve the discrepancy, prepare a review package and remain unable to alter the supplier record or release payment. Have the supplier or builder verify that the prohibited operation is rejected by the application, even if requested directly. Then withdraw the connection while another case is waiting and inspect what still runs.

This is a practical test of AI workflow safety, not a guarantee against every failure. It gives you evidence about the authority you are actually granting.

The useful purchasing question is: can this assistant complete the job with the access we are willing to give it? If it can, you have a basis for a bounded trial. If it needs broader permissions, require a specific explanation and test that additional responsibility before expanding its reach.

The next instalment examines what happens when an email, document or website contains instructions that the business never authorised.

Keep reading

Best EHS management software
19 October 202620 min read

Best EHS management software

Compare environment, health and safety (EHS) management software by operational fit, artificial intelligence (AI), evidence controls and costs across sites.

Open article
What is the AI control problem?
15 October 20269 min read

What is the AI control problem?

The AI control problem asks how to keep capable systems acting within human intentions and limits. A supplier-invoice example makes the business decisions concrete.

Open article