Keep AI data in your own tenant: Azure OpenAI, LiteLLM, and self-hosted endpoints

Short answer: Most AI purchases do not fail on the demo. They fail three weeks later in the security questionnaire, on three questions: where does our data go, whose keys are used, and who can see what. Crew Orbit answers the first with configuration rather than a caveat. AI Assist credentials accept a custom HTTPS base URL, so Ask, description optimize, and knowledge embeddings run against your Azure OpenAI deployment, your LiteLLM gateway, your corporate proxy, or your own OpenAI-compatible host. The second is bring-your-own keys, stored encrypted and never returned to a client. The third is layered tenant isolation, permission-filtered retrieval, and an audit trail you can walk somebody through.

One thing this post will not do is tell you whether that satisfies your obligations. Crew Orbit is a platform, not a law firm and not an auditor. What follows is a description of product capability, in enough detail that your legal, IT, and procurement people can map it onto whatever framework they are working from.

The demo was never the problem

The pattern is familiar to anyone who has sold or bought engineering tooling in Europe. An engineering lead runs a pilot, it works, the team is enthusiastic. Then the request goes to procurement, and a different set of people ask a different set of questions:

• Which processor receives the prompt text, and where is it processed?
• Are we using the vendor's model account or our own?
• Can a developer on Project A retrieve a document that belongs to Project B?
• Is the text of what people ask the AI stored anywhere?
• Who changed this configuration, and when?

These are good questions. The reason they kill deals is not that the answers are always bad. It is that they are usually architectural, and by the time somebody asks, the architecture is fixed. A tool that routes every request through its own model account cannot become a tool that does not. The only way to pass this stage is to have made different decisions earlier.

Point the AI at your own endpoint

In Crew Orbit, AI Assist credentials for OpenAI and Anthropic carry an optional Use custom endpoint setting: a public HTTPS base URL plus either Bearer authentication or an Azure API key with an api-version. It is surfaced as its own settings section rather than buried in an advanced accordion, because for a lot of buyers it is the first thing they need to see.

With a base URL set, three surfaces use that endpoint instead of the vendor host:

• Ask Crew Orbit: the chat surface, including @crew in rooms and work-item comments.
• Description optimize: the AI Assist path that sharpens a work-item description before anyone submits it.
• Knowledge embeddings: the vectors behind your document library and every cited answer.

That covers the surfaces where free-text business content is most concentrated: the questions people ask, the specs they refine, and the documents they index. Common configurations are an Azure OpenAI deployment in a European region, a LiteLLM gateway that fronts several models with your own logging and rate policy, a corporate proxy that already carries your egress rules, or an OpenAI-compatible model you host yourself.

A few operational details that matter more than they sound:

• Egress is SSRF-guarded and the base URL must be a public HTTPS address. This is a real constraint, not a formality: you cannot point Crew Orbit at http://10.0.4.19:8000. A self-hosted model needs to be reachable over HTTPS at an address the guard accepts, typically through the same gateway or reverse proxy you would already put in front of it.
• Changing endpoint metadata resets the last-test result and probes the new host. You do not get to keep a green badge earned against a different server.
• The credential row shows the hostname when an endpoint is custom. Somebody reviewing settings can see at a glance which rows leave your tenant and which do not.
• Secrets are never returned to a client after save. Not masked-with-a-reveal-button. Not returned.

The honest limit: worker CLI providers stay on vendor hosts

Custom endpoints cover the AI Assist surfaces listed above. They do not currently redirect the CLI providers that execute AI runs. Those still talk to their vendor hosts. If your procurement question is specifically about the code-generation step inside a delivery run, the honest answer today is that this control does not cover it.

We would rather you learn that here than in month two. It is also a narrower gap than it first looks, because the content profiles differ: run execution operates against your repository under your Git credentials, while the free-text surfaces most reviewers worry about (the questions, the specs, the indexed documents) are exactly the ones a custom endpoint does cover. But narrower is not zero, and if your requirement is absolute across every surface, you should evaluate on that basis.

Bring your own keys, and your own provider relationship

Crew Orbit uses your OpenAI and Anthropic credentials, stored encrypted in the platform credential store at organization or project scope. There is no platform model account in the middle, which has three consequences worth naming.

First, your contractual relationship with the model provider is yours. Whatever terms, region commitments, or data-processing arrangements you have negotiated apply to your traffic, because it is your account making the calls. You are not inheriting a sub-processor relationship you did not choose.

Second, there is no opaque resale margin on tokens. You see provider spend where you always see it: on your provider bill.

Third (and this is the one auditors tend to like): there are no platform-owned embedding keys. Knowledge embeddings are computed with your credential, against your endpoint if you configured one. There is no Crew Orbit-owned embedding path that quietly processes your documents on the side.

Credentials live at organization scope for shared integrations, with projects either inheriting or holding their own project-only secrets, so a sensitive project can use a different key from the rest of the workspace without duplicating everything. This sits on top of the same org and project structure described in scaling AI work across orgs, projects, and permissions.

Retrieval respects permissions, and queries are not logged in the clear

"Who can see what" is the question that most often gets a hand-wavy answer, because retrieval-augmented answers make it genuinely hard: the model produces one paragraph, and the paragraph does not obviously carry the permissions of the twelve chunks behind it.

In Crew Orbit, retrieval is filtered by what the caller is allowed to read, computed on the server. A project-scoped session searches workspace-wide documents plus that project's. A workspace-scoped session searches workspace-wide documents plus the projects the person actually has read permission on, and the allow-list is computed server-side, never supplied by the client. Projects excluded from a search are disclosed rather than silently dropped, so a user can tell the difference between "there is nothing about billing" and "you cannot see the project where billing lives".

Non-members of an organization receive a 404 rather than a 403, so the API does not leak the existence of resources they have no business knowing about.

And the retrieval audit trail stores a hash of the query, not the question text. You keep the operational signal (volume, patterns, repeat queries, isolation checks) without accumulating a searchable archive of every sensitive thing an employee typed into the AI. If you have ever had to answer "what exactly do you retain about user prompts?", this is the shape of answer that ends the thread.

Tenant isolation an auditor can actually follow

Isolation claims are cheap. What a reviewer wants is a mechanism they can understand well enough to poke at. Knowledge tenancy in Crew Orbit is treated as a precondition rather than a quality target, and it is built in reinforcing layers:

• One authority for scope. A single resolver is the sole factory for the scope token that every retrieval carries. Controllers and tools cannot construct one. The organization id comes from the authenticated principal and the authorized path, never from a request body, header, query parameter, tool argument, or connected tool server.
• Storage partitioned by organization. Knowledge chunk and audit rows are keyed by organization, and retrieval and purge always query a single partition. Visibility and project filters run inside that partition: they are least-privilege axes, not the tenant boundary itself.
• An IAM deny that survives application bugs. The task role carries a policy that denies table scans and PartiQL on the knowledge chunk and audit tables. That denial is evaluated by the cloud provider even if application code regresses. A deployed environment refuses to start if knowledge is enabled and the deny policy is missing.
• No alternative access paths. The chunk and audit tables have zero secondary indexes, because a secondary index would reintroduce a read path that is not tenant-partitioned. Infrastructure tests fail if one appears, and exactly one repository class is permitted to touch the chunk table, enforced by architecture tests and a CI check, not by convention.
• A canary, plus a defined response. A nightly in-process canary probes for cross-tenant reachability. A clean run emits a pass metric. A breach raises a P1 alarm and flips the knowledge kill-switch off fleet-wide, with a written incident procedure: preserve logs, page on-call, do not redrive queues until root-caused.

Two related controls tend to come up in the same conversation. Stored objects use envelope encryption with the organization id in the encryption context, so decryption with the wrong context fails rather than succeeding quietly. And documents, fetched URLs, and content returned by connected tool servers are treated as untrusted data, never as instructions, with human approval required before any write or external action.

Credential testing: fail before you waste a run

This one is not usually on the questionnaire, but it is the control your engineers will thank you for. Git tokens and AI credentials each have an explicit Test action in settings, and every row carries a status badge: never tested, OK with an age, failed, or stale. Inherited organization rows show the organization's last test result. The test itself runs at the scope that owns the credential.

The status is not decorative. Submitting or scheduling a run hard-blocks when the selected AI worker credential or any git connection the work item uses is failed, never tested, or stale (where stale means more than seven days since the last successful test), and the error deep-links to the settings row that needs fixing. AI Assist credentials are deliberately not part of that gate, because a broken Assist key should degrade a chat answer, not block delivery.

The alternative is the thing everybody has actually experienced: a rotated token discovered forty minutes into a run, and a bill for the discovery.

Who changed what, and when

Configuration is the part of a security posture that decays. The endpoint was right at signature. Six months later somebody changed it and nobody remembers who or why.

Crew Orbit keeps permission-gated Created and Last edited stamps plus a History sheet on work items, projects, project settings, and labels. Timestamps stay visible to anyone who can read the entity. Actor identity and the history API require an explicit audit-read permission, which can be granted to a custom role without also granting settings management. A compliance reviewer can therefore see history without gaining the ability to change anything. The History sheet shows recent events with field-level diffs and pages further back on request. Soft and hard deletes emit events too.

Worth knowing up front: history starts when the feature ships for you. There is no backfill of changes made before that point.

What this is not: a certification

Crew Orbit holds no SOC 2 report, no ISO certification, and no attestation of any kind. We are not going to imply otherwise with a badge or a carefully worded "aligned with" sentence, because that ambiguity is precisely what wastes a security reviewer's afternoon.

The honest framing, which we use consistently: Crew Orbit supports your security program. It does not replace it. No platform can. What a platform can do is make your answers correspond to observable mechanism instead of intention. When you tell an auditor "AI processing happens at this endpoint", there is a configuration row with a hostname on it. When you say "retrieval follows permissions", there is a server-computed allow-list rather than a client-supplied one. That is the difference between a questionnaire answer you can defend and one you have to hedge.

Equally, nothing here is legal advice. Whether a given configuration meets your obligations under the GDPR, the EU AI Act, sectoral rules, or your own customer commitments is a determination for your counsel and your data protection officer, with your processing purposes and your risk assessment in front of them. Our job is to make the technical facts precise enough for them to work with. The same reasoning applies to the broader governance story in how you govern AI changes before the pen test asks.

What to put in your own questionnaire

If you are the one evaluating AI tooling (for any vendor, not just this one), these are the questions that separate a configurable posture from a fixed one:

• Can I direct model traffic at an endpoint I control, and which surfaces does that cover? A yes that covers one of six surfaces is a different product from a yes that covers all of them. Make the vendor enumerate.
• Whose provider account is billed and contractually engaged: mine or yours?
• Are embeddings computed with my credential, or does the platform hold its own embedding key?
• Is retrieval filtered by per-user permissions, and is the allow-list computed server-side?
• Is query text retained? In what form, for how long, and who can read it?
• What is the tenant boundary in storage terms, and what stops an application bug from crossing it?
• Can I see who changed a security-relevant setting, without also being able to change it?

A vendor who answers those crisply has thought about your security review. A vendor who answers them with adjectives has not.

Get through the security review

Where the data lives is one half of the question. What the AI is allowed to do with it is the other. Read how to give an agent read access without write access, and why grounded answers should cite their sources for how retrieval stays inside a person's permissions. For the wider procurement framing, see governing AI changes before the pen test asks.

If legal and IT are the reason AI adoption has stalled in your organization, the mechanisms above are the ones to put in front of them. Start at crew-orbit.com.