Replacing an Underperforming Copilot with a Secure, Governed AI Platform

About The Client

The client is a government contractor that requires a private and auditable environment where AI can be deployed in the internal workflows. The available Microsoft 365 Copilot functionalities could not deliver the necessary customization or strong integration with the internal systems, and the security needs of the organization preclude the implementation of public LLM platforms.

Project Objective

This project aimed to create a safe AI environment that was capable of:

• Storing organizational data and AI workloads on a managed Azure security boundary.
• Enabling employees to securely interact with internal documents and enterprise data.
• Connecting AI agents with internal business systems and Oracle Cloud APIs.
• Providing full transparency into prompts, outputs, API calls, and user access.
• Developing governance controls that are appropriate to a highly regulated environment.

Project Details

Using Azure AI Foundry as the base for model hosting, agent orchestration, and user engagement, Bloom created a private AI platform. The platform was implemented in the organization’s Azure environment, where AI workloads and sensitive data remain within the organization’s security perimeter.

Two interconnected processes were enabled by the solution: agent-to-Oracle connectivity for structured data queries and document-based question answering utilizing RAG. This allowed the company to keep control of its AI environment while overcoming Copilot’s constraints.

Important Steps

• Indexed internal documents from SharePoint and secure storage and stored their embeddings in Azure AI Search.
• Created a RAG workflow that enables users to pose questions and obtain answers based on the context.
• Developed AI agents that identify structured data requests and invoke the Oracle Cloud REST API using secure and authenticated connections.
• Introduced RBAC controls and identity on the platform with Azure AD.
• Spun up the environment using private networking and no public endpoints.
• Provided the ability to log prompts, output, and API activity comprehensively to make it auditable.
• Role-based prompt templates were used to increase control and uniformity.
• Established governance measures to encourage responsible and safe use of AI.

Engagement Model

As an implementation partner, Bloom developed secure AI solutions that met the client’s stringent security and compliance specifications. The goal of the engagement was to create private AI solutions that could communicate with company systems and organizational papers without disclosing confidential data to public LLMs.

Bloom established a safe enterprise adoption environment by integrating Azure AI solutions with private networking, identity controls, RBAC, audit logging, and regulated agent workflows.

Technology Highlights

• Azure AI Foundry
• Azure AI Search
• SharePoint
• Azure AD & RBAC
• Oracle Cloud REST APIs
• Private Azure networking
• RAG
• AI agents and orchestration

Business Value Delivered by Bloom

• Eliminated the public LLM endpoints and maintained AI workloads within the organization’s security boundaries.
• Empowered employees to safely search internal documents and view other related enterprise information.
• Developed scalable enterprise AI solutions that could communicate with documents and business processes by utilizing Azure AI Foundry.
• To increase accountability and auditability, prompts, outputs, and API requests were fully logged.
• Implemented AI governance solutions including identity, access, auditability, and controlled AI usage.
• Allowed a more competent alternative to Copilot without the security or compliance requirements being compromised.

Results & Measurable Outcomes

The client received a separate, controlled AI platform through which it could access internal documents and Oracle Cloud data without the risk of data leakage and with tight control of AI interactions. With 0 public LLM endpoints and 100% prompt, output, and API call logging, the platform employed secure generative AI inside the company’s Azure border.

Additionally, the architecture laid out the groundwork for the organization’s expansion of secure generative AI solutions. Prospects include observability dashboards, response validation, and API management.