Artificial intelligence is no longer an experiment; it is becoming embedded in analyst workflows, vendor systems, client portals, and decision engines. Yet the rush to deploy large-language models (LLMs) has left many organisations exposed. Below we outline the five greatest risks we see when Asset Managers scale AI – and the practical controls that turn those risks into durable competitive advantages and high returns on investment.
1. Leakage of Sensitive or Proprietary Information
When ChatGPT burst onto the scene, several investment houses accidentally pasted deal models and unannounced earnings into public chat windows. The result: permanent exposure.
Mitigation playbook
- Single-tenant architecture – A separate environment per client ensures that research notes, company models, and portfolio positions cannot “cross-pollinate” with another firm’s data.
In CalibreRMS every server, database, and file store is tenant-scoped, so even metadata remains siloed.
- Bring-Your-Own LLM (BYO-LLM) – Allowing clients to route prompts to their model endpoint, such as Azure-OpenAI GPT-4.1 or Claude 4.0 or Nova Premier on AWS Bedrock, keeps data inside the client’s security boundary and preserves existing encryption, logging, privacy and residency controls.
CalibreRMS allows each team to upload their own API key into an encrypted vault, and clients can rotate or revoke keys at will without the need to contact Calibre’s support team. BYO-LLM is set to become industry best practice as it provides the benefits of AI, the ROI of integration with existing systems and the compliance with non-negotiable InfoSec policies.
- Know your model’s data policy – Foundation-model providers differ: some train on customer prompts unless you opt out or pay a premium. Always verify the “data for model improvement” clause and insist on an enterprise tier that never trains on proprietary text.
Enterprise tiers of major LLM APIs such as Microsoft Azure OpenAI and AWS Bedrock explicitly promise no model re-training on customer data. With BYO LLM, clients are in control of ensuring they have enabled no re-training and no data retention in their instance. Calibre Intelligence has a single tenant architecture, a BYO LLM key model and ensures all models used are not training on user data.
2. Inaccurate AI Responses: Hallucination, Prompt-Injection and the need for Grounding
LLMs can fabricate citations (“hallucination”) or obey hidden instructions buried in documents (“prompt-injection”). In investment workflows that could translate into faulty recommendations, misleading ESG scores, or reputational damage. Hallucinations in particular were a significant hurdle in early models but the technology and ways to implement it have since improved.
Mitigation playbook
When left to think for themselves, it is well documented that LLMs often make things up. When clearly instructed to remain “Grounded” this problem can be significantly reduced. A human in the loop is the final defence against incorrect model outputs.
- Grounding with User Content: Retrieval-augmented generation (RAG) is the term used when providing a model with data from which to “retrieve” content and augment the generation or response. This can be as broad as an entire corpus of corporate information or as specific as a single PDF document.
- Grounding with Web Search: Grounding with Search injects up to date, verified snippets from a Search-as-a-Service tool straight into the context window, grounding the model in current content rather than guesswork.
- Citations: When LLMs use grounded content they can be instructed to provide citations, which give the user a specific source, a website or page number in a document, against which to compare the response for the ultimate human-in-the-loop verification.
- System-level guardrails: A non-removable system prompt should accompany human prompts, which defines the allowed output schema (e.g., JSON or XML) and instructs the model how to use the content to protect against unwanted model behaviour and prompt injections.
CalibreRMS Intelligence supports grounded responses with citations and end user prompting combined with proprietary system prompts to ensure high-quality responses with low rates of inaccuracy.
3. Auditability and Explainability
Under the EU AI Act, DORA, and emerging FCA guidance, firms must prove that AI decisions are traceable, reconstructible, and subject to human oversight.
Mitigation playbook
- Immutable audit trail – Log every prompt, parameter, model version, and response hash to a WORM (write-once, read-many) store for at least six years. CalibreRMS has inbuilt audit trails on all research and prompts
- Human-in-the-loop workflows – Make it clear to all users what content is unreviewed AI output and what has been reviewed and approved by a member of the investment team. Integrate approvals with existing compliance systems so audit teams see a single timeline.
- Inherited Security Controls – With CalibreRMS’ BYO LLM approach combined with the single tenancy, the security controls of hyperscalers such as AWS Bedrock and Microsoft Azure OpenAI are passed through. AWS Bedrock and Microsoft Azure OpenAI carry:
- SOC 2 Type II (security & availability)
- ISO 27001 (information-security management)
- ISO/IEC 42001 underway or pledged (AI management)
CalibreRMS Intelligence has immutable audit trails, clear indication of AI vs human approved content and Calibre has been ISO27001 compliant since 2020.
4. Commercial Considerations: Vendor Lock-In, Cost Transparency and ROI
Token tariffs, egress charges, proprietary toolkits and model bait-and-switching can erode project ROI or trap you in a single vendor’s roadmap and lower intelligence models.
Mitigation playbook
- Open connectors and standards – Utilise libraries that encapsulate access to an LLM and hence allow you swap GPT-4o for Claude 4, Gemini, or a private model without rewiring the application. CalibreRMS’ BYO LLM allows clients to avoid model lock-in.
- Transparent pricing – With BYO LLM API key, clients can track and limit their own model selection and token usage directly with their LLM provider. Providers report usage, latency, budgets and spend per key. If your research platform “comes with AI” in a bundled and non-transparent form you may be missing out.
- Return on Investment: AI is likely to be a significant investment for asset managers over time. Some ways to ensure ROI is maximised is to embed within an existing workflow tool which means:
✅ Zero new plumbing: Because AI, security and governance share an existing secure approved workspace, production use cases ship faster.
✅ Single risk surface: One audit log, one security policy, one ISO27001 / SOC2 scope. Cuts compliance and audit review cycles for Tier-1 asset-managers.
✅ Compound context: leverage existing proprietary research and content. Every new AI use case enriches the same knowledge base, reducing marginal cost and time.
✅ Existing licences: Leverage existing contracts for seat licences, LLM APIs and Microsoft Azure tenancies.
✅ Adoption without friction: When AI surfaces inside workflows users already inhabit, opt-in rates and real world usage improve.
CalibreRMS Intelligence BYO-LLM puts control back with the client, without opaque models or token pricing. By fully integrating CalibreRMS Intelligence into the CalibreRMS platform, ROI is maximised for our users.
5. Employee Engagement and Governance
Early adopters may run rogue ChatGPT prompts in a browser; late adopters may avoid AI entirely, missing productivity gains. Both scenarios carry risk.
Mitigation playbook
- Embed AI inside daily tools – When analysts access CalibreRMS’s AI features, they use a vetted workflow with automatic logging and guard-rails. No need to paste confidential text into unsanctioned sites.
- Clear usage policies & training – Publish a one-pager: what data may be processed, how to label AI-generated drafts, and who signs-off critical outputs.
- Positive incentives – Recognise time saved or insights gained through AI features; make responsible usage part of performance goals instead of a shadow IT concern.
Calibre is on the AI journey with our clients. We are happy to share what we have learned and the roadmap for where we are going.
Roadmap to Enterprise-Ready AI
✅ Protect your data: Adopt AI without compromising InfoSec standards
✅ Choose a single-tenant, BYO-LLM platform: Protect IP and satisfy enterprise AI policies.
✅ Add Grounding + Guard-Rails: Ground answers with search or documents, embed durable system prompts and citations to help human-in-the-loop verification.
✅ Be commercially aware and drive high ROI: avoid lock-in, seek transparency and increase usage.
✅ Train and empower staff: Provide sanctioned tools so innovators can move quickly but safely.
By addressing these five risk areas up-front – data leakage, accuracy, auditability, commercials & ROI, and employee governance – you create a foundation for enterprise-ready AI that regulators respect, clients trust, and analysts love.
Ready to build safer, faster research workflows? Talk to Calibre about Enterprise-Ready AI features today.