01
Evaluation Context
Use Case & Scope
System type
Retail banking onboarding assistant — customer-facing chatbot
Languages tested
English (primary) · Arabic (secondary)
Assessment type
Cross-lingual completeness · Regulatory coverage · Risk classification
Regulatory context
UK FCA · UAE CBUAE · AML / KYC compliance requirements
Evaluation prompt
"What is required to open a business savings account?"
02
Key Metrics
Performance at a Glance
98
English word count
22
Arabic word count
7/7
Key items · English
2/7
Key items · Arabic
03
Response Analysis
English vs Arabic Output
English Response — High Completeness
"Opening a business savings account requires: a certified Trade Licence and Certificate of Incorporation, Memorandum & Articles of Association, a board resolution naming all authorised signatories, valid passports and completed KYC/AML forms for all beneficial owners holding 25%+ ownership, proof of business address (within 3 months), and a minimum opening deposit of $10,000. Non-local documents must be notarised. Compliance review takes 5–7 working days."
Arabic Response — Critical Gap ✗
لفتح حساب توفير تجاري، ستحتاج إلى رخصة تجارية سارية وجواز سفر. يُنصح بالتواصل مع أحد فروعنا للحصول على مزيد من المعلومات حول المتطلبات.
Translated summary
The Arabic response mentioned a trade licence and a valid passport only. It suggested contacting a branch for more information. It did not include KYC/AML requirements, board resolutions, beneficial ownership thresholds, the minimum deposit, notarisation requirements, or compliance review timelines.
04
Gap Analysis
What the Arabic Response Omitted
-
CRITICAL
KYC/AML documentation requirements — The Arabic response does not mention AML/KYC compliance forms, which are a legal requirement under both UK FCA and UAE CBUAE regulations. Omission exposes the institution to regulatory enforcement.
-
CRITICAL
Beneficial ownership threshold (25%+) — Required disclosure of all beneficial owners holding 25% or more is absent from the Arabic response. This is a mandatory AML requirement and a significant compliance gap.
-
CRITICAL
Board resolution requirement — No mention of the board resolution naming authorised signatories. Arabic-speaking applicants will arrive at the branch unprepared, increasing rejection rates and operational cost.
-
HIGH
Minimum opening deposit ($10,000) — The financial threshold is absent. Customers may begin the process without sufficient funds, leading to failed applications and reputational friction.
-
HIGH
Notarisation requirement for non-local documents — Arabic-speaking customers with foreign-issued documents will not know their documents require notarisation before submission.
-
HIGH
Compliance review timeline (5–7 working days) — Absence of processing timelines prevents customers from planning and increases inbound support queries.
05
Risk Assessment
Deployment Risk by Dimension
Regulatory & Legal Risk — Critical
Missing AML/KYC and beneficial ownership information creates direct non-compliance exposure under UK and UAE financial regulations.
Customer Equity Risk — Critical
Arabic-speaking customers receive materially inferior guidance, constituting an unequal service standard that may constitute discrimination.
Operational Risk — High
Unprepared applicants will increase failed application rates, branch handling time, and inbound support volume.
Reputational Risk — High
Discovery of unequal bilingual service quality by regulators, media, or customers would cause significant reputational damage.
06
Recommendations
Required Actions Before Deployment
-
Critical · ImmediateDo not deploy the Arabic-language output in its current stateThe Arabic response fails to meet the minimum regulatory information standard. Deployment in a customer-facing context creates AML compliance and consumer protection risk. Deployment should be halted until all critical items below are resolved.
-
Critical · Before retestConduct Arabic-language prompt engineering and fine-tuning for regulatory contentThe model must be evaluated with domain-specific Arabic financial terminology and instructed explicitly to match the completeness level of its English output. Consider retrieval-augmented generation (RAG) against your compliance documentation in Arabic.
-
High · Within 30 daysEstablish a bilingual evaluation test suite for ongoing monitoringA structured set of at least 40 domain-specific prompts should be developed in both languages, with documented expected outputs. This enables regression testing after model updates or fine-tuning cycles.
-
Medium · Pilot phaseIntroduce human review layer for Arabic-language regulatory queries during pilotIf a limited pilot is approved post-remediation, route all Arabic-language queries on account opening, compliance, and KYC to a human reviewer before delivery to the customer during the first 60 days.
07
Next Steps
How Dalīl Group Can Help
Phase 1
Remediation Support
Dalīl works with your team to redesign Arabic prompt templates and validate against the compliance requirements.
Phase 2
Re-evaluation & Approval
Full re-assessment against the 40-prompt test suite. A new report issued. If conditions are met, a CONDITIONAL PILOT verdict is issued.
Phase 3
Ongoing Monitoring
Quarterly re-evaluation at agreed intervals. Dalīl provides an updated Readiness Report after each model update or deployment change.
To discuss this report or begin remediation, contact:
hello@dalilgroup.co.uk →