TLDR;

Technology is entering critical operations faster than organisations can fully understand its dependencies.Effective oversight begins with visibility across systems, data, providers and decision processes.
Controls should reflect the potential consequence of each use case. Customer facing, financial and regulated actions require stronger approval and monitoring than internal productivity tasks.
Human confirmation remains essential when AI moves from recommendation to execution. Institutions must define who approves, interrupts and remains accountable for critical actions.
Operational resilience depends on tested intervention and recovery arrangements. Institutions need to detect service degradation, activate alternatives and maintain critical services when technology behaves unexpectedly.

Technology is entering critical operations faster than organisations can fully understand its dependencies. Almost half of Malaysian respondents, 47.3%, identified cybersecurity and compliance as a priority in AIBP ASEAN Market Overview 2025/26.

At the recent 53rd AIBP Conference & Exhibition Malaysia, NACSA and enterprise leaders from Malaysia’s financial and technology sectors shared perspectives on how organisations can retain oversight as emerging technologies become embedded within critical operations.

Artificial intelligence is gaining access to enterprise data, customer journeys and operational processes. Quantum computing could eventually make some encryption methods used today ineffective, affecting the systems that protect transactions, identities and sensitive information.

Both developments lead to the same priority: organisations must preserve visibility, accountability and the ability to intervene as technology scales.

Visibility comes before control

Oversight begins with knowing what technology touches.

For AI, this means identifying which models are being used, what information they process, which systems they access and whether they can recommend or execute an action. For post quantum readiness, it means locating cryptography across applications, infrastructure, digital services and external providers.

These dependencies are often distributed across software components, cloud environments and vendor platforms. An organisation may understand the primary system while having less visibility into the cryptographic functions or AI services embedded beneath it.

NACSA’s national post quantum approach therefore begins with a cryptographic inventory. Organisations identify where cryptography is used, which assets depend on it and how critical those assets are to operations.

“Before organisations can migrate, they must first know what they have, where it is and how critical it is.”

- Ts. Dr. Azlina Ab Aziz, Principal Assistant Director, National Cyber Security Agency

The principle applies equally to AI governance and operational resilience. An institution cannot apply proportionate controls or prepare alternative arrangements until it understands the systems, data and providers supporting a critical service.

Oversight follows consequence

The appropriate level of control depends on what is at stake.

Aishah Farha Mohd Raih, Chief Information Security Officer at Permodalan Nasional Berhad, outlined essential controls including clear ownership, data classification, auditability, monitoring and protection across the different layers of the AI environment.

“Making sure there is clear ownership of those AI models. Who owns those AIs?”
- Aishah Farha Mohd Raih, Chief Information Security Officer, Permodalan Nasional Berhad

Ownership establishes responsibility for how a system is used and how the organisation responds when its behaviour differs from expectations. Data classification determines which information the model may process. Monitoring and audit records support investigation when service quality or system behaviour deteriorates.

An internal assistant that summarises documents carries a different exposure from a system processing customer information, moving funds or influencing a regulated decision.

Institutions should consider whether the action is reversible, how many customers may be affected and whether a person can intervene before execution. The greater the consequence, the stronger the approval, escalation and continuity arrangements must be.

Assistance and authority need different boundaries

Ryt Bank illustrates how this distinction can work in practice.

Its customer facing AI can interpret a request and prepare an action such as a bank transfer. The customer reviews and approves the instruction within the application before the transaction proceeds.

“It translates into an action for the customers to review and approve in the app before it is being executed.”
- Ser Yoong Goh, Chief Information Security Officer, Ryt Bank

The process is supported by logs, traceability, data classification and monitoring of the sources available to the AI. Internal use cases remain outside production while the bank develops confidence in the controls surrounding them.

Razak Idris, Head of Innovation and Excellence Department at Bank Simpanan Nasional, offered a similar dividing line. AI can analyse information, explain potential impacts and recommend an option, while a person retains responsibility for confirming critical actions.

For financial institutions, this creates a practical principle: AI may assist with judgment, while authority over high impact execution remains clearly assigned.

Institutions must also determine who can stop an action, escalate an incident or activate an alternative process when the primary arrangement becomes unavailable.

Autonomous systems require technical boundaries

Human approval provides one layer of oversight. More autonomous AI agents also require controls within the technology itself.

Gary Liu, Chief Executive Officer at Terminal 3, distinguished AI agents from conventional automation and chatbots. Agents may access internal systems, use credentials, interact with sensitive information and complete tasks through behaviour that varies according to context.

“We need to protect against agents taking deterministic instructions we have given them and turning them into probabilistic behaviour.”
- Gary Liu, Chief Executive Officer, Terminal 3

As agents gain access and autonomy, organisations need technical boundaries that restrict their permitted actions, protect sensitive information and preserve a record of their activity.

Alex Chi, Chief Information and Digital Officer at SP Setia, added the human dimension. User awareness remains a final safeguard because employees still decide what information to share and when to trust an AI generated output.

Effective oversight therefore combines technical controls, accountable ownership and informed human judgment. In critical banking environments, these controls must extend across applications, cloud platforms, networks, data and external providers.

Resilience must be demonstrated

Post quantum migration and AI adoption both require phased testing, clear accountability and the ability to intervene when systems behave unexpectedly.

Institutions should know how quickly service degradation can be detected, who has authority to activate alternative arrangements and how customer, banking and payment services will continue during disruption.

Enterprises will be better positioned to scale emerging technologies when they can still explain, interrupt and recover from them.

We will explore these priorities further at the closed door roundtable, When Critical Systems Fail: Keeping Critical Banking Services Running Under BNM’s Revised RMiT. Senior leaders across technology resilience, payments, cybersecurity and operations will discuss service continuity, escalation, dependencies and failover readiness.

Register your interest here.

Oversight Before Technology Outpaces Control