Market hype on AI and accounting roles overlooks critical issues, says AI specialist

Empathetic AI chief executive and founder Angela Shi says that while there has been considerable hype from AI technology firms about AI agents taking over accounting roles, there are many more factors to consider than just the technology itself.

Microsoft chief executive Mustafa Suleyman recently predicted that most of the tasks accountants, lawyers and other professionals currently undertake will be fully automated by AI within the next 12 to 18 months. This echoes similar predictions from other major AI companies.

Speaking on a recent panel discussion hosted by CA ANZ, Shi said in her opinion, AI agents should not be given full autonomy to perform the role of accountants due to issues around professional judgement, accountability and data privacy laws.

"There's such hype in the market [around] AI agents taking over the world in the next six to twelve months but in terms of the accounting profession, we really need to consider a lot of the factors besides the technology itself such as data privacy and security," Shi said.

"Can we provide the information to the third party or not? Obviously not. How can we guarantee that our clients' data is not going to a third party for training purposes? How are we going to guarantee that the boundary is properly set between the human and AI?"

Shi said there are also major considerations in terms of judgement and accountability.

"What if the AI makes a mistake? Who will be accountable for that? There are lots of issues right now that we have to take into consideration," she said.

Shi said that the fact that AI agents can automatically take action on behalf of accountants without even notifying them does not mean they should be given this level of autonomy.

"In my personal opinion, having been a finance professional in the past decade, we still need to have a certain level of segregation or boundary between what the AI can do and the remaining parts which are supposed to be done by a human," she said.

"The human should be mastering the AI instead of the other way around."

Speaking on the same Panel, CA ANZ chief executive Ainslie van Onselen said that when accounting professionals use AI, they must ensure they apply their critical thinking and business judgement.

Van Onselen noted that the Accounting Professional & Ethical Standards Board (APESB) released a technical alert in October last year, reminding accountants that they are still "responsible for the analysis, the underlying professional judgements, and the outcomes of the professional activity [the provide] to a client or employer".

"Members should be mindful of the extent of use of technology and avoid undue reliance on or undue influence from technology such as AI, which at times can hallucinate and present convincing answers that are not accurate," the alert said.

"Members must have an inquiring mind to critically assess and verify any information used or produced for a professional activity, which includes AI-generated information."

Van Onselen said this means accounting professionals are responsible for what they sign off on, whether they've used AI or not.

"When you use AI you need to understand it. Has it been attested? Is data security in place? Is privacy in place? They're all the questions you have to ask yourself when you sign off on advice, when you sign off on a financial statement or an audit," she said.

"The APES board has made it clear that you're on the hook for that advice, regardless of what technology you have used."