Saturday, March 30, 2013

Survey of more than 1,500 Auditors Concludes that Audit Professionals are Not Maximizing Use of Available Audit Technology



Key findings from the survey include:
·        while audit software tools have been available for almost 2 decades, auditors and audit departments are not making full use of the technology
·         auditors use audit software tools mostly on an ad hoc basis with some repetitive use, and departments do not have a strategy or plan to integrate technology in the audit process
·         the main reason for limited use of audit technology tools is the cost of the software and training and  management resistance to change
Auditors have a number of technology tools and techniques available for each area of the audit process. Software vendors comprised of former technology audit professionals have developed products that can increase productivity and expand the capabilities of audit shops whether they are large or small. Despite this fact we have noticed that surveys performed by leading professional associations, consulting and auditing firms consistently report that while auditors have made gains in utilizing technology there is plenty of room for improvement.
The AuditNet® 2012 State of Technology Use by Auditors was launched in order to determine what tools audit departments are using, how the tools are being used, the level of use and where auditors see themselves on the Audit Utilization of Technology Optimization Scale (AUTOS). AuditNet® has been on the leading edge of the profession when it comes to encouraging auditors and audit departments to integrate the use of technology in their methodology.
Executive Summary
Information technology is now a way of life in the business community. The decreasing cost of hardware and the wide variety of software available has put computing power on the desk of most employees of even the smallest companies. This technology has also made it possible for auditors to undertake procedures that were previously impossible due to time constraints and cost.
From Application of Computer Assisted Audit Techniques using Microcomputers –
CICA 1994
Computers assisted audit tools and techniques (CAATTs) have been available for auditors since the early 1990’s. The 2012 AuditNet® Audit Use of Technology Survey found that despite the recognized added value that technology affords auditors a paradigm shift in maturity is not imminent. We received responses from almost 1,500 auditors and over 2/3 completed all the survey questions. The most widely used category of audit software used by auditors is data analysis software. That being said 59% of those surveyed said they were using data analysis tools on an ad hoc basis (when needed) or not at all. Only 21% were using data analysis tools and techniques on a repetitive basis (used for most audits). While few respondents indicated continuous use (used for every audit or project) There is clear evidence that using data analysis software results in cost savings as demonstrated by research from the Corporate Executive Board’s Audit Director’s Roundtable which reported in 2005 that Internal Audit Internal Audit Should Embrace Data Analytics:
Data analytic procedures are a much more cost-effective way to collect audit evidence. Analytic procedures cost $0.01 compared to $4 for a standard audit of the same evidence.
The survey results showed that other audit software technologies were either being planned or at the ad hoc use level. Repetitive use of audit software by most auditors is far from mature in all of the software categories included in the survey. For a profession that prides itself on embracing technology it was clear from our results that auditors are far from reaching an optimum maturity level.
Professional associations have promoted the benefits of enhanced technology to their members and have produced guidance on integrating technology in the audit process. The IIA implemented Standards requiring auditors to leverage technology stating that:
In exercising due professional care, internal auditors must consider the use of technology-based audit and other data analysis techniques. (Standard 1220.A2)
Professional associations have done significant research and invested significant resources into developing strategies and plans for greater use of technology by auditors however they have not openly disseminated this information to the global audit community. Association members have free access but non-members must purchase the guides. Additionally audit and consulting firms have spent significant time and resources to develop unique and separate approaches and methodologies on technology solutions. If we expect to affect a paradigm shift in audit use of technology we need standards, action plans and guidance coordinated through a task force of professional associations and audit and consulting firms. These “standards” and guidance should be widely disseminated to the global audit community. Social media and networks have facilitated communication to the global audience of auditors and could be effectively used to “raise the bar” on the audit use of technology maturity scale.  
Additional results of our survey showed the following:
1.      Training is primarily on the job as opposed to a formal setting.
2.      Software cost and lack of management support continue as significant reasons for not implementing technology
3.      Technology continues to be seen as the domain of IT auditors rather than all staff
4.       Most audit department have not:
a.      defined a strategy for implementing audit technology,
b.      defined the benefits of implementing audit technology,
c.       defined ways to measure the effectiveness of technology investments, processes, and activities
5.      Size of audit department has no apparent impact on maturity level of use of audit technology
6.      Most audit departments were at the lowest level of maturity on the audit use of technology maturity level (from little or no use to planning and ad hoc) and few departments have matured to repetitive or continuous use of audit technology.
Action Plan for Auditors
1.      Conduct an inventory of technology tools and create a matrix linking tools to activities
2.      Prepare a technology skills inventory for current staff and perform a gap analysis
3.      Perform a gap analysis to determine target areas for improvement and determine financial resources required for the software, training and maintenance
4.      Build a business case for integrating technology in the audit process
5.      Hire auditors with the necessary skills to fill the gap.
6.      Acquire technology for the target areas if not already purchased
7.      Assign a technology champion and influencer to lead the integration and develop a succession plan to ensure technology use continuity
8.      Develop an implementation strategy to integrate audit technology for administrative tasks and in each phase of the audit process
9.      Annually conduct an audit technology assessment and benchmark progress against initial initiative
10.  Develop metrics to demonstrate the impact of technology on the audit process to include outcomes as well as outputs.


AuditNet® 2012 State of Technology Use by Auditors Survey

Computers assisted audit tools and techniques (CAATTs) have been available for auditors since the early 1990’s. The 2012 AuditNet Audit Use of Technology Survey found that despite the recognized added value that technology affords auditors a paradigm shift in maturity is not imminent. We received responses from almost 1,500 auditors and over 2/3 completed all the survey questions. The most widely used category of audit software used by auditors is data analysis software. That being said 59% of those surveyed said they were using data analysis tools on an ad hoc basis (when needed) or not at all. Only 21% were using data analysis tools and techniques on a repetitive basis (used for most audits). While few respondents indicated continuous use (used for every audit or project) There is clear evidence that using data analysis software results in cost savings as demonstrated by research from the Corporate Executive Board’s Audit Director’s Roundtable which reported in 2005 that Internal Audit Internal Audit Should Embrace Data Analytics:
Data analytic procedures are a much more cost-effective way to collect audit evidence. Analytic procedures cost $0.01 compared to $4 for a standard audit of the same evidence.
The survey results showed that other audit software technologies were either being planned or at the ad hoc use level. Repetitive use of audit software by most auditors is far from mature in all of the software categories included in the survey. For a profession that prides itself on embracing technology it was clear from our results that auditors are far from reaching an optimum maturity level.
Professional associations have promoted the benefits of enhanced technology to their members and have produced guidance on integrating technology in the audit process. The IIA implemented Standards requiring auditors to leverage technology stating that:
In exercising due professional care, internal auditors must consider the use of technology-based audit and other data analysis techniques. (Standard 1220.A2)
Professional associations have done significant research and invested significant resources into developing strategies and plans for greater use of technology by auditors however they have not openly disseminated this information to the global audit community. Association members have free access but non-members must purchase the guides. Additionally audit and consulting firms have spent significant time and resources to develop unique and separate approaches and methodologies on technology solutions. If we expect to affect a paradigm shift in audit use of technology we need standards, action plans and guidance coordinated through a task force of professional associations and audit and consulting firms. These “standards” and guidance should be widely disseminated to the global audit community. Social media and networks have facilitated communication to the global audience of auditors and could be effectively used to “raise the bar” on the audit use of technology maturity scale.  
Over two thirds of our responses came from audit departments with less than 10 auditors. However, based on the responses provided there appears to be no variation in how auditors use technology. We have always assumed that the larger the department the greater the utilization of audit technology but based on the results of this survey that is not the case.  
Availability of Audit Technology Tools
Audit technology tools other than data analysis were not available in more than half of the audit departments according to the survey responses. The following chart represents the technology tools currently available by those who responded to the survey. This chart clearly demonstrates that of the nine categories of technology tools those used most and fully deployed are data analysis software (ACL, IDEA etc) followed by electronic working papers such as CCH TeamMate. Categories where little progress has been made include GRC, continuous monitoring and fraud detection/prevention software. The majority of audit departments had not even begun the evaluation process and had no plans to make technology tools available to their staff. Less than 5% were evaluating vendors. If audit technology tools are not available in more than half of audit departments then how reliable are results from other surveys regarding widespread use of computer assisted audit tools and techniques?
Proficiency with Technology
When asked to categorize their technology skill set 28% of the respondents indicated that few of their auditors were proficient with audit technology tools and 31% said some of their auditors were proficient. The IIA standards do not require all internal auditors to possess technology skills so the responses to this question are not surprising. The emphasis by professional standards is that technology skills should be available rather than having proficiency for all staff. This would be a good area to expand this question and correlate staff level to technology skill set. Perhaps the senior staff represents auditors without technology skills as their primary role is administration. Even so we would expect that some of the audit management tools involve technology skills.

Few of our auditors are proficient with our audit technology
27.7%

Some of our auditors are proficient with our audit technology
30.9%

Most of our auditors are proficient with our audit technology
24.0%

All of our auditors are proficient with our audit technology
17.4%

 
Technology Training
Most of the technology training received by both new hires and current audit staff is accomplished by on the job training. Only 7.5% reported formal training by third parties or outside the office. Training is an important factor in audit use of technology. Training must be supplemented by using the technology on a repetitive basis. Using audit technology on an ad hoc basis may result in loss of knowledge acquired from formal or on the job training i.e. use it or lose it.  According to TeamMate’s 2011 Internal Audit Technology Survey
 Despite the importance of the right “Tone at the Top” and technology skill sets, training does more than any other practice to help respondents utilize technology more effectively, according to the 2011 IATS results. In response to an open-ended question receiving dozens of responses, more than 50 survey respondents identified some form of on-the-job or formal training, whether delivered by in-house staff or a third party, as having done the most to help them enhance technology value and effectiveness.
Keep in mind, however, that technical skills fade quickly if they are not used; the “use it or lose it” saying has definite application when it comes to complex technology requiring frequent use in order to master and retain what is learned. For this reason, it is useful to conduct a strategic analysis of your training needs when it comes to technology, a process that includes determining what levels of proficiency are required for various members of your group in order to achieve departmental objectives. You may only need one expert in data mining or data analytics, for example, and this expert, in turn, would be asked to train others in the department in a systematic manner.
Therefore one of the critical steps in moving the bar on the audit use of technology maturity scale is training tied to repetitive or continuous use of the tools.
Inventory of Audit Technology Tools
How can an audit department maximize the use of audit technology tools if they don’t know what tools they have? The following chart shows that according to the survey responses most audit department do not maintain an inventory of audit technology tools. So the question is if audit departments do not inventory their audit technology tools how can they begin to put into place a strategy or plan to use technology? This may be an area for further study as turnover in audit departments would be a key factor in loss of institutional knowledge or a growing inventory of shelf ware (audit software purchased but not deployed). The question of available technology tools should be one of the questions asked by new hires or a new chief audit executive (CAE).
A current and updated inventory of available audit technology tools represents another critical step in the audit use of technology.
Current Use of Technology by Audit Departments
Almost two thirds of those responding said that their audit departments performed an assessment of the current use of technology by internal audit. A key metric to establish whether audit uses technology would be to measure the current state of technology use by audit. By benchmarking and measuring the current state of technology use by audit a CAE would be able to create a plan and then evaluate on a periodic basis (annually) whether the use of technology has matured beyond an ad hoc level. 
Auditor Maturity Ratings on the Use of Audit Technology 
The IIA Global Technology Audit Guide (GTAG) 16 Data Analysis Technologies provides the following group types of data analysis tasks:
AD Hoc
Repetitive
Continuous
Explorative and investigative in nature
Periodic analysis of processes from multiple data sources.
“Always on” — scripted auditing and monitoring of key processes.


Seeking documented conclusions and recommendations.
Seeking to improve the efficiency, consistency, and quality of audits.
Seeking timely notification of trends, patterns and exceptions.
Supporting risk assessment and enabling audit efficiency.
Specific analytic queries — per-formed at a point in time — for the purpose of generating audit report findings.
Managed analytics — created by specialists — and deployed from a centralized, secure environment, accessible to all appropriate staff.
Continual execution of automated audit tests to identify errors, anomalies, patterns and exceptions as they occur.



Using similar group types for all audit technology AuditNet© developed the Audit Utilization of Technology Optimization Scale (AUTOS) to measure the perceived maturity level on the use of audit technology by auditors. We asked respondents to rate their department maturity for the categories of audit software. 
The survey results demonstrated that most auditors are still at the ad hoc or lowest level of maturity for most audit technologies. For most of the audit software categories more than half of the auditors reported that they do not use the technology. For some categories such as governance, risk and compliance, continuous controls monitoring and fraud prevention and detection more than two thirds reported no use. The only exception was data analytics (ACL, IDEA Excel) where almost 38% reported not using the technology. Another 21% reported that they only use data analysis tools on an ad-hoc basis (defined below in GTAG 16). In the category of electronic working papers and audit management software (TeamMate, Pentana and other vendors) close to 50% do not use the technology. Based on subsequent survey questions one of the delimiters to greater use of audit technology is cost of the software and training. There is a significant opportunity for vendors to develop affordable tools to capture the market of those audit functions not currently using audit software technology. Additionally affordable training opportunities for each of the software categories could have a positive impact on audit departments moving the bar on the maturity scale.  
Satisfaction with Implemented Audit Technology and Leveraging the Tools
We asked survey participants to rate their satisfaction in each of the audit technology categories that their department has implemented. The responses demonstrated a relatively low level of satisfactions in most software categories. The only technology tool where more than half the auditors were either moderately or completely satisfied was Microsoft Office (Word, Excel and Access). For the data analytics (ACL, IDEA) and electronic working papers (TeamMate, Pentana and others) categories almost 40% of those surveyed were moderately or completely satisfied. In the data analytics and electronic working paper categories 10% of those responded they were not satisfied with the technology. When looked at in the context other survey responses the level of satisfaction is directly related to the ad hoc use of audit software, the training factor as well as support by senior management. If auditors are not using audit software technology on a repetitive or continuous basis they may not be seeing the positive impact results. If results from the use of audit software technology is not being measured or evaluated then satisfaction may be less than optimal.
We also asked the previous question from the perspective of how satisfied respondents were with the audit functions leveraging of the tools. For this question approximately 20% for each software category were not satisfied with how their departments were leveraging the technology. Again the highest ratings were for audit management software and data analytics. Leveraging the technology is highly dependent on whether the audit department has a plan on how to use the technology and it is clear from other survey questions that ad hoc use can be correlated to less satisfaction. If audit departments established a plan for how to integrate technology into all areas of the audit process we should expect a resulting shift in the satisfaction level by staff and management. The benefits of  leveraging technology to support the internal audit mission would become evident.
Detailed responses for this question are available in the full survey report.
Software and Training Costs are the Primary Reasons for Not Using Audit Technology
We provided a list of 15 reasons cited for not using audit technology and asked survey participants whether they agreed or disagreed. The main reason for not using audit technology is cost according to 57% of the responses. The next major reason for not using the technology was that their department had no funds allocated for training (50%). While software and training costs were the primary factors cited for not using the technology they were followed by audit management not supporting use and organizational roadblocks. Management buy in to how technology can help is a critical factor and a cost benefit analysis could contribute greatly to overcoming cost issues.  Management support is critical as reported in TeamMate’s 2011 Global Technology Survey where they found that tone at the top was the single factor that does the most to enhance technology effectiveness.  Developing a training plan that leverages internal knowledge can also facilitate the process. In some companies organizational roadblocks such as IT (Information Technology) management of the Chief Technology Officer (CTO) impact the use of audit technology. It is therefore important for audit management to make the case for acquiring audit technology and demonstrating how it can benefit the organization. Key stakeholders and management need to be part of the process.   Detailed responses for this question are available in the full survey report.
Audit Department Technology Plans and Strategies
The following chart shows responses to the question does your audit department have a plan or strategy to address key areas.  More than half of those surveyed reported that they did not have a plan or strategy for these areas. 


Auditors Rank Departments at Informal or Ad Hoc Use of Technology
Too few auditors and audit organizations have invested much thought and resources into computer-based tools and techniques, let alone information technology.
David Coderre from Internal Audit: Efficiency through Automation 2009
AuditNet© developed an Audit Use of Technology Optimization Scale and asked auditors where they would rank their department. More than one third (36%) of survey respondents ranked their audit department at the informal or ad hoc maturity level for audit use of technology. Another 18% ranked their department at the needs assessment level which means that they are not using any audit technology.
The responses from those surveyed would contradict many other reports from the profession, vendors and consultants that indicate that most auditors are using technology on a repetitive or continuous basis. In order for auditors to achieve the efficiencies afforded by technology they need to integrate and incorporate tools as part of the audit process. To move the bar on the Technology scale auditors need to be using available tools for all audits rather than on an ad hoc basis. As organizations have embraced technology in all of their systems and operations, auditors need to embrace technology to approach how they do their work and meet the objectives of their audits. This requires a cultural change that must come from the top down as well as the bottom up. The technology option must be considered from the planning phase of the audit right through to the reporting and follow up phase. Staff auditors should be encouraged to suggest how they can use technology to improve and facilitate the audit. Technology can be applied for risk assessment, planning the annual audit schedule and projects, fieldwork, reporting and follow up. 
Factors that will Influence a Shift in Technology Use
We asked what factors have the greatest influence on making a paradigm shift towards technology maturity.  The highest ranked areas were as follows:
1.      Financial resources – greater investment in audit technology
2.      Leadership – greater commitment  by the CAE to embrace technology
3.      Planning – having a clearly defined plan
4.      Training – ensuring that all staff receive training
5.      Human resources – hiring the right auditors with technology experience
The lowest ranked areas included compensation incentives, vendor suite of tools, and organizational infrastructure changes.
The issues identified coincide with surveys by other organizations that emphasize cost as a major factor. This is perhaps an area where organizations who do not have the financial resources can start their technology plan using available tools such as Microsoft Office and add Excel add-ins for data analytics. However, perhaps the greatest impact can be made by organizational leadership combined with planning, where the CAE and management embrace technology as an enabler to benefit the organization.
The final survey question was open ended and asked auditors to provide the one factor above all others that they saw as impacting a paradigm shift in auditors integrating audit technology in their audit function. More than 30% of the responses centered on the cost of either the software, training or overall technology and in many cases whether the benefit justified those costs. More than 20% indicated that management (CAE) support was the one most important factor that would contribute to a resulting shift.  Following are examples of some of the thoughtful responses from auditors:
·         Individuals are not willing to deal with change.  A lot of people are not willing to learn something new because they think they will not be able to master it.
·         Demonstrating value.  Once auditors being to see value in audit effectiveness, audit efficiency, results, management responsiveness and/or board responsiveness, they get the bug.  As data drives the entire business more and more, and more and more controls are automated, we will have no choice.
·         The more auditors that have used technology and are familiar with the benefits it brings, the more push there will be from within audit departments to adapt technology.
·         Biggest factor is the value of the technology.  The technology would have to significantly reduce current workload. Currently our audit staff is pressed for time on all audits.  Which means we don't have a lot of time for training on new software, but if we found a solution that could be integrated with our current financial systems (if required), and it was cost effective, and allowed us to reduce much of the current workload for conducting non-essential audit tasks, I could see a very good possibility of acquiring and using such audit technology.
·         Clearly demonstrated benefit / differentiation of using technology vs not using technology in audits.  What would help me would be hearing about real world examples of how things went wrong (got missed) at organizations who were not using technology in their audit function and how that cost the organization.  While difficult / unfair to necessarily pin such events squarely on IA, there's enough press re: "Where was Internal Audit?" for technology providers to be able to provide examples of how their product(s) could have helped.
·         In my opinion, the one factor impacting a paradigm shift is change.  For some experiences auditors, the technology is so very new to them that they will resist as long as possible.  For new auditors, without the skills with audit technology, they won't get the job to learn the technology.
·         Being open to change and willing to take the time to learn how new technology can be used.  They need to acknowledge that while using a new technology will seem like more work and that it takes longer in the short term, use and familiarity will increase productivity and the ability to think to new ways to expand effectiveness.
Clearly auditors gave great thought time and effort into responding to this survey and it represents insight into where auditors see themselves and their organizations in relating to use of audit technology. The complete results of this survey are available from AuditNet© and are provided as a service to the profession. Based on the response and the information gleaned from the survey we plan on repeating this to determine the trends with respect to audit use of technology.

For the summary survey results click here

No comments:

Post a Comment