• Author / Uploaded
• Dian Abiyoga

Citation preview

The research register for this journal is available at http://www.emeraldinsight.com/researchregisters

IJQRM 19,1

The current issue and full text archive of this journal is available at http://www.emeraldinsight.com/0265-671X.htm

Self-audit of process performance Stanislav Karapetrovic

24 Received January 2001 Revised June 2001

Department of Mechanical Engineering, University of Alberta, Edmonton, Alberta, Canada, and

Walter Willborn

Faculty of Management, University of Manitoba, Winnipeg, Manitoba, Canada Keywords Audit, Self-assessment, Performance measurement, Quality management Abstract Quality audit, as a methodology for evaluating system, product and/or process performance against established requirements, has experienced substantial growth in worldwide use in recent years. This is largely due to the steady increase in ISO 9000 registrations, which topped 350,000 in the year 2000. Based on the fundamental principles of independence, objectivity and professionalism, the audit is an irreplaceable tool when confirmation of compliance with standards is sought. However, it commonly fails in enabling continuous improvement and spanning the differing aspects of business performance beyond conventional ``quality assurance''. This paper argues for removing one of the principles of traditional auditing, namely independence, to empower the process owner to conduct periodic self-evaluations of process performance. Such ``self-audits'' would be less formal than quality audits, and, much like the better-known self-assessments against business excellence models, aimed at continuous quality improvement. The concept, principles and practices of a self-audit are focused on.

International Journal of Quality & Reliability Management, Vol. 19 No. 1, 2002, pp. 24-45. # MCB UP Limited, 0265-671X DOI 10.1108/02656710210413435

Introduction In our information-driven society, the planning, controlling, and improving of quality becomes more complex and demanding. Modern technology at the workplace, new global and local competition, strong pressures for value-added processes and performances, and integrated management decision-making are main factors for these changes. The meaning of ``quality'' has expanded beyond ``customer satisfaction with products and services'' to the ``creation of worth for all stakeholders''. Overall business excellence is replacing the narrow objective of meeting customer specifications; the performance of the whole system, and not just its outputs, is focused on; and an array of issues, including environment, occupational health and safety, social responsibility, corporate finances, and human resources, must be incorporated into the management of ``quality''. The new ISO 9000: 2000 standards for quality management systems reflect some of these advancements and higher demands. For example, instead of the sole emphasis being on the customer needs and expectations, all stakeholders are now concerned, including customers, suppliers, managers, employees, local community and society, in general. Such developments continue to challenge quality practitioners to ``master additional, modern, total quality management approaches and integrate them into management methods'' (Andersen, 2000). In the words of Chong (2000), only those professionals ``who understand that quality is derived from effectively

managing systems will (be able to) provide leadership in the new millennium''. In this paper, we consider the impact that the expanding role of quality has on the quality audit, specifically addressing the possibility to use a self-audit for control and improvement of process performance. Much like all quality professionals, quality auditors will have to adapt to new conditions and demands with higher qualifications and competence. The new ISO 19011 standard for auditing of quality and environmental management systems will stipulate such requirements. However, can the effectiveness and efficiency of the quality audit be substantially improved by simply imposing a standard and expanding it into environmental management? Performance of auditors has come under considerable criticism with regard to the actual value-added for clients and business in general (for example, see Beeler, 1999 and Stamatis, 2000). Self-assessments against business excellence models are widely considered to be more advantageous in that respect (van der Wiele et al., 2000). On the other hand, the inability to continuously ensure product quality still plagues the image of the ISO 9000 standards, and consequently, quality audits. In a highly publicized recent case of sport-utility vehicle tire recall, a quality system registrar has been apparently implicated by the tire manufacturer's top management for the failure to identify the problem (Zuckerman, 2000). Even if such accusations are dismissed as ``mud-slinging'' and ``blame-shifting'', and even if they are explained by the misunderstanding of the main audit purpose (that is to verify compliance with the appropriate standard, and not to inspect product quality), the concerns about the audit effectiveness and its shortcomings in fostering continuous improvement will still loom. Referring to the tire recall incident, and supporting his doubts that a quality audit could have prevented it, Arter (2000) states that ``auditors don't go around looking for field failures, (and) often don't even have access to the data''. This is true, but the question is why? Is it perhaps because they are not empowered to do so, since they are an entity independent of the function being audited? Would it have helped if the process owners, that is the auditees, had performed the audit themselves? In an audit, the auditor and the auditee must cooperate in order to arrive at a correct and reliable conclusion. The auditee is normally more familiar with the actual process than the auditor. The auditor, on the other hand, is more knowledgeable about the audit criteria, such as a quality management system standard. Therefore, each party contributes to the expected value-added component of an audit: the auditor does so by providing an independent and objective assessment of the process strengths and weaknesses, and the auditee by knowing how best to capitalize on the strengths and to eliminate the weaknesses, in other words to improve the process in the most effective and efficient way. However, in order to alleviate the problem of the external imposition of judgment and the consequent lack of auditee motivation to follow-up on it, innovative and strengthened approaches to auditing are required. One such approach may involve the empowerment of the auditee to conduct an evaluation of his/her own performance, effectively conducting a

Self-audit of process performance 25

IJQRM 19,1

26

self-audit. Tiemeyer (1997), for example, suggests a method of individual assessment in order to ``encourage initiative and a sense of co-responsibility in the achievement of total quality'', in which ``the auditor takes on the role of staff trainer and controller of the self-evaluation process''. Goldstein (1983) presents a ``two-tiered'' combination of a self-audit and an external audit to alleviate the problem of ``relying on outsiders to identify non-conformances'', while Pasco and Brown (1989) illustrate a distributed self-audit system at corporate, plant and station levels. This paper argues for the ``owner'' of a process to conduct a periodic audit of the process performance. As we will show, a ``self-audit'' is not the traditional product quality inspection, ``self-inspection'', nor just another form of internal auditing. It is not designed to add a new undesirable form of process verification. On the contrary, a self-audit, if properly designed and implemented, should actually reduce the current inspections and formal internal audits. The following sections discuss this concept in detail. We begin by analyzing the definitions, reasons for introduction and applications of a selfaudit, as well as its relationships with other evaluation methodologies, followed by an outline of the fundamental principles for self-audits. Subsequently, the process of conducting a self-audit is addressed, and the qualifications of participants are illustrated. A hypothetical example of performing a self-audit is provided, and the paper is concluded by a discussion of future implications and directions for further research. Concept What is a ``self-audit''? Basically, evaluation of one's work is as old as work itself. By virtue of being a goal-oriented activity, actual work and its achieved results (also called ``performance'') are always compared with the planned objectives. A typical evaluation includes the measurement of effectiveness (``extent to which planned activities are realized and planned results achieved'' (ISO 9000, 2000)) and efficiency [``relationship between the result achieved and the resources used'' (ISO 9000, 2000)] of the work process, and subsequent comparison of process performance with the expected goals. When such goals are presented in the form of a documented standard, policy, model or procedure, the evaluation is referred to as ``audit'' or ``assessment'' against ``criteria'' (for example, the ISO 9001 standard and quality awards). The process owner, in other words, the person or unit performing the work itself, undertakes self-evaluations. These evaluations may be done using formal, standardized procedures (internal audit) or informal guidelines (self-assessment and benchmarking). They can also be conducted on an individual basis, or may involve a whole department or organization. For example, every conscientious worker checks his/her work (``Do I continuously achieve what is expected in the proper way? Can I make improvements? What can I learn from the best people doing the same job?''), much like entire organizations evaluate their business performance against standards, excellence models and best-in-class companies. In today's

competitive work environment with higher demands and pressures, linking individual with organizational evaluation is of paramount importance. We believe that the ``self-audit'' concept may provide a way to meet these demands. Although it does not represent a completely new idea or methodology (for example, see Goldstein (1983), Pasco and Brown (1989), and Arter (1989) for references in the 1980s, as well as Bishara and Wyrick (1994) and Tamimi et al. (1995) in the 1990s), a ``self-audit'' is innovative in the manner, objective and scope of its application. It combines the features of the traditional internal quality audit, self-assessment and benchmarking. The main notion is that the person or unit in charge of the process (``owner'') conducts the audit, at the process location. Other people or organizational units directly involved with the process may participate under the leadership of the owner. The primary objective of the self-audit is to evaluate and improve performance, by continuously examining both the performance enablers and achieved results. Business performance enablers can be grouped into three categories: goals (including strategy, policy and objectives), resources (people, material, information and infrastructure) and processes (including leadership and realization of the outputs). These enablers are assessed for their suitability to achieve set performance levels (effectiveness), and the ability to achieve them with minimum effort (efficiency). The results, which may include the ones related to employees, the customer, society and key financial and non-financial outputs (EFQM, 1999a), are measured and compared with target levels. Naturally, performance levels are also periodically examined for purpose and feasibility. Because self-audit includes the measurement and comparison of actual versus desired levels of performance, it may be represented as a system with a negative feedback loop (Figure 1). The self-audit can be conducted at different hierarchical levels (Pasco and Brown, 1989). At the bottom level, an individual worker compares his/her performance with set standards (e.g. a job description), identified strengths, weaknesses, opportunities and threats (SWOT), and the work of best-in-class people. At the top level, whole organizations, which use ISO 9000 standards, quality awards and benchmarks for reference, perform self-audits of their business performance. Such audits must be coordinated in both vertical directions (Figure 2), so that individual self-audit results are utilized at the unit level, as well as in organizational audits. Conversely, audit criteria must be communicated throughout the organization and interlinked, so that each person is aware of how his/her individual audit criteria correspond to departmental or company level ones. Traditional quality audits are never ``self'', but rather independent of ``self'', because they are, by definition, performed by somebody who must be ``unbiased and independent from the function or subject matter being audited'' (ANSI/ASQC, 1986). However, despite the oxymoron-sounding name, a selfaudit may be defined as a system for obtaining and verifying audit evidence, objectively examining the evidence against audit criteria, and incorporating the audit findings into business planning, for the purpose of continuous

Self-audit of process performance 27

IJQRM 19,1

28

Figure 1. Self-audit concept

improvement. We discuss some of the reasons for the introduction of self-audits into practice. Why do we need a self-audit? While quality system audits, self-assessments and benchmarking are all used to evaluate performance against some set of criteria (ISO 9000 standards, business excellence models (e.g. Malcolm Baldrige National and European Quality Awards) and state-of-the-art approaches, respectively), each methodology has its own advantages and shortfalls. For example, a selfassessment by far outperforms an audit in terms of identifying strengths and opportunities for continuous improvement, prevention of problems and incorporation of assessment results into the strategic and operational business planning. The traditional quality audit is most often externally required, focuses on the quality assurance function within an organization only, and

Self-audit of process performance 29 t

Figure 2. Coordination among different levels of self-audit

relies on competent quality professionals for judgments on compliance with audit criteria. Conversely, self-assessments are cross-functional, less formal, and almost always intrinsically motivated (van der Wiele et al., 2000). However, the audit is still superior with respect to objectivity of the evaluation process, reliability and consistency of results, as well as the identification of systematic failures. It has an added benefit of providing a fresh, independent, and unbiased outlook of the audited process. On the other hand, while benchmarking provides a good understanding of best practices, facilitates factbased decision-making and clearly illustrates weaknesses (Dale, 1999), it essentially amounts to copying somebody else's successful processes to reach targets of questionable trustworthiness and permeating the loss of potential for innovation (Besterfield et al., 1999). Therefore, an integrative approach is required for a solid, systematic, fact-based evaluation of process performance, aimed at continuous quality improvement. To enhance intrinsic motivation and the depth of the required organizational change, which, according to van der Wiele et al. (2000), are the two most important variables for the sustainable success of a management ``fad'' (e.g. ISO 9000 and BEM), such an integrative approach must inevitably be based on selfevaluation. Any assessment that is externally-driven or performed by an outside party is likely to induce fear, perceptions of uncertainty and coercion, as well as to produce only a short-lived and ineffective effort. Witness what happened to ISO 9000 quality systems in some companies that were forced by their customers to implement the standards: the system worked only on paper, tucked away out of anybody's sight, until the time for the next registrar's visit

IJQRM 19,1

30

(Karapetrovic, 1999). In the words of the cartoon character Dilbert, the ISO documentation binder is ``treated like a dead raccoon'' (Adams, 1997). In other words, since total quality management requires a worker to be motivated and empowered in the workplace, facilitating an assessment of performance by the worker him/herself is most meaningful and effective. Owing to the need for objectivity and consistency of judgments, this method should be augmented by a periodic independent evaluation, which is, in effect, an audit (Goldstein, 1983; Pasco and Brown, 1989). In difference to traditional auditing and selfassessments, which are both relatively formal and essentially off-line methodologies, a self-audit would provide immediate or on-line feedback on performance. Consequently, an approach that combines a quality audit with self-assessments using benchmarked criteria, and capitalizes on the strengths of the constituting methodologies, is necessary. What are the relationships with other evaluation methods? Since a self-audit is a system evaluation tool, it shares some commonalities, but also differs in certain respects from other assessment methodologies. While Table I presents several of such methods and brief comments on their relationships with the self-audit, we will proceed with a discussion on performance measurement, benchmarking, self-assessment, internal quality audit, management review and self-inspection, in particular. Performance measurement is a methodology that complements the selfaudit, since it focuses on identifying metrics related to business results (e.g. quality, time, flexibility and cost (Neely et al., 1995)), and on evaluating the actual performance with reference to developed metrics. An organization implementing a self-audit should use a performance measurement system, defined in Neely et al. (1995) as a ``set of metrics used to quantify the effectiveness and efficiency of actions'', to develop indicators of performance that are measured and analyzed in the audit process. Desired levels can be obtained from the benchmarking process. For instance, a machinist can measure the operation time and the rework rate and compare these indicators with the levels achieved by the best machinist in the company. Then he/she will examine the ways in which these rates can be improved, and implement corrective and preventive actions to attain set levels. Similarly, the machinist's company could decide to measure product cycle times, prototype introduction times, as well as the defect rates, and compare them with a similar best-in-class operation. In this sense, self-audits apply performance measurement and benchmarking to quantify and qualify results, rather than enablers of performance. Benchmarking may be used to identify the latter, as well. On the other hand, the techniques of self-assessment and internal quality audit, applied for evaluation against the requirements of BEM and ISO 9000 standards, respectively, may be used to focus on the existence, suitability and adequacy of performance enablers. Table II illustrates some differences with respect to internal, external and self-audits, as well as self-assessments. For example, a machinist ensures that adequate ISO 9001 quality system

Term

Definition

Source

Comment

Inspection

Conformity evaluation by ISO 9000 observation and judgment (2000) accompanied as appropriate by measurement, testing or gauging

Refers to the evaluation of whether a product or service conforms to specified requirements. Product audit is a broader term, since it involves an evaluation of decisions related to product inspections (Juran and Gryna, 1993), i.e. whether specifications are suitable and whether inspection methods are capable

Selfinspection

Inspection of the work by ISO 8402 the performer of that (1994) work, according to specified rules

Involves evaluation of product or service conformity only. It is a part of the self-audit, but only referring to the output

Assessment

Judgment or decision on the quality, importance, amount or value of something

Cambridge When assessment is done with (2000) reference to a management system standard, a system audit is performed. An audit is a special case of assessment

Selfassessment

Comprehensive, systematic and regular review of an organization's activities and results referenced against a business excellence model (BEM)

EFQM (1999)

Quality evaluation

Systematic examination of ISO 8402 the extent to which an (1994) entity is capable of fulfilling specified requirements

Entity here refers to a product, service, process, project, contract or a system. A self-audit may include the evaluation of compliance of all of these entities, but is broader because it involves the examination of effectiveness and efficiency

Quality surveillance

Continual monitoring and ISO 8402 verification of the status (1994) of an entity and analysis of records to ensure that specified requirements are being fulfilled

Surveillance involves continual evaluation of product, service or system conformance to requirements. A self-audit is a continual evaluation of activities and results aimed at their improvement

Audit

Systematic, independent ISO 9000 and documented process (2000) for obtaining audit evidence and evaluating it objectively to determine which agreed criteria are fulfilled

While the audit defined in this manner is restricted to the evaluation of compliance with audit criteria, a self-audit is not. Also, a self-audit may not always obey the independence principle

Self-audit of process performance 31

This definition refers to a selfassessment where reference criteria for assessment are contained in a BEM, and the evaluation is performed by an organization itself. A self-audit would incorporate this kind of a self-assessment

(continued)

Table I. Evaluation terms and methodologies related to the self-audit

IJQRM 19,1

32

Table I.

Term

Definition

Source

Verification

Confirmation, through the ISO 9000 provision of objective (2000) evidence, that specified requirements have been fulfilled

Reflects the confirmation of product conformance to requirements. A selfaudit verifies that requirements have been met

Validation

Confirmation, through the ISO 9000 (2000) provision of objective evidence, that the requirements for a specific intended use or application have been fulfilled

Reflects a product's fitness for use and application, or a system's suitability to achieve objectives. A self-audit includes these

Review

Activity undertaken to determine the suitability, adequacy and effectiveness of the subject matter to achieve established objectives

Self-audit focuses not only on the suitability, adequacy and effectiveness, but also on efficiency. The results can be used as an input into management review

ISO 9000 (2000)

Comment

procedures are documented and followed, and the existing instructions are suitable for achieving performance goals. Otherwise, a procedural change is requested. A self-audit is less formal and independent than an internal audit that complies with the applicable audit standards (e.g. ISO 10011). As both are evaluations of processes, they naturally have much in common and need coordination for proper results. Under certain conditions, such as in a small business and/or relatively simple processes, internal and self-audits could be identical. In that case, the auditor and the auditee are the same person. Naturally, a periodic assessment by an external auditor should be conducted to verify self-audit results. These results should also be used as an input into the management review process, which is required by the ISO 9000 standards. In fact, incorporating the outcomes of the audit process in management review and business planning is probably the most important feature of selfassessments that should render the self-audit more useful for continuous improvement. Finally, when self-audits are performed at the individual level (Figure 2), they may be confused with self-inspections. Juran and Gryna (1993) explain the difference between a self-inspection and a product audit. While the former is related to the decision on whether the product conforms to specifications or not, and is performed by the operator him/herself, the latter is conducted by a person other than the operator and evaluates the appropriateness of inspection decisions and criteria. In other words, self-inspections are product quality evaluations, while self-audits are process performance evaluations. Also, while inspectors cannot make a decision on the corrective action resulted from the product nonconformance (Juran and Gryna, 1993), self-audits would allow

Criteria

External audit

Internal audit

Self-audit

Purpose

Quality system registration

Control of Identification of Attaining business product, process, process strengths excellence through or system and weaknesses strengths and improvement opportunities

Scope

Standard requirements (e.g. ISO 9001)

By directive of Specified area of management (e.g. operational plant/unit) responsibility

By directive of management and award guidelines

Applicability

Registration procedure

Mainly in large multi-unit companies

In companies of any size or industry

When goal is business excellence

Client

Applicant, customer or registrar

Management

Management and Management and process owner stakeholders

Auditor

Certified professional

Appointed and Authorized and trained employee trained process owner

Trained managers (internal) or team of experts (external)

Auditee

Company employees

Supervisory management

Management and supervisors

Auditing process criteria

Compliance with Documented and Appropriate published audit authorized evaluation standards (e.g. procedures methods (e.g. ISO 10011) checklists)

Documented, comprehensive evaluation and analysis

Reporting and follow-up

Formal report Report to and and follow-up decision by by management management

Report to management (internal) and award committee (external)

Process owner, person in-charge

Submittal of suggested improvement(s)

Self-assessment

qualified ``process owners'' to make such decisions, subject to periodic external reviews. It is important to note that the advantages of self-inspection, reported by Juran and Gryna (1993), including the psychological benefit of not having the outsider judge one's work and job enlargement, would be translated to the self-audit. When and where is self-audit applicable? Every process involves at least one person who should be able to assess the performance of that process. It does not matter if it is preparing soup in a hotel kitchen or producing a circuit board for a computer. Therefore, much like the traditional quality audit, a self-audit is applicable to any organization, regardless of its size, position in the market, or the type of product or service delivered. The main idea behind the self-audit is to broaden its scope beyond quality assurance to other functions and areas in an organization, involving not

Self-audit of process performance 33

Table II. Examples of differences among several types of audits

IJQRM 19,1

34

only quality professionals, but also basically all employees and concerned parties. The introduction of self-assessment methodologies should facilitate such an increase in the scope of implementation. On the other hand, self-audits for environmental, occupational health and safety, maintenance, ergonomic and accounting applications may not be far-fetched, either. For further discussion on the fundamentals of cross-discipline auditing, an interested reader may refer to Karapetrovic and Willborn (2000). Principles Any transformation from traditional quality auditing to the proposed self-audit framework requires a significant shift in responsibility and authority for process evaluation from the auditor to the process owner. However, an abrupt allocation of such responsibilities to the process owner, especially in organizations which have just initiated systematic performance improvement efforts (e.g. see Dale, 1999), may cause more harm than good. Lack of knowledge and understanding of the audit criteria, subjective evaluations of one's own work, and deficiencies in corrective and preventive action follow-up are but a few examples that may be expected as a consequence of a sudden and unprepared empowerment of the auditee to conduct self-evaluations. Therefore, such a transformation can begin only after certain prerequisites are met. What are the prerequisites for performing a self-audit? According to Forsha (1992), the basic conditions for continuous improvement are: ``identifying a problem, motivating yourself to do something about it, and taking action''. While the introduction of the traditional auditing methodology facilitates the attainment of the first stated condition, it is expected that selfevaluation will increase the motivation to act on the findings, and thus foster the achievement of the second and third condition. However, improvement can only occur if the process is stable and capable of meeting set requirements first, in other words, if it is controlled. Since in a self-audit, the process owner conducts the evaluation of process performance, the most important prerequisite becomes self-control, both psychological and technical. Juran and Gryna (1993), and later Shirley and Gryna (1998), discuss this concept in detail, and emphasize that, in order to be in the state of self-control, process owners must know: . what they are supposed to do; . what their actual performance has been; and . how to regulate their performance. These conditions point out the need for proper training of self-auditors, allocation of sufficient time to perform the audit, preparation of audit aids such as checklists and scoring sheets, ensuring management support, and adequate follow-up with respect to audit findings. Enclosed is a more detailed list of the requirements for a successful self-audit:

.

.

Self-interest of the process owner must be considered of fundamental importance. Management must not only be supportive of the self-audit, but also actively involved, including fulfilling the following functions: ± initiation and approval of the self-audit plan; ± provision of adequate resources; ± acceptance of the self-audit report; ± incorporating the results into business planning.

.

.

Short of drafting a formal self-audit procedure, basic guidelines for the evaluation process should be available to the auditors, including auditor checklists. Roberts (1993) discusses the design of such personal checklists to let ``employees keep track of desirable personal jobperformance traits and failures (defects) to achieve goals in certain categories''. Each audit must have a clear objective and scope, including having the knowledge of: ± process boundaries; ± suppliers and customers.

. .

. .

.

.

Process ownership must be formally established. Major stakeholders in the process, including workers, suppliers, as well as internal and external customers, must be informed, and invited to actively participate under the auditor's leadership. For example, it is customary for the quality award and registration bodies to provide a set of auditor checklists to their client's audit (Willborn and Cheng, 1994). A written report on the self-audit results should be prepared. The auditor should be actively involved in any internal or external audit of the process, and should receive a copy of the audit report. No punitive actions of any kind, nor actions appearing as such, should be taken in conjunction with and as a result of a self-audit. Process improvements resulting from a self-audit, should be recognized by superior management, possibly in relation with an achievement award.

Evidently, quality-driven organizations with well-established internal audit programs will be able to quickly adapt their systems to these requirements and move toward self-evaluation. However, while a traditional audit is conducted in a very formal and mechanistic manner, self-audits are declarative in nature, performed not in accordance with set procedures, but rather guided by several fundamental principles. In the words of Pyzdek (1999), we should ``replace

Self-audit of process performance 35

IJQRM 19,1

36

management control systems with a system of general rules that lead inexorably to self-control''. What are the basic self-audit principles? Much like a conventional quality audit, according to Karapetrovic and Willborn (2001), a self-audit can be conceptualized as a system of interrelated activities that are aimed at evaluation of process performance to enable continuous improvement. In a sequence of activities that resembles a plan-docheck-act cycle, the objectives for conducting a self-audit are determined, the audit is planned and adequate resources are allocated and deployed, followed by the execution and incorporation of results into business planning and follow-up actions (Figure 3). In contrast to the internal quality audit, the client, auditee and the auditor are the same person, organizational unit or the organizational itself. Figure 3 presents the fundamental rules for conducting a self-audit, arranged along the above-mentioned sequence of audit activities. It is based on the model of the principles for generic auditing, found in Karapetrovic and Willborn (2001). Most of these rules are declarative and selfexplanatory, from the clear understanding of the purpose, objective and scope of the audit, through the existence of audit criteria and evidence (otherwise, an audit would be impossible to undertake), to the verification of the collected information and inclusion of results in future planning. The main purpose of such a flexible set of principles is to allow organizations to design and implement self-audits in whatever form they deem most appropriate, as long as some basic rules that ensure audit integrity are followed.

N

Figure 3. General principles of self-auditing

Methodology In the following sections, general steps for the development of a self-audit program, and required qualifications of involved parties are discussed. How can we introduce a self-audit program? Simplicity and informality are essential for the process owner, who is now acting as an auditor, to readily accept the additional evaluation task. Nevertheless, in addition to the above-mentioned general principles, an elementary guideline for the execution of an audit may be suitable. The complexity of such a guideline will depend on the relative complexity of the auditing task at hand, understanding of the audit criteria (standards and excellence models), as well as the intricacies of the evaluated process itself. The following seven basic steps, similar to Juran and Gryna's (1993) suggestions for the gradual introduction of self-inspections, are advisable: (1) The purpose and benefit of a self-audit are studied and explained to selected and/or concerned personnel. The initiative may come from the process owner, executive management, or outsiders such as a consultant or auditor. All questions and concerns should be addressed. If resistance prevails, further steps should be taken to dispel any fears or perceptions that the introduction of self-audits would be useless. This could be done by emphasizing the benefits and positive results that are expected for each individual involved. Take an example of a production worker who is concerned about a possible negative result stemming from her self-audit. Let us say that the audit shows a decrease in output productivity. In this case, the fact that the self-audit is meant for improvement and carries no punitive action of any kind should be underscored. The worker herself would conduct such an audit. Being the most knowledgeable person about her line of work, she is also the most likely person to discover areas for improvement that would directly benefit her individually. (2) A test project with a suitable process is conducted. The project must include the process owner. For example, a quality champion or an internal quality auditor may initiate a self-audit on their own work. Experiences and results should be presented to a wider audience of people who are considering the introduction of self-audits. The process owner should also decide on the further self-action resulting from the audit. If the test project is successfully completed, others should be planned and implemented, following the principles outlined above. In the case that there are still doubts about the usefulness of the exercise, additional self-audits should be planned and executed. (3) The process owner should be formally empowered and assisted without undue interference. For example, the manager of the department in which such an audit is planned should encourage the process owner to conduct the audit. The manager should also offer assistance, for instance, in terms of additional auditor training or time to analyze results. Mandating the

Self-audit of process performance 37

IJQRM 19,1

38 (4)

(5)

(6)

(7)

execution of a self-audit or the expected results will only hinder the success of the effort. An oral or written self-audit plan is prepared and submitted to the executive management, with a particular focus on clear objectives and scope. The emphasis may rest with certain problems with the process and/or potential improvements. Adequate resources are provided, including time off regular work to conduct an audit, as well as the required training. The process owner decides on the timing of the audit and additional team members, if necessary. A self-audit is conducted with changes made to the plan and approach as found necessary by the owner. Unforeseen difficulties should be brought to the attention of those who can help. Technical advice and assistance should be readily available. This is particularly important for process owners who have just started to implement self-audits. People who undertook pilot self-audits from Step 2 can greatly assist new ``selfauditors''. In the case that technical help with respect to auditing methodology is required, experienced internal quality auditors or even consultants can provide the necessary knowledge. An oral or written report of results is drafted according to the self-audit plan. Although an oral report is convenient in most cases, it might lack a detailed list of possibilities for improvement. Therefore, it may be appropriate only after several self-audits have already been accomplished. In the initial stages of the self-audit effort, it is advisable for the process owner to prepare a written report. It is crucial that the report focuses on both the strengths and identification of real opportunities for improvement, and not just self-congratulatory statements of compliance with the selected criteria. On the basis of the self-audit report, prioritized follow-up actions are decided and approved by the process owner and management. These actions are incorporated into the business plan (for example, in several phases suggested by EFQM, 1999b). It is expected that the initial selfaudit will uncover many areas for improvement. Thus, some prioritization is required, depending on the impact that identified areas have on process performance. As a result of conducting several such audits, more evident problems will be corrected. Also, the process owner will gain experience in identifying potential problems and thus move from correction to prevention. Implemented follow-up actions are continuously reviewed for effectiveness and efficiency. Self-audits are conducted regularly, as required by the process owner or the results of previous audits. For example, if a previous audit discovered a significant number of issues that demanded immediate corrective actions, the next self-audit may be scheduled earlier than planned. Although the regularity of audits will depend on each individual case, several annual self-audits are advisable. It is important that they do not become overly bureaucratic, so if

circumstances do not warrant more than, for example two audits per year, then so be it. These seven steps of the self-audit process are illustrated with a flowchart in Figure 4. The ongoing training of participants underlines all seven steps. What are the required qualifications of participants? The self-auditor, that is, the owner of the audited process, should be familiar with the purpose and approaches of quality auditing, self-assessments and benchmarking. Experience and confidence will normally come with practice.

Self-audit of process performance 39

Figure 4. Self-audit program ± an example

IJQRM 19,1

40

Assistance from process operators, and representatives of internal and external suppliers/customers, should be available when requested. The self-auditor could further call on technical experts and professional auditors for assistance, in essence forming a self-audit team (Pasco and Brown, 1989). He/she should also be free to admit observers, when, for instance, the self-audit is conducted against a specific standard, business excellence model, or benchmarking scheme. A self-audit is not to be understood and handled as a traditional audit. The process owner wants to check performance for possible problems and improvements with more or less his/her own initiative and interest. This requires an adequate direct familiarity with the process at hand. Daily experience will help identify problems and nonconformances before they become critical. Flaws in all aspects of the process and its individual input, throughput, and output phases should not escape the attention of the person in charge. Of course, this individual must be aware of such shortcomings and potential improvements, if not during daily operations, then during the selfaudit. In brief, the qualifications of the self-auditor include: . competency to undertake the audited process; . knowledge and experience regarding the relevant technology, equipment, material and outputs (product), work places and flow, and interdependencies with other processes; . good working relationships with co-workers and superiors; . training in relevant policies and procedures governing the process; . introduction to basic principles, policies, and approaches/methods of process evaluation; . ability to evaluate objectively and without fear own work and process related performance; . ability to cooperate with professional auditors, trained assessors and technical experts. While some of these qualifications simply require a level of professionalism that comes with normal education and training for the job, the more technical requirements can be fulfilled by adequate training. For example, good working relationships with colleagues and expertise in the process itself are required for successful process ownership and management. Training in internal quality auditing and process evaluation techniques may be obtained by taking auditing courses that are readily available through quality system registrars or national quality organizations. Experienced self-auditors may be able to provide additional training, as well. All other participants should assist in the self-audit, as requested by the process owner. They should be adequately informed about the ongoing self-audit project, especially its objectives and results.

Example The following is a hypothetical case, designed to simultaneously describe a self-audit process and to further outline the difference between a self-audit and its traditional counterpart. As a process under evaluation, we have chosen the traditional quality audit, conducted by the process owner (internal auditor). Any audit is in itself a process, an integral part of an organizational management system (Karapetrovic and Willborn, 2000), and therefore can be examined for effectiveness and efficiency. ``Who audits me, the auditor?'' This is the question asked by an auditor, let us call her Karla, of a hotel chain with an ISO 9001 registered quality management system. The hotel's senior manager Mike answered: ``The auditor of our registrar will have to assess our internal audit. Furthermore our regular management review will have to decide if our internal audits are satisfactory.'' The internal auditor, however, wanted to assure herself prior to the external audit and the management review, that her audits can stand-up to the ISO 10011 audit guidelines. Karla had been impressed by the self-audits both the kitchen chef and the headwaiter had conducted. These departments had achieved significant improvements, recognized by her recent internal audit. The hotel manager was surprised by this plan for a ``self-audit of the audit''. Nevertheless, having become familiar with the other two self-audits in the hotel, there was no reason why the auditor should not proceed. Self-audit of the internal audit process was certainly innovative and would help the hotel to stand out among the other hotels in the chain. Having gained the manager's support, the internal auditor proceeded as outlined in Figure 4. Step 1, initiation, and Step 2, test project, had already been accomplished in her case, since two self-audits had been conducted within the kitchen and dining room departments. Step 3, planning, involved the assurance of qualifications for performing a self-audit, which the internal auditor felt she possessed by having participated in several self-assessments and benchmarking projects while she was with a previous employer. This step also included the empowerment to conduct a self-audit granted by the hotel manager. Karla also called on the kitchen chef (Paul) and the headwaiter (Vladimir) to assist her. Both were initially surprised, because they were used to being Karla's auditees. But they rightly expected some personal benefits from participating in the auditor's self-evaluation, and therefore agreed to participate. Subsequently, Karla prepared a written self-audit plan, mainly for informing her two assistants. The project aimed at evaluating the last two comprehensive audits of the assistants' areas, the kitchen and the restaurant. The main purpose was to evaluate if these recent audits had been properly planned and executed. Naturally, it would be hard to answer this question during the execution of internal quality audits, since both the auditor (Karla) and the auditees (Paul and Vladimir with their staff) were under pressure to verify ISO 9000 compliance. This time, in a much more relaxed state of mind, the two assistants made a few suggestions to the self-audit plan, based on their recent experiences as auditees. For instance, they suggested two afternoons

Self-audit of process performance 41

IJQRM 19,1

42

(relatively quiet time of the day) as the scheduled time for the self-audit, as well as drafting the audit report in point form on one page only, for brevity and clarity. Step 4, execution, included the preparation of a brief checklist, containing the requirements of the ISO 10011 (1990) standard. Karla then discussed the checklist with Paul and Vladimir, who suggested some additional questions. For instance, they pointed out that the operations in the patisserie, for certain food processing procedures, and training of sous-chefs were missed last time the audit was conducted. Karla then retrieved the audit plans and checked these with the procedure suggested by the ISO 10011 guideline. Following the checklist, she revisited the individual phases of the internal audit, such as the opening meeting, interviews, observations, presentation of the findings, and follow-up measures. The opening meeting was too time consuming, a brief memo would have done sufficiently. It was found that the auditor had overlooked the requirement that auditees have to be properly informed about an impending audit in their area of responsibility. Food samples taken, as well as hotel guests selected for the audit, were not sufficiently representative of the population. The audit was poorly timed during the ``rush hours'' when staff members could not pay adequate attention to the auditor's questions. In the restaurant, the auditor had taken the position of a guest, while at the next table an important regular guest had to be attended to. Finally, the audit report had not been sent to the auditee prior to its submittal to management and its management review. Step 5, report, concentrated on making notes of the findings directly in the checklist and a brief discussion of future improvements with the assistants. Karla informed the hotel manager that the self-audit was completed, giving full credit to the kitchen chef and the headwaiter for their support. The manager suggested that the self-audit should be further discussed in the next management review meeting. Other department heads would be interested, so that gradually a self-assessment program against a business excellence model may evolve. This would also ease the change to the new ISO 9001 (2000) standard, which requires continual measurement and improvement of performance results. During the next Step 6, follow-up, actions were taken to correct perceived shortcomings, as well as to prevent future inconsistencies with the ISO 10011 guideline. As a result, Karla made changes to the internal quality audit plan, strengthening, among others, the applied sampling and scheduling methods. For the next self-audit in the kitchen and restaurant departments, she was invited to participate and assist in internal benchmarking. The results of the three conducted self-audits were reviewed in the hotel management meeting. It was decided to broaden their scope to other departments, together with the Step 7 continual review of the performance of consequent self-audits and business plans.

Conclusions In recent years, it has become apparent that organizations competing in any kind of market cannot rely solely on ISO 9000 standards to meet the increasing demands for continuous improvement and business excellence. Consequently, the traditional quality auditing methodology designed to test quality assurance systems against the standards falls well short of enabling performance improvement. While there is little doubt that a system audit is an excellent tool for independent, objective and systematic evaluation against the standard's minimum requirements, based on professional and statistically sound judgments, there is even less doubt that some changes are required. Among the problems that continue to plague the audit are the lack of motivation of participants, narrow focus on quality assurance, non-existing linkages with business planning, and the inability to measure efficiency or at least focus on business results. Meanwhile, self-assessments against quality award models have gained prominence in exactly the areas where quality audits were lacking, most importantly performance improvement. Although they cannot replace quality audits, self-assessments possess one crucial characteristic that can improve their more formal counterpart: self-evaluation. In this paper, we proposed that the process owner conduct periodic selfevaluations of process performance. This concept, branded as a ``self-audit'', is largely based on the internal quality audit and self-assessment methodology. Its strength, compared to the conventional audit, lies in a less rigorous evaluation, little external control of compliance with set procedures and official guidelines, and the opportunity for joint identification of improvements in the work place. In a small business, a self-audit may be the only practical process evaluation method available. Here the manager, auditor, auditee and the client are usually functions of the same person. The independence principle of the traditional audit would require at least two internal auditors in the company, regardless of how small it is. A self-audit is also expected to bring a solid foundation in data collection, verification and evaluation, as well as more objectivity to self-assessments. Apart from defining the concept of self-audit, stating the reasons for its introduction, discussing the relationships of the selfaudit with other evaluation methodologies, and analyzing the scope of potential application, this paper formulated the main principles of the self-audit, and provided some of the prerequisites for the application. Subsequently, the process of implementation was discussed and illustrated with a hypothetical example. Finally, further research in the area of quality auditing, aimed particularly at improvements in efficiency and effectiveness of the methodology, is particularly encouraged. Specifically to the self-audit concept, testing and verification in a business environment, as well as an empirical study of the application, are suggested. Research into the conversion from internal quality auditing to self-auditing in different industries would also be beneficial. Of particular interest would be an analysis of the usefulness and applicability of the concept in small to medium sized enterprises. Furthermore, integrative

Self-audit of process performance 43

IJQRM 19,1

approaches to performance evaluation, including auditing, self-assessments, benchmarking and performance measurements are required. Examination of the possibility of expanding the self-audit concept into areas other than quality, such as environmental, safety and dependability management, is another possible topic.

44

References Adams, S. (1997), The Dilbert Principle ± A Cubicle's-Eye View of Bosses, Meetings, Management Fads and Other Workplace Afflictions, Harper Business, New York, NY. ANSI/ASQC (1986), American National Standard: Generic Guidelines for Auditing of Quality Systems, American Society for Quality Control, Milwaukee, WI. Arter, D.R. (1989), ``Improvement through different forms of the audit'', ASQC Quality Congress Transactions, Toronto, p. 920. Arter, D. (2000), ``Internal, external quality audits couldn't have prevented recall'', Quality Progress, Vol. 33 No. 12, pp. 33-4. Andersen, B. (2000), ``21 voices for the 21st century'', Quality Progress, Vol. 33 No. 1, p. 33. Beeler, D.L. (1999), ``Internal auditing: the big lies'', Quality Progress, Vol. 32 No. 5, pp. 73-8. Besterfield, D.H., Besterfield-Michna, C., Besterfield, G.H. and Besterfield-Sacre, M. (1999), Total Quality Management, 2nd ed., Prentice-Hall, Englewood Cliffs, NJ. Bishara, R.H. and Wyrick, M.L. (1994), ``A systematic approach to quality assurance auditing'', Quality Progress, Vol. 27 No. 12, pp. 67-70. Cambridge (2000), Cambridge International Dictionary of English, Cambridge University Press, Cambridge. Chong, A. (2000), ``21 voices for the 21st century'', Quality Progress, Vol. 33 No. 1, p. 33. Dale, B.G. (1999), Managing Quality, 3rd ed., Blackwell Publishers, Oxford. EFQM (1999a), Assessing for Excellence: A Practical Guide for Self-Assessment, European Foundation for Quality Management, Brussels. EFQM (1999b), The EFQM Excellence Model, European Foundation for Quality Management, Brussels. Forsha, H.I. (1992), ``The pursuit of quality through personal change'', Quality Progress, Vol. 25 No. 2, pp. 61-4. Goldstein, R. (1983), ``The two-tier audit system'', ASQC Quality Congress Transactions, Boston, MA, pp. 14-19. ISO 9000 (2000), Quality Management Systems ± Fundamentals and Vocabulary, Final draft International Standard, International Organization for Standardization, Geneva. Juran, J.M. and Gryna, F.M. (1993), Quality Planning and Analysis, 3rd ed., McGraw-Hill, New York, NY. Karapetrovic, S. (1999), ``ISO 9000: the system emerging from the vicious circle of compliance'', TQM Magazine, Vol. 11 No. 2, pp. 111-20. Karapetrovic, S. and Willborn, W. (2000), ``Generic audit of management systems: fundamentals'', Managerial Auditing Journal, Vol. 15 No. 6, pp. 279-94. Karapetrovic, S. and Willborn, W. (2001), ``Audit system: concepts and practices'', Total Quality Management, Vol. 12 No. 1, pp. 13-28. Neely, A., Gregory, M. and Platts, K. (1995), ``Performance measurement system design: a literature review and research agenda'', International Journal of Operations & Production Management, Vol. 15 No. 4, pp. 80-116.

Pasco, G.K. and Brown, V.S. (1989), ``Self auditing in a continuous improvement culture'', ASQC Quality Congress Transactions, Toronto, pp. 912-22. Pyzdek, T. (1999), ``A road map for quality beyond control'', Quality Progress, Vol. 32 No. 12, p. 34. Roberts, H.V. (1993), ``Using personal checklists to facilitate TQM'', Quality Progress, Vol. 26 No. 6, pp. 51-6. Shirley, B.M. and Gryna, F.M. (1998), ``Work design for self-control in financial services'', Quality Progress, Vol. 31 No. 5, pp. 67-71. Stamatis, D.H. (2000), ``Design and manufacturing FMEAs are called for'', Quality Progress, Vol. 33 No. 12, pp. 43-5. Tamimi, N., Gershon, M. and Currall, S.C. (1995), ``Assessing the psychometric properties of Deming's 14 principles'', Quality Management Journal, Vol. 2 No. 3, pp. 38-52. Tiemeyer, H. (1997), ``Mitarbeiter einbinden'', QualitaÈt und ZuverlaÈssigkeit, Vol. 42 No. 2, pp. 190-5. Van der Wiele, T., Dale, B. and Williams, R. (2000), ``Business improvement through quality management systems'', Management Decision, Vol. 38 No. 1, pp. 19-23. Willborn, W. and Cheng, T.C.E. (1994), Global Management of Quality Assurance Systems, McGraw-Hill, New York, NY. Zuckerman, A. (2000), ``Some believe QS-9000 plays no role in situation'', Quality Progress, Vol. 33 No. 12, pp. 45-6.

Self-audit of process performance 45