Service Provider Assessment Framework
Service Provider Assessment Framework
A Platform for Building Synergies between Clients and Service Providers for Trusted Global Sourcing
A Study Report Data Security Council of India in collaboration with Ernst & Young December 2010
Under Cyber Security Awareness Program, Department of Information Technology, Government of India
About DSCI
Data Security Council of India (DSCI) is a section 25, not-for-profit company, setup by NASSCOM as an independent Self Regulatory Organization (SRO) to promote data protection, develop security and privacy codes & standards, and encourage the IT/BPO industry to implement the same. For more information about DSCI or this report, please contact: Data Security Council of India Niryat Bhawan, 3rd Floor, Rao Tula Ram Marg, New Delhi 110057, India Phone: +91-11-26155070 Fax: +91-11-26155072 Email: info@[Link] 2010 DSCI. All rights reserved.
Disclaimer
This document contains information that is Intellectual Property of DSCI. DSCI expressly disclaims to the maximum limit permissible by law, all warranties, express or implied, including, but not limiting to implied warranties of merchantability, fitness for a particular purpose and non-infringement. DSCI disclaims responsibility for any loss, injury, liability or damage of any kind resulting from and arising out of use of this material/information or part thereof. Views expressed herein are views of DSCI and/or its respective authors and should not be construed as legal advice or legal opinion. Further, the general availability of information or part thereof does not intend to constitute legal advice or to create a Lawyer/ Attorney-Client relationship, in any manner whatsoever.
Foreword
The IT (Amendment) Act, 2008 has established a strong data protection regime in the country, by requiring body corporates to implement reasonable security practices to protect sensitive personal information. What is reasonable security though? An organization is expected to have a comprehensive information security program, with appropriate controls that are commensurate with its information assets and risk assessment. In the event of a security breach, it should be able to demonstrate that its practices were in conformance with its written security policy, and that its controls were adequate. It is, however not that easy, since enterprises are outsourcing some of their work, and they must manage information risk across a vast global network of Service Providers. Outsourcing thus brings into focus the practices followed by Service Providers, and their accountability. Service Providers are subjected to ongoing assessments and on-site audits, which are laborintensive and costly for both the sides. Likewise, Service Providers with hundreds of Clients distributed in various geographies must submit themselves to several audits by the Clients. Moreover, the multiple assessments are based on different frameworks, questionnaires and audit approaches clearly they result in wasted effort and time; and, of course, higher costs. It is the wish of both - Clients and Service Providers - that third-party evaluations that are standards-based, or framework-based, may ease the assessment burden. But how do they view the implementation of a standard, or best practices for security; and an assessment framework to validate that this has indeed made the organization secure? Again both of them will have a different perspective on this. Can enterprises take a methodical approach to assessing and managing the risks through frameworks like ISO 27001; BITS Shared Assessment Program, Moodys Vendor Information Risks ratings, Information Security Forum, COSO, NIST or COBIT? Will attestation of a Service Providers practices necessarily be in the form of a third-party certification, or a maturity rating of its practices?
With DSCI best practices and data-centric methodology, weve rolled out a solution for adoption by Service Providers to make them secure. DSCI Security Framework (DSF) is based on a number of security principles, that help make the security program of an organization dynamic, instead of a static checklist approach that relies on bulky documentation. We wanted to review the available assessment frameworks, to see how DSF could fit into them, and how rating of practices may give a sense of security to organizations, and also show them the direction for improvements. In short, itll help realize an effective security program, and transparent assessment framework, that may address the concerns of both Clients and Service Providers. In the process, reasonable security practices will get implemented. It is with this in view that DSCI partnered with Ernst & Young Pvt. Ltd. (EY) in this study, which required extensive knowledge and experience in the domain, to review the existing frameworks and think through the advantages of certification/ratings. Survey of Clients and Service Providers, based on an in-depth questionnaire gives key pointers to the concerns of both the groups, and points towards a possible third-party ratings approach that may be useful and acceptable to both, namely Clients and Service Providers. I would like to acknowledge the great team effort of DSCI and EY in conducting this study, and creating a useful analysis. I hope this report will generate sufficient interest among Clients, Service Providers, and even governments and regulators that will help DSCI arrive at the right decisions in taking the next steps in certification/rating of Service Providers.
Executive summary
Businesses today are global, complex and fast evolving, and technology has made business transactions independent of space and time. This has enabled businesses to focus on its core competencies and outsource non-core business operations to Service Providers, who are capable of providing services to the businesses from around the world round the clock. Information Security and Privacy becomes crucial when it comes to outsourcing as technology enables free flow of information across borders between Clients and Service Providers. This information could be business sensitive information and / or sensitive personal information of the Clients end customers including but not limited to health related information, credit card details, social security number, etc. Also, stringent global data protection regulations make the businesses liable for loss, misuse, wrongful disclosure of any personal information of any citizen irrespective whether the failure is at outsourcers end or Service Providers end. The Indian IT/BPO Service Providers are striving hard to ensure that security and privacy of data is well maintained. They follow stringent security controls specified by the Clients through contractual obligations. The Clients conduct regular Information Security and Privacy assessments of the Service Providers to ensure compliance with the contractual obligations and / or regulatory requirements or to simply assess the security posture of Service Providers. In this outsourcing ecosystem, many Clients have developed and applied their own proprietary assessment frameworks for evaluating their Service Providers. Service Providers, on the other hand, strain their resources to respond to diverse client information requests. This isolated approach proves to be an inefficient and costly affair, both for the Clients and the Service Providers. Inconsistencies arising from use of different assessment methodologies cause delays, resulting in inefficient use of time and resources. Aggravating the problem is the unavailability of generally accepted standard for Service Provider assessments. To overcome these issues and challenges, DSCI as an industry initiative seeks to establish a well defined Service Provider Assessment Framework in order to have a common assessment approach that can be used to assess different Service Providers. This study especially through its survey attempts to understand the perspective of Client and Service Provider organizations with respect to Service Provider assessments and takes inputs to define a Service Provider Assessment Framework.
DSCI should play a vital role in conducting Service Provider assessments and sharing the outcome in the ecosystem. It should:
have an Service Provider assessment program that comprises of framework, processes, and methodology for assessments provide an organization wide security and privacy maturity rating, and domain specific maturity rating that may be shared in the ecosystem after taking the due permission of the Service Providers
A new standard mapped to prevalent standards should be considered as a potential assessment standard for third party assessments of Service Providers DSCI as an industry initiative and a Self Regulatory Organization having representation from both the Client and Service Provider organizations should empanel auditing firms for conducting independent third party assessments of Service Providers
The study also focused on understanding of various assessment models which included Malcolm Baldrige Framework, Capability Maturity Model Integration (CMMI), CRISIL Ratings, BITS framework, e-Sourcing Capability Model (eSCM), Moodys assessment framework. The study of assessment models reveals that:
Service Provider Assessment Framework should be easy to comprehend and adaptable regardless of size of the organization and nature/ complexity of its processes The framework assessment areas should be outlined in the form of best practices rather than a stringent set of controls. This would provide an opportunity to organizations for implementing / performing the control activities according to the needs of the organization specific environment The framework should follow a process-approach and outline measurable assessment areas It should be reviewed and updated (if required) on a periodic basis The maturity criteria should be transparent, and should help in assigning a formal maturity rating to a Service Provider
Overall, DSCI may develop a Service Provider Assessment Framework that is aligned to DSF Best Practices & the maturity criteria defined for each of its sixteen security disciplines and the study results elucidated in this report; and make it popular in the ecosystem by performing pilot runs. The framework may follow a CMMI-like rating methodology which is assessment of the security and privacy practices at both the layers capability/ maturity of the business processes, and maturity of the organization.
Content
Introduction .................................................................................................................. 1 Survey Highlights......................................................................................................... 5 Detailed Survey Results ...............................................................................................7 Key drivers for Service Provider assessments ....................................................7 Scale of Service Provider assessments ..............................................................9 Current assessment program/ mechanism .......................................................11 Focus on Data Privacy in Service Provider assessments .................................13 Types of Service Provider assessments ...........................................................14 Level of perceived risk IT services .................................................................15 Level of perceived risk BPO services .............................................................17 Risk profiling of Service Providers.....................................................................18 Frequency of Service Provider assessments ....................................................19 Budget and cost for Service Provider assessments ..........................................21 Modes of Service Provider assessments ..........................................................23 Service Provider assessment challenges..........................................................25 Service Provider assessments solutions and future landscape .....................27 Influence of IT (Amendment) Act, 2008 on Service Provider assessments ......29 Third party assessments ...................................................................................31 Third party assessors ........................................................................................33 Standards for Service Provider assessments ...................................................35 Role of DSCI in Service Provider assessments ................................................37 Outcome of Service Provider assessments ......................................................39 Sharing of Service Provider assessment results ...............................................41 Recommendations......................................................................................................43 Annexure ....................................................................................................................45 Glossary .....................................................................................................................57 References .................................................................................................................57
Introduction
Background
As buyers of Information Technology (IT) and Business Process Outsourcing (BPO) services become increasingly sophisticated and demanding, Service Providers are challenged to achieve new levels of efficiency, agility and transparency in service delivery and protection of information. Clients increasingly expect real evidence of robust process management, continuous improvement, effective governance, and measures adopted for ensuring Information Security and Privacy.
Objective
DSCI engaged EY to study the current landscape of Service Provider (IT/BPO organizations) assessments conducted by the Client organizations, and assist in documenting the assessment approach that may be adopted in order to minimize the challenges of both, Client and Service Provider organizations, with an intent of evaluating and reporting on Information Security and Privacy posture of the Service Providers.
Approach
In order to achieve the project objectives, the joint study team undertook the following steps:
Primary research: A survey of Client and Service Provider organizations was undertaken to gain an insight into the current Service Provider assessment program. The survey covered the following aspects:
The value that various Service Provider assessments conducted by Client organizations bring to the Service Providers Investments made, and challenges faced by the Service Provider and Client organizations in driving such assessments Possible solutions for overcoming the current challenges Role of DSCI and third parties in Service Provider assessments
Secondary research: A study was undertaken to document the pros and cons of prevalent assessment frameworks like Capability Maturity Model Integration (CMMI), BITS shared assessment program, Carnegie Mellon University e-Sourcing Capability Model (eSCM), etc. The list of assessment frameworks was documented on the basis their widespread use, and international recognition in performing assessments. The study areas included the following:
Assessment areas / ease of use by the organization being assessed Assessment methodology / scoring pattern / process of sharing assessment results Acceptability / popularity of the framework Independence of examiners Frequency of framework update to cater to future requirements
The team also studied DSCI Security Framework (DSF) Best Practices and maturity rating criteria for each of its sixteen disciplines to gather inputs (in addition to the inputs provided by primary and secondary research) for defining Service Provider Assessment Framework.
Profile of participants
The survey respondents were a set of Client and Service Provider organizations. The respondents were majorly from Information Technology (IT), Business Process Outsourcing (BPO), Telecommunications and Financial Services verticals. Correspondingly, the survey results have been divided into two perspectives Clients perspective and Service Providers perspective, and may be read accordingly.
Client organizations
Industry wise distribution
9.00%
37.00% 36.00%
42.00% 50.00%
BPO
IT Services
KPO
The sample size selected for the survey was limited and this should be taken into consideration when interpreting the survey results.
Client organizations
More than $ 24 billion $ 1 billion to $ 24 billion $ 100 million to $ 249 million Less than $ 100 million 1 2 3 3
Survey highlights
Service Provider assessments are conducted by Client organizations in order to protect business sensitive information, and mitigate security & privacy risks while outsourcing work to Indian IT/ BPO companies. These assessments help Service Provider organizations to align security & privacy initiatives to their Clients requirements and build on the existing relationship with the Clients Comprehensive risk based assessments covering all the domains of security are carried out annually by majority of Client organizations. Vulnerability assessments and penetration testing continue to display strong acceptance (100%) by Client organizations in Service Provider assessment programs Most of the Service Provider organizations reported that ISO 27001 controls checklist is used as a mechanism by their Clients for conducting assessments. On the other hand, Client organizations revealed that a proprietary Service Provider assessment program has been developed to conduct Service Provider assessments Provisions of the IT (Amendment) Act, 2008 (ITAA 2008) need to be appropriately incorporated in the Client-Service Provider contracts High number of assessments around the year is the most critical challenge faced by Service Providers at the time of assessments, followed by meeting diverse & varied assessment. Whereas for Clients, rising legal liabilities, regulatory requirements, level of security awareness in the Service Providers, ensuring compliance by Service Providers, and Service Provider commitment to ensure Information Security & Privacy are some of the critical challenges faced in assessing Service Providers
Currently, Service Provider assessments are majorly conducted onsite by Clients internal staff. Majority of the Client organizations indicated that auditing firms empanelled by a joint industry consortium of outsourcers and the Service Providers could act as the third party assessors for conducting independent Service Provider assessments More than half of the Service Provider respondents suggested that DSCI should have a Service Provider assessment program that comprises of framework, processes, methodology for assessments Clients and Service Providers reveal that third parties should conduct Service Provider assessments, based on a standardized assessment methodology. This would save costs and efforts by avoiding the need for conducting assessments of multiples Service Providers Both Client and Service Provider respondents suggested a new standard mapped to ISO 27001, NIST SP, COBIT, ITIL etc. that meets all the regulatory requirements like GLBA, HIPAA, PCI DSS etc., as a potential assessment standard for third party assessments of Service Providers DSCI should provide organization wide security and privacy maturity rating, and also domain specific maturity rating
Clients perspective
Business drivers for conducting Service Provider assessments
Protecting business sensitive information including intellectual property Mitigating security and privacy risks that exist in outsourcing arrangements To address the security and privacy concerns of some of the key stakeholders within our organization Achieving end customer confidence and preventing loss of reputation by mitigating risks of privacy/ information leakage that may arise at Service Provider end Strengthening of data protection regime in the geographies where we operate, stipulating stringent requirements and heavy fines for a data breach Data protection regulations demand our organization to undertake regular assessments of third parties Our corporate policies require us to undertake a comprehensive vendor risk assessment Addressing security and privacy risks that arise from use of emerging technologies Use Service Provider assessments as a mechanism to foster a culture of compliance at all Service Providers and introducing a sense of competition among them with regards to fulfillment of their data security and data privacy needs 88.89%
88.89%
77.78%
77.78%
55.55%
55.55%
44.44% 44.44%
33.33%
Service Provider assessment as a mechanism to foster a culture of compliance was selected by the least number (thirty three percent) of Clients while the same response was selected by fifty percent of the Service Provider organizations, as a reason for conducting assessments.
66.67%
66.67%
58.33%
58.33%
Protecting business sensitive information and mitigating security and privacy risks are the major drivers for conducting Service Provider assessments
50.00%
50.00%
41.67%
41.67%
Clients perspective
Number of Service Providers the organization is engaged with
600
Clients
Number of Clients
10
11
12
Service Providers
10
Clients perspective
Service Provider assessment program/mechanism used by the organization
Assessment program developed by our organization (proprietary)
ISO 27001 controls checklist
77.78%
44.44%
Reliance on Statement on Auditing Standards (SAS) No. 70 report provided by the auditing firm assessing your Service Providers
Asking the Service Providers to get ISO 27001 certified thereby eliminating the need for getting assessed
44.44%
33.33%
Assessment program of the appointed external auditor Asking the Service Providers to provide self declaration/attestation for compliance to our security policies/requirements Use pre-defined controls list provided by an assessment tool BITS shared assessment program
22.22%
22.22%
11.11%
0.00%
11
78% Client organizations use proprietary assessment programs for conducting Service Provider assessments. However, the Service Providers report that their Clients use ISO 27001 checklist for conducting security and privacy assessments
12
Clients perspective
Coverage of privacy in Service Provider assessments
Majority of the Service Providers report that their Clients do not cover Privacy during assessments while Clients strongly perpetuate the coverage of Privacy in Service Provider assessments
11.00%
33.00%
56.00%
Strongly
25.00% 41.67%
33.33%
Minority of clients Service Providers assessment programs cover Privacy Nearly half of the clients assessment programs cover Privacy
Majority of clients Service Providers assessment programs cover Privacy None of the clients Service Provider assessment programs cover Privacy (0%)
13
Clients perspective
Different types of Service Provider assessments conducted by the organization
100.00%
88.89%
Lines of Service specific assessment (e.g. conducting application security assessment for application development services) Regulatory / Compliance: Assessments to check compliance with applicable regulations (e.g. HIPAA, GLBA) or Assessments based on compliance to Standards like ISO 27001 and PCI DSS
77.78%
Service Providers reveal that Client organizations display a strong propensity towards undertaking comprehensive risk-based assessments, and compliance based assessments
77.78%
14
Clients perspective Custom Application Development, Network and Desktop Outsourcing together with Infrastructure outsourcing are current watchwords in the context of Service Provider assessments
Level of perceived risks in the services outsourced by Client organizations: IT services
Custom application development Network and desktop outsourcing Infrastructure services outsourcing Software deployment and support Application management Software testing Hosted application management Hosted infrastructure services System integration Network consulting and integration Hardware deployment and support IT consulting IT education and training 77.78% 66.67% 66.67% 66.67% 55.56% 44.44% 44.44% 44.44% 44.44% 33.33% 33.33% 22.22% 11.11% High 22.22% 11.11% 22.22% 22.22% Medium Low 11.11% 22.22% 11.11% 11.11% 11.11% 11.11% 33.33% 44.44% 33.33% 33.33% 22.22% 22.22% 33.33% 33.33% 44.44% 22.22%
15
16
Clients perspective
Level of perceived risks in the service outsourced by Client organizations: BPO services
Finance and accounting Human resource management Knowledge services Vertical specific BPO services Customer interaction and support Procurement services 66.67% 44.44% 44.44% 44.44% 44.44% 22.22% High Medium 33.33% Low 11.11% 33.33% 22.22% 22.22% 22.22% 11.11% 11.11%
11.11% 11.11%
Finance and Accounting services are considered important by majority of the organizations in the context of Service Provider assessments
17
Clients perspective
11.00%
89.00%
89% of the Client organizations rely on risk profiling to determine the frequency of Service Provider assessments
18
Clients perspective The Service Providers with critical risk undergo quarterly assessments as per thirty three percent of Client organizations
Frequency of assessing the Service Providers
Critical risk
33.33%
11.11%
22.22%
High risk
22.22%
22.22%
22.22%
Medium risk
55.56%
11.11%
Low risk
22.22%
44.44%
Negligible
11.11%
33.33%
Quarterly
Half yearly
Yearly
19
20
Clients perspective The cost of periodic Service Provider assessments is built into the service delivery cost of Service Providers, and is a part of the contractual terms
Portion of the IT security budget allocated for conducting Service Provider assessments
Significant Considerable Small Negligible 22.22% 11.11% 22.22% 44.44%
44.44%
22.22%
11.11%
11.11%
11.11%
21
22
Clients perspective
*For Client organizations that undertake risk profiling of Service Providers
Type Category Self Assessment (offshore) Telephonic (offshore) Onsite by Org Internal staff Onsite by org internal staff and sourced assessors from auditing firms 4 3 3 1 1 Onsite by sourced assessors Third Party Assessments
2 2 4 5 3
2 2 2 3 2
6 6 6 4 4
3 4 1 0 0
3 3 2 0 0
*This data table is for eight Clients. Eight out of nine Clients interviewed undertake risk profiling.
23
100.00%
75.00%
66.67%
66.67%
Onsite assessments by Clients internal staff or sourced assessors is the preferred mode of assessment by Clients
25.00%
8.33%
8.33%
24
Clients perspective
Challenges faced by Client organizations
Comfort/ assurance provided by certifications like ISO 27001 Subcontracting by the Service Providers Adoption of Non standardized Information Security and Privacy framework 44.44% 44.44% 33.33% 22.22% 22.22% 22.22% 22.22% 22.22% 44.44% 22.22% 22.22% 22.22% 22.22% 33.33% 55.56% 44.44% 44.44% 22.22% 55.56% 22.22% 55.56% 55.56% 44.44% 55.56% 44.44% 11.11% 44.44% 44.44% 33.33% 11.11% 33.33% 44.44% 33.33% 55.56% 55.56% 11.11% 11.11% 44.44% 22.22% 11.11% 33.33%
Subcontracting by the Service Providers and comfort provided by certifications like ISO 27001 are most significant assessment challenges faced by Client organizations
Inadequate budget Availability of skilled resources for conducting the assessments Auditor accreditation and Auditors management Service Provider commitment Meeting multiple customer requirements
Quantum of assessments 11.11% Rising legal liabilities/ 11.11% regulatory requirements Level of security awareness in 11.11% the Service Providers Multiple Service Providers for different 11.11% lines of services in multiple geographies Ensuring compliance by your 11.11% Service Provider Sensitizing key resources of 11.11% Service Providers High direct and indirect costs 11.11% Nature of outsourced work Tracking and closure of assessment findings
High
Medium
Low
25
Factors such as cost, quantum of assessments were the least important challenges as perceived by Client organizations. Whereas, majority of Service Providers perceive high number of assessments around the year as one of the most significant challenges. This difference in opinion regarding the challenges faced by Client and Service Provider organizations clearly indicates development of a robust assessment solution that meets the requirements of both parties.
33.33%
41.67%
8.33%
33.33%
33.33%
25.00%
25.00%
33.33%
16.67%
50.00%
16.67%
8.33%
33.33%
25.00%
Medium
Low
High number of assessments around the year, and meeting diverse Client requirements are critical challenges faced by most of the Service Providers
26
Clients perspective
Possible solution to overcome identified challenges
As per Client organizations, industry and Service Provider promoted and standardized third party assessment program can be used for assessments. This is closely followed by development and adoption of an international standard
Industry & Service Provider promoted and standardized third party assessment programs like BITS Development and adoption of international standards for Service Provider Assessment Independent third party assessments conducted by Self Regulatory Organizations (SRO) promoted by the Service Providers ISO 27001 certification should be accepted globally as a seal of trust and assurance; eliminating the need for Service Provider assessments
44.00%
11.00%
33.00%
11.00%
22.00%
11.00% 11.00%
22.00%
33.00%
11.00%
Self declaration by Service Providers for complying / fulfilling clients security requirements, thereby 11.00% making them liable for any security incident/data breach / violation should suffice There is no need for Service Provider assessments as data security and privacy risks are already addressed through contracts First reference 11.00%
22.00%
11.00%
Second preference
Third preference
27
33.33%
25.00%
8.33%
25.00%
8.33%
33.33%
8.33%
41.67%
8.33%
8.33%
25.00%
16.67%
0.00%
First preference
Second preference
Third preference
Development and adoption of an international standard is the first preference chosen by Service Providers
28
IT (Amendment) Act, 2008 needs to be incorporated in ClientService Provider contracts this would assist in strengthening the data protection initiatives of Service Providers
Clients perspective
Influence of IT (Amendment) Act, 2008 on Service Provider assessment strategy
Provisions of IT (Amendment) Act, 2008 need to be appropriately incorporated in the client-Service Provider contracts IT (Amendment) Act, 2008 will have no impact as we need to comply with regulations we are subjected to IT (Amendment) Act, 2008 will strengthen the data protection initiatives of Indian Service Providers and therefore will help provide greater assurance to us for outsourcing our work to India Im not aware of IT (Amendment) Act, 2008 11.11%
66.67%
33.33%
33.33%
Self declaration by Service Providers for complying/fulfilling clients security requirements, 0.00% thereby making them liable for any security incident/data breach/violation should suffice
29
58.33%
0.00%
8.33%
30
Clients perspective
Options for third party assessments
Third parties can conduct assessments of the Service Providers, based on a standardized assessment methodology, at a defined frequency Third Party assessments would save costs and efforts by avoiding the need for conducting assessments of multiple Service Providers Our regulators / customers may not approve or recognize Third Party assessments Third party assessments can be successful only if it is accepted by the outsourcing community and regulators Third Party assessments will bring transparency and independence The Third Party assessments will ensure that our resources are able to focus on improving security & privacy posture Third Party assessments may not be able to address the specific assessment requirements arising out of a particular Client-Service Provider relationship My organizations Executive Management may not approve or recognize Third Party assessments Adopting Third Party assessments may raise trust and accountability issues 22.22%
Majority of Clients and Service Providers report that third parties should conduct Service Provider assessments, based on a standardized assessment methodology at a defined frequency
66.67%
66.67%
55.56%
55.56%
55.56%
55.56%
11.11%
11.11%
31
Thirty three percent of Service Providers expressed their concerns regarding the use of third party assessments stating that they third party assessments could be helpful if their Clients accept these.
41.67%
41.67%
41.67%
33.33%
Third party assessments would save cost and efforts by avoiding multiple assessments from different Clients
16.67%
16.67%
32
Clients perspective
Potential entity acting as third party for conducting independent Service Provider assessments
Auditing firms empanelled by a joint industry consortium of outsourcers and the Service Providers Auditing firms empanelled by the outsourcers industry consortium
66.67%
55.56%
Auditing firms empanelled by a joint industry consortium of Client and Service Providers can serve as third party assessors for conducting Service Provider assessments
Self Regulatory Organizations (SRO) promoted by the Service Providers Auditing firms empanelled by the Service Providers 0.00%
33.33%
33
Self Regulatory Organizations (SRO) promoted by the Service Providers Auditing firms empanelled by a joint industry consortium of outsourcers and the Service Providers Auditing firms empanelled by the outsourcers industry consortium Auditing firms empanelled by the Service Providers 8.33% 25.00%
58.33
50.00%
34
A new standard mapped to ISO 27001, NIST-SP, COBIT, ITIL etc. that meets all the regulatory requirements like GLBA, HIPAA, PCI DSS etc. as a standard for third party assessments
Both Client and Service Provider organizations have similar number of respondents who selected ISO 27001 (sixty six percentages). It seemed that organizations are satisfied with the acceptance of ISO 27001 as a
Clients perspective
Potential assessment standards for third party assessments of Service Providers
A new standard mapped to ISO 27001, NIST SP, COBIT, ITIL, etc. that meets all the regulatory requirements like GLBA, HIPAA, PCI DSS, etc. ISO 27001 standard 22.22% 66.67% 88.89%
BITS shared assessment framework Security and Privacy practices defined by SRO Others
22.22%
11.11%
35
standard bearing in mind the challenge faced by Client organizations with respect to the comfort/ assurance provided by Service Providers through ISO 27001 certification. Respondents in the Others category also suggested the use of a unified compliance framework for assessments.
36
Majority of Clients and Service Providers perpetuated that DSCI should have a Service Provider assessment program that consists of framework, processes and methodology of assessments
Clients perspective
Role of DSCI in Service Provider assessments
DSCI should have code of practices for security and privacy that need to adopted by its members The code of practices should have some criteria for assessing the maturity of the Service Providers DSCI should create a panel of competent auditors who will conduct the assessments on behalf of DSCI DSCI should establish a mechanism to manage the assessment results including sharing of results with clients and respective Service Providers DSCI should have a Service Provider assessment program that comprises of framework, processes, methodology for the assessment DSCI should have mechanism to review the Service Provider assessments results on a regular basis Code of practices should take a note of existing preparedness and initiatives of Service Providers in the areas of security and privacy Others 11.11% 33.33% 66.67%
66.67%
66.67%
55.56%
55.56%
55.55%
37
Clients perspective
Outcome of Service Provider assessments Data Security
DSCI should provide organization wide security maturity rating DSCI should provide domain specific maturity rating (e.g. Application security maturity rating) DSCI should provide organization wide security certification to Service Providers 77.78%
55.56%
44.44%
Organization-wide security and privacy maturity ratings may be provided as a result of Service Provider assessments
88.89%
44.44%
39
40
Clients perspective
Most suitable assessment process in case DSCI assumes the role of a third party assessor
On receiving request from the client, DSCI conducts the assessment of the targeted Service Provider and shares the report with the client
66.67%
In case DSCI assumes the role of a thirdparty assessor, Client and Service Provider organizations strongly support DSCI conducting the assessments of the targeted Service Provider and sharing the report with the Client on receiving requests from the Client
On receiving request from the Service Provider, DSCI conducts the assessment of the Service Provider and based on the authorization of Service Provider, DSCI shares the report with Service Providers clients On receiving request from the Service Provider, DSCI conducts the assessment of the Service Provider and submits its report to the Service Provider. Service Provider then shares this report with his clients when requested or otherwise Based on DSCI assessments, Service Providers are benchmarked against defined parameters and the report is made public 11.11%
33.33%
11.11%
41
More than forty percent of the Service Provider respondents suggested that in case DSCI assumes the role of a third party assessor, DSCI should conduct the assessment of the targeted Service Provider on receiving request from the Client and share the report with the Client. Same number of Service Provider organizations also supports the process of DSCI conducting the assessment on receiving request from the Service Provider and submitting the report to Service Providers Clients upon authorization.
41.67%
41.67%
33.33%
Others
8.33%
42
Recommendations
The survey revealed some interesting findings and facts, both from Client and Service Provider perspective which were further validated by the secondary research. Based on the study of different assessment frameworks and findings of the survey, following are some of the salient preliminary recommendations for developing a Service Provider Assessment Framework:
DSCI should play a vital role in conducting Service Provider assessments and sharing the outcome in the ecosystem. It should:
Have an Service Provider assessment program that comprises of framework, processes, and methodology for assessments Provide an organization wide security and privacy maturity rating, and domain specific maturity rating that may be shared in the ecosystem after taking the due permission of the Service Providers
A new standard mapped to prevalent standards should be considered as a potential assessment standard for third party assessments of Service Providers DSCI as an industry initiative and a Self Regulatory Organization having representation from both the Client and Service Provider organizations should empanel auditing firms for conducting independent third party assessments. The advantages of prevalent assessment frameworks like adaptability, flexibility, comprehensibility of assessment areas, process-driven, and measurement-based assessment process should be the characteristics of the Service Provider assessment framework that may be developed.
43
The assessment model should not become an overhead for an organization. It should be able to provide specific improvement opportunities that an organization should be able to imbibe. The assessment criteria should be transparent to the extent possible. The framework should be reviewed at least on an annual basis by a competent set of technical and process experts, preferably comprising DSCI members, members from third party assessors, and the industry. The assessment framework should be applicable regardless of size of the organization and nature/ complexity of its processes. For this purpose, the assessment methodology adopted should contain a preliminary set of questions that can be self-assessed by an organization. The Service Provider assessment framework should provide opportunities to organizations for implementing / performing the control activities according to the needs of the organizations specific environment. The framework should follow a process-approach and outline measurable assessment areas. The assessment areas that would link to specific business processes in an organization will be easy to align with overall business goals and objectives. The framework should provide both assessment area/ domain based maturity rating and organization-wide security and privacy maturity rating that summarizes the appraisal results and permit comparison amongst organizations. The assessment model should be easy to comprehend and companies should be able to adopt on their own. All assessment areas should be broken down into a detailed list of specific and measureable steps that are easy to comprehend for assessment purposes.
44
Annexures
The study team analysed the shortlisted assessment frameworks for their advantages and disadvantages when applied to Clientdriven Service Provider assessments. A summary of each framework is provided below:
45
activities according to their organization specific environment. For assessments the framework undertakes a Process based approach thereby adding value to the organization in the process of being assessed for maturity.
46
trusting relationships with other Service Providers, and to meet the dynamic demands of business. The eSCM model provides the organizations the flexibility to choose from framework based (using the framework as best practices) or evaluation based (using the framework to undertake a formal assessment). The eSCM for Client organizations is composed of 95 practices covered under three dimensions Sourcing Life-cycle, Capability Area, and Capability Level.
Assessment Areas/ ease of use by the organization being assessed Assessment Methodology/ scoring pattern Sharing of assessment results Acceptability/ popularity of the framework Independence of examiners Frequency of framework update to cater to future requirements
47
48
The assessment criteria are nonprescriptive and adaptable by any organization regardless of its nature and size. It covers adequate details of the assessment criteria for any company to implement in a manner they desire and yet meet the expectations of the framework. In other words, the framework is not dictative or stringent in nature, thus making it easily usable by any organization. The assessment areas are designed to cover the breadth of the organization which would be an expectation from any good assessment framework for greater adaptability in the industry regardless of the nature or size of the organization. The flip side is that the framework does not provide quantitative requirements for the criteria laid down. The requirements are subjective and hence there are chances that the results when examined by different examiners may not be reproducible. The implementation of the assessment areas is person-dependent even for the organizations applying for the award. In other words, the judgment of the Point of Contact (POC) from the company applying for the award would greatly matter.
The framework is flexible and provides opportunities to organizations for undertaking the activities according to their organization specific environment. The assessment model uses areas of assessment that are process driven namely, process management, project management, engineering, and support. The framework is beneficial in aligning the organizations business processes to business goals and objectives. Apart from specific goals and practices against each Process Area (PA) being assessed, the model contains generic goals and generic practices which an organization is required to meet and implement. This leads to unnecessary redundancies and are a major source of complaints on the standard in its current form. The ease of use of the CMMI model by smaller organizations is a challenge due to its very nature of being long drawn. The model was originally designed for large companies for them to be able to streamline the processes surround the software development and engineering.
Disadvantages
Disadvantages
49
CRISIL rates companies in variety of sectors. Since each sector has its own nuances, CRISIL has customized rating criteria and methodology for each sector to make the ratings exercise apt and meaningful. Therefore, these are easy to use and understand by the organization being assessed. Moreover, the rating criteria are publicly available. CRISIL performs extensive research when finalizing the rating criteria for an industry. The assessment areas are limited to specific risks and issues, and do not help in tying to the overall business goals and objectives of an organization.
The BITS assessment is nonprescriptive and is intended for consideration in conjunction with overall risk map of the organization. The framework serves as a best-practices guide for financial organizations to implement. The guidelines are fairly detailed and easily used/ understandable without the need of a formal training. The BITS shared assessment program has proved to be a boon for the Service Providers because it allows the companies to offset the cost, time and resources which would otherwise be spent in catering to the need of multiple financial companies. The BITS framework does not provide quantitative criteria, and therefore the implementation carried out by organizations using BITS may be difficult to measure or compare against another organization.
The model not just helps in evaluating the capability of Service Providers; sourcing capability of Clients but also helps both, Service Provider and Client organizations to improve their capability across the sourcing lifecycle. The assessment/ capability areas are fairly detailed and serve as best practices guidelines for the organizations to adapt. The model clearly defines the purpose, outcome, team, sponsor, and model scope of the assessment which makes it clear and easy to use by the organization adopting the model. The eSCM model defines capability areas and capability levels in a subjective fashion. The demarcation between different capability levels cannot be measured in absolute terms. The assessment areas are way too many and this may make the implementation/ evaluation process time consuming.
The assessment is specific to a particular instrument/ covenant and the outcome may be used as a comparison factor with other covenants. The assessment is point-intime, and for new securities only. There is no ongoing monitoring of the assessed securities. The assessment model does not draw inference from specific assessments to be able to rate the maturity/ capability of the organization issuing the instrument/ covenant.
Disadvantages
Disadvantages
Disadvantages
Disadvantages
50
Sl. no. 2
The assessment methodology requires a self-assessment by the company applying for the award which assists in dissolving the disparities between small, medium and large sized companies, and the way the control is implemented at the organization. There are weights assigned to each criterion of the framework, which gives the organization a sense of direction on emphasis or focus for implementation/ assessment purposes. The calculations involve an aggregate rather than an average of the evaluation dimensions, thus giving the overall picture more accurately despite being result-focused. The framework lays down the criteria for assessment but it is silent (in other words not transparent) about the method of assessment. Therefore, an organization applying for the award may not know what it missed vis--vis other companies, and why it did not receive the award. The assessment methodology is subjective, and the scoring patterns assigns discrete percentage values to the evaluation dimensions. Which of the percentage would be chosen for a particular company is left to the judgment of the examiner. In the self assessment process, the guidelines for High, Medium, and Low levels of importance are not defined. The highest weights are assigned to the Results criteria and therefore, the framework is very result-focused rather than process focused. A highly result oriented approach may lead to dissolution of importance that processes play in an organization.
The maturity model adopts a process approach, thereby adding value to the organization in the process of being assessed for maturity. An organization being assessed against this model would see increased number of processes under statistical control. The model adopts a measurement based process improvement. Individual processes operating in the company can be assessed or assigned capability levels namely, Optimizing, Quantitatively managed, etc. The assessment model may be complicated for companies to comprehend and adopt on their own. Hence, training by Software Engineering Institute (SEIs) licensed consultants becomes imperative for companies. Consequently, the associated costs and administrative overheads required to undergo and maintain the assessment increases. The model consists of two representations continuous and staged which may lead to conflicting situations at some point during the assessment. The numbering scheme of goals and practices is complicated. Example, goals of a process area are numbered 1 to n. A specific goal of a process area therefore may have the label SG 3 and a generic goal GG5. The practices corresponding to a goal use the index of the related goal and another index 1 to m numbering it as one of the practices of this respective goal. Specific practices also have a capability rating indicated by a dash and the respective capability level. A specific practice therefore may have the label Service Provider 3.3-1 and a generic practice GP 5.1. The rating is subjective to the extent of the examiners understand of the business processes. The framework is rigid as the organization has to complete level 3 before completing level 4.
Disadvantages
Disadvantages
51
The rating methodology adopts a risk-based approach as compared to CMMI that adopts a processbased approach and Malcolm Baldrige that adopts a results- focused approach. The advantage of risk based approach is the prioritization of the organizations activities inline with the risks that matter or high risks. The rating methodologies and criteria are clearly spelt out, published and consistently applied. The rating/ grading scales are also publicly available. CRISIL gives a detailed analytical report on the companys strengths and weaknesses along with the ratings. The report helps in strengthening operations and improving the working of the company. The rating scale used is objective and easily understood.
The framework adopts a riskbased approach and unlike CRISIL, applies to the overall risk map of the company. The assessment is asset driven, helps in improving processes and controls in the organization. A by-product for financial organizations is compliance to the requirements laid down by the Federal Financial Institutions Examination Council (FFIEC). The BITS shared assessment framework is flexible and can be as detailed and as objective driven as required by an organization. There is no scoring pattern or rating guidelines provided to measure the effectiveness of implementation of the BITS framework. In other words, the assessors knowledge, competence and understanding of processes would play a big role in the deciding the final results. While the assessment has resulted in lowering the overall cost of assessments. However, the assessments are fairly detailed and carry on for 3 to 8 weeks. The time and resources that an organization would spend may be difficult to offset for smaller organizations.
The assessment model helps Service Providers to objectively differentiate themselves from the rest. The model provides for two methods of adoption framework-based (which is using the practice area details as best practice guidelines for an organization to implement), and measurement-based (which allows the organization to undertake a formal assessment of its practices and obtain certification). The assessment model contains evaluation types namely mini self-appraisal and mini evaluation which allows the flexibility to an organization to use only a subset of the practice areas.
The framework adopts a risk-based approach, and looks after the needs of the investors. The framework uses a qualitative rating scale of Strong/ good/ weak across a number of parameters associated with the level of protection offered to an investor. The definitions of Good, Strong, etc. levels are objectively defined and easy to understand. The frameworks assessment methodology in assigning a rating is transparent and objective. The assessments are quantitative in nature. The assessment methodology does not relate back to the business goals and objectives of the company issuing it.
Disadvantages
Disadvantages
Disadvantages The model uses five different types of assessment methods, and does not require the organization to undertake a formal assessment by an external party. This may lead to the model being used mainly as a reference guide and lead to non-seriousness in adoption by many companies. The mini evaluation and mini self-appraisal assessment strategies may give comfort over existing controls to the management of the organization without realizing that in reality these may be covering just some aspects of the overall areas of assessments.
Disadvantages
The assessment methodology does not focus on overall strengths, weaknesses, processes operating in the organization, rather specific instruments.
52
Sl. no. 3
An organization, upon receiving the award becomes ineligible to apply for it after for a period of 5 years. This means that the framework allows more and more companies to become eligible to receive the award. The award restriction is 18 in a year and looking at the subjectivity or, non-transparency of the assessment methodology, this number may not be a good picture of the processes implemented by an organization applying for the award. No communication is sent to the organizations whose applications are not selected by the Board of examiners.
The model provides a single maturity rating that summarizes the appraisal results and permits comparisons across and among organizations. The process of sharing results is transparent and the maturity ratings for assessed organizations are publicly available. The assessment model contains a unidirectional mapping between staged and continuous representations. However, if an organization chooses a maturity rating (staged representation); it does not indicate a reverse mapping to the capability levels at which the processes are operating.
Disadvantages
Disadvantages
Advantages
Advantages
The results are conferred by the highest office in the United States which helps in making the framework widely acceptable. The framework is acclaimed by the President of the country, thus making it popular and trusted. The framework is promoted by publicprivate partnership, thus making it more acceptable. The flip side of the popularity is the bureaucracy and red-tapism that the adoption of the assessment framework may bring with it. The value brought by the assessment framework to the organization while going through the assessment process cannot be measured or, is not visible. It does not influence or change the way things happen in an organization. Thus, many organizations may construe it as an overhead of documentation rather than being able to use it to bring about improvements in the organization.
While going through the assessment process, the organization sees direct benefits / improvements being brought about in the organization. Thus, the assessment process is not seen as an overhead but something that adds value. The bodies of knowledge captured in CMMI models are only available for systems engineering, software engineering, product/ process development, supplier sourcing. Consequently, the maturity model is system/ software focused, and as such its tenets may not apply to industry sectors other than Services/ Technology/ Hardware.
Disadvantages
Disadvantages
53
The ratings are under constant surveillance. They are revised as and when circumstances so warrant, and disseminated through the news media. A rating history is maintained for a period of 5 years which gives an idea of an organizations progression/ regression in the risks that matter. The outcome of the CRISIL ratings are issued in the form of an opinion on specific instruments, which does not have a fixed format that highlights what are the positives versus the negatives. Therefore, the results of the rating exercise cannot be directly used to improve on processes in the organization. Relating the opinion issued back to the business goals, objectives and processes may be a tedious task.
The outcome of the assessment is an Agreed Upon Procedures report which the Service Provider is free to share with all its Clients or outsourcers. The outcome of the assessment also includes a remediation plan in-line with the requirements of the organization. There is no maturity rating or score provided to the organization that can be used to compare the companys practices to other organizations.
The outcome of each assessment method is clearly defined. The outcome of the assessment gives the interested parties the comfort that the sourcing capabilities are indeed working fine. The outcome of the assessment may be used as a differentiating factor by the organization thereby supporting the outsourcing process.
The outcome of each assessment is a rating assigned to a covenant which is easy to understand basis the guiding factors issued. The results are available in the form of opinion and do not always brings out the positives.
Disadvantages
Disadvantages
Disadvantages
Disadvantages The model focuses on capabilities and does not give a clear indication of the maturity of those processes.
Advantages
Advantages
Advantages
Advantages
The CRISIL ratings are widely acceptable owing to the strong independence, right amount of confidentiality, and transparency of the final ratings. None noted.
The assessment is popular among financial institutions and helps them regulate the environment they operate in. The BITS shared assessment program is not popular among industries other than financial.
The models acceptability may be reasonable in the outsourcers-outsourcing community. The model may not gain as much popularity because it does not force a formal evaluation or certification. The model may not be very popular because of the cost, time and resources required to be spent in reading, interpreting, implementing, and then evaluating (if need be) the detailed requirements.
Disadvantages
Disadvantages
The objective and quantitative nature of the assessment framework makes it popular and acceptable in the community. The framework cannot be extended to other industries.
Disadvantages
Disadvantages
54
Sl. no. 5
The Panel of judges for the award work without compensation and nothing except limited travel expenses are reimbursed to them, which serves s a step towards ensuring unbiased assessments. The Panel chairperson is reviewed every year by the Secretary of Commerce in the US. Panel member has a chance of serving a term of 6 years at a stretch, and yet continue to work with the company to which they belong. Hence, the independence of assessment process may be jeopardized. There are no licensed personnel made available for spreading training and awareness on the framework.
The Software Engineering Institute (SEI) provides a list of licensed consultants who can provide appraisal/ training services. None noted.
Disadvantages
Disadvantages
Advantages
Advantages
Over the period time, the framework has matured to keep in mind the future and dynamism associated with the marketplace, and is therefore now slated for a review once every two years. None noted.
The assessment framework has matured from the time of its inception, and released guidelines for implementation, and list of licensed consultants, since then. The framework does not have a set frequency of review/ revision.
Disadvantages
Disadvantages
55
The rating committee comprises members who have the professional competence to meaningfully assess the credit analysis that underlies the rating, and have no concern with the company being rated. This makes the rating highly independent. None noted.
There are a set of listed companies that conduct the assessment of organizations. None noted.
Disadvantages
The assessors are trained and certified by Carnegie Mellon Universitys IT Services Qualification Center (ITSqc). Apart from the universitys ITSqc, the success of the model depends largely on the industry adopting it. Therefore, independence may be construed as an issue.
The assessment makes use of analytics software which gives a sense of reliability to the assessees. In case the software solutions were not to work, the people trained in carrying out the assessment may not be available.
Disadvantages
Disadvantages
Disadvantages
Advantages
Advantages
Advantages
Advantages
The ratings provide both short-term and long-term ratings, and the rating are re-evaluated as and when required. CRISIL evaluates its rating criteria, methodologies, and procedures on a regular basis; and modifies or enhances them as necessary to respond to the needs of the markets. There is no specific frequency of review/ revision of CRISIL rating methodology.
The Agreed Upon Procedures (AUPs) are updated at least once annually. None noted.
None noted.
None noted.
Disadvantages
Disadvantages
Disadvantages
The eSCM model does not have a fixed frequency for revision. It was last updated in the year 2004.
Disadvantages
56
Glossary
BPO Client organization or Client CMMI COBIT COSO DSF GLBA HIPAA ISO 27001 IT ITIL NIST PCI DSS Business Process Outsourcing Organizations that outsource work to Service Providers Capability Maturity Model Integration Control Objectives for Information and related Technology that provide a set of best practices for Information Technology management Committee of Sponsoring Organizations DSCI Security Framework Gramm-Leach-Bliley Act Health Insurance Portability and Accountability Act An Information Security Management System standard published in October 2005 by International Organization for Standardization (ISO) and the International Electrotechnical Commission (IEC) Information Technology Information Technology Infrastructure Library is a set of concepts and practices for Information Technology Services Management (ITSM), Information Technology (IT) development and IT operations. National Institute of Standards and Technology PCI DSS or Payment Card Industry Data Security Standard is a worldwide information security standard defined by the Payment Card Industry Security Standards Council. The standard was created to help payment card industry organizations that process card payments prevent credit card fraud through increased controls around data and its exposure to compromise. Statement on Auditing Standards No. 70 Organization that work for Client organizations in an outsourcing model
References
Malcolm Baldrige Framework: [Link] Capability Maturity Model Integration (CMMI): [Link] Crisil rating methodology: [Link] BITS framework for managing technology risks for IT service provider relationships: [Link]
Carnegie Mellon University-e Sourcing Capability Model: [Link] [Link] Moodys Working Paper: [Link] DSCI Security Framework: [Link]
57
50%
This publication contains information in summary form and is therefore intended for general guidance only. It is not intended to be a substitute for detailed research or the exercise of professional judgment. Neither EYGM Limited nor any other member of the global Ernst & Young organization can accept any responsibility for loss occasioned to any person acting or refraining from action as a result of any material in this publication. On any specific matter, reference should be made to the appropriate advisor.
L: Niryat Bhawan, 3rd Floor, Rao Tula Ram Marg, New Delhi-110057, India P: +91-11-26155071 | F: +91-11-26155070 | E: info@[Link] | W: [Link]