Part of selling GSI's managed services is dealing with the competitive comparison issue, as prospective clients attempt to determine which managed service provider is the best fit for their IT management needs. The need to compare GSI versus our competition is understandable, but what do you do when there is little to compare us against?

The old and overused idiom, "Apples to Oranges" comes to mind, but does not accurately describe the situation. GSI is most often compared to "hosting" companies because we have hosting capabilities, but to say we are just a hosting company is grossly inaccurate. GSI is a managed services company with hosting capabilities. We provide the entire service – soup to nuts. Most competitors, while they claim to offer a total solution, barely get to the soup.

Maybe a metaphor using the auto repair industry can help clarify our situation as it relates to comparison shopping.

• Space. Every auto repair shop must have a garage in which to put the equipment and vehicles involved. Much like IT hosting companies must have data centers. It is a given that one garage looks much like another. Picking an auto repair shop solely on the presence of a garage would not be prudent. The need for them to have a workable and available garage is a requirement, but you need more criteria to narrow down your comparison. The same goes for hosting companies and their data centers.
• Range of services. Another area of comparison is the type of auto repair services and the expertise with which they are provided. A lot of shops provide standard services and have qualified technicians to provide those standard services on the most common vehicles. These services work fine for a standard high-production vehicle. What happens if your vehicle is a specialized, custom-built unit? The number of auto repair shops that can provide you services drastically decreases. Your selection now depends on the unique traits of the specialized services you require. This applies to IT services, as well. IT systems requiring advanced security and regulatory compliance necessitate specialized services, and narrow your service provider options.
• Who performs the actual work? Let's muddy the waters a bit and carry the metaphor further. Say you have narrowed your auto repair provider selection down to three shops. They all three advertise knowledge of your specialized vehicle. Each shop defines their services using similar terms. Even so, as you dig into one shop's methods, you find that they provide work space and tools, but do not actually do the work. You and your brother-in-law will need to be available to actually perform any work. This kind of discrepancy can be very difficult to discover when picking an IT services provider, and it is painful to realize it after contracts have been signed with an auditor breathing down your neck.
• Do they need direction from you? So, now you are down to two shops that may be able to perform work on your specialized vehicle. As you question the two, you again find a deficiency in one of them. They have the capabilities, the tools and the space, but they can only do exactly what you tell them to. They cannot or will not assist with designing your solution, and cannot point out issues you will encounter, much less account for them. This, again, puts you in a hot seat. Your services contract has turned into space and tools for you to implement.
• Finally ... the needle in the haystack. The last shop you review meets all the requirements and represents the complete solution. Almost like having a dedicated race crew to work on your specialized vehicle. Clearly this is the best choice for the services you need, but it was not easy to identify them. There were not many clues to lead you to them. All the same terminology was used to describe the services, and all three shops were giving assurances that you would get the services you expect.

There are only a small handful of managed security service providers that actually take on the specialized tasks for regulatory compliance, yet you will find many claiming to do so. They all use the same language to describe what they provide, but many simply provide the tools. Since most regulatory requirements include an analysis component, any service provider that is not doing analysis along with data collection will not meet the requirements. Yet they will advertise a fully compliant service offering by assuming the client will spend the time necessary to cover the analysis needs.

When GSI states that a service we provide can meet a requirement, we mean that the service fully meets the requirement. Just as if a full IT team were hired on. The challenge is educating prospective clients on the difference when being compared to providers with less-developed ideas about services.

When a company fails to show even basic security capabilities, causing a breach, a knowledgeable public takes their business elsewhere.

An interesting site to read through is the Ponemon Institute, which includes independent research on privacy and IT security issues.

One of studies that caught my eye involved the cost of data breach.

The study I reviewed dealt with data from 2008, and compared the data to 2007. It also reviewed the impact of both direct and indirect costs. Direct costs are the efforts taken to remediate the direct causes and cleanup of a breach. Examples would be: hiring consultants, implementing technologies, or providing credit protection. Indirect costs tend to be costs to business that are not directly attributable to a breach, yet still feel the impact of a breach. Examples would be: customer churn, lost prospects, reputational impacts, or additional support costs.

The overall result of the study shows that direct and indirect costs are rising. This reflects the vast amount of attention that has been given to breaches -- even as far as state governments legislating how the breach victims must be notified. As companies involved in data breaches learn the proper ways to handle the event, the costs will rise due to their increased engagement. In the past, a breach could be handled internally with little external involvement. No longer is this the case. A company with a breach had better be able to show due diligence by engaging professionals to remediate.

Additionally, the general public, which is increasingly plugged in and online, has become savvier regarding how their data should be handled and protected. When a company fails to show even basic security capabilities, causing a breach, a knowledgeable public takes their business elsewhere.

Something to note is that customer churn rates due to a breach event were highest in the healthcare industry. It seems folks care more about their doctors/hospitals losing their information than they do their financial institutions.

One of the most relevant discoveries in this report is that breaches involving third-party or partner mistakes are the most expensive to remediate, and that 44% of the breaches reviewed involved third-parties. There is not really a good reason as to why third-party involvement causes a breach to be more expensive, but we can guess that inefficiencies are introduced when a third-party must be taken into account. The lesson is to only share data with third-parties you are sure have a good security program that includes event handling capabilities.

Below are some statements directly from the report that I found relevant. These do not represent the full report, and an interested person should read directly from the links provided below.

Over the past four years, lost business cost component grew by more than $64 on a per-victim basis, or a 38% overall percentage increase. Our research finds organizations in highly trusted industries, such as banking, pharmaceuticals and healthcare, are more likely to experience a data breach with high abnormal churn rates. In contrast, retailers and companies with less direct consumer contact seem to experience a lower overall data breach cost.

The most significant cost decrease concerns ex-post response, which implies organizations are becoming more cost-efficient in their management of the data breach. Despite efficiency gains, consulting, legal defense and, as mentioned previously, lost customer business have increased in this year’s study.

The range of total cost among the 43 data breach incidents contained in this year’s study is a minimum of $613k to more than $32 million. The magnitude of the breach event ranged from 4,200 to 113,000 lost or stolen records. As in prior years, data breach cost appears to be linearly related to the size or magnitude of the breach event.

In this year’s study, average abnormal churn rates across all 43 incidents is 3.6%, which was measured by the loss of customers who were directly affected by the data breach event (i.e., typically those receiving notification). The abnormal churn or turnover rate in 2007 for customers receiving notification was 2.7%.

Healthcare and financial service companies have the highest average rate of churn at 6.5% and 5.5%, respectively. High churn rates reflect the fact that these industries manage and collect consumers’ most sensitive data. Thus, consumers may have a higher expectation for the protection and privacy of their financial and healthcare records.

Over 44% of all cases in this year’s study involved third-party mistakes or flubs. Data breaches involving outsourced data to third parties are the most costly. This could be due to additional investigation and consulting fees. As shown in Bar Chart 5, per victim cost for data breaches involving third parties is $231 versus $179, more than a $52 difference.

Further Reading

• Ponemon Institute
• Ponemon Institute's Fourth Annual US Cost of Data Breach Study (PDF)

PCI DSS requirement 11.3.1 indicates the need for annual network penetration testing, which includes an "internal" penetration test by an experienced security tester. GSI has always supported this activity by working with whomever our clients ask to perform the testing. This requirement can be met by using experienced internal staff or third-party professionals (see 11.3 Supplement), and we have seen both. Even though the selection of who does the testing is easy to make, the location in the network from which to perform the internal testing is another matter.

The environments GSI manages at PCI levels of security are particularly isolated with strong two-factor controls required for all administrative entry points. These environments are characterized by strict firewall change-control procedures and system hardening with accompanying audits. Another feature of these secure environments is the complete lack of desktop-level systems. Environments meant to house highly secure single-function payment processing systems do not require the abundance of services seen in systems that support desktop access for users. There are not any personal directory shares, email clients, productivity packages, or Internet user communication technologies present on these systems.

All of this culminates into a distinct lack of what the PCI DSS would determine to be an "internal" network.

So where does that leave us regarding internal penetration testing for requirement 11.3?

There are two scenarios that can play out:

One is for the client tester to be placed directly into the cardholder environment. This type of test assumes the attacker has compromised at least one system in the environment. The result of this test reflects how much information could be accessed post-compromise. Many QSAs will accept this as a proper internal test of the environment even though it is not strictly adhering to the intent of 11.3. In my opinion, it is not a valid test of risk exposure for the environment, because it completely disregards the security protections it is meant to test.

The second scenario is for the client's QSA to give them a "pass" on the internal penetration test requirement with the knowledge that, by definition, an internal penetration test is not possible. There is not an internal network to test from. We are starting to see more of this as QSAs better understand how the environments are managed at GSI. This decision is not arrived at lightly and even when the QSA gives a pass for an internal penetration test, it is only after the environment meets specific criteria. We must prove that administrative access is strictly protected by two-factor authentication and VPN connectivity. We also must show that there are not any desktop networks directly connected to access shares or services. At GSI, this is easily done with configuration diagrams and firewall configurations.

I have stated in a previous article that the PCI standards will inevitably lead to risk-based implementations. The ability for highly protected environments to forgo an internal penetration test is an excellent example of how a QSA can take the risk approach into their own hands and utilize the PCI standard in a flexible way to meet the intent, save clients money, and concentrate security where needed -- all at the same time.

Briefly: What is a Compensating Control?

The Payment Card Industry Data Security Standard (PCI DSS) roughly defines compensating controls as a method to meet the intent of a DSS requirement, while not implementing the control as written by the PCI Security Standards Council. This was a smart move on the part of the Council. They are saying that the prescriptive requirements may not be the best way for all situations and allow implementers to work outside the box, as long as the intent of the standard is being met.

Compensating controls have become necessary for situations where the combination of technical and business constraints prevent a control from being implemented. The typical reason for implementing compensating controls is the inherent expense involved with implementing the original DSS controls. This is not surprising considering the requirements for such things as automatic access control management, centralized logging, integrity monitoring, and all the vulnerability management technologies.

QSA's Perspective on Compensating Controls

QSAs (Qualified Security Assessors) really dislike compensating controls. It has been my experience that the dislike is due to the additional effort required to report compensating controls. For a QSA, the DSS is a list of requirements for which they test the controls. A compensating control does not have a clean pass/fail and must be fully documented in the Report on Compliance (RoC) in such a way to fully explain how it meets the intent of the requirement it replaces. Further, the QSA cannot simply use documentation provided by the merchant/service provider. The QSA must fully understand the new control so that they can make a judgment call as to whether it meets the original intent, as well as document it for the RoC. An assessment that involves multiple compensating controls will drag out much longer than one without them. Many assessment engagements use a fixed project pricing method, which means the longer an assessment takes, the thinner the profit margin for the assessment company.

Managed Hosting Impact on Compensating Controls

Compensating controls all by themselves are a wrench in the PCI DSS assessment process. Add in a third-party managed hosting provider and things get real sticky. Especially if the hosting provider does not fully support the PCI requirements, leaving the client to dig details out of a set of offsite tools that only partially describe an environment. A managed hosting provider utilized for systems requiring PCI DSS compliance will need a capability to deal with compensating controls. It means providing reasonable customization with the expertise to understand how that customization will affect a PCI DSS assessment.

Being a managed services provider, GSI must deal with the engineering, documenting, and implementation of compensating controls for client PCI DSS environments. It is not easy and there are many challenges. Any customization that pushes the boundaries of our standard operating procedures risks failure, and we must be vigilant with our audits and reporting to catch any discrepancies. Still, without doubt, it is worth having the capability to do it. Having that flexibility really tells the story when our clients continually pass PCI DSS assessments year after year.

The week of Sept 21st, I attended the PCI Council Standards Training and Community Meeting in Las Vegas. The training was offered after merchants and service providers asked for more education on the PCI standards and how the assessors are being directed. The training was a positive experience for me. It showed me that, compared to many other organizations, we (GSI) have a greater understanding of standards implementation, and how to be successful and compliant. It was clear by most of the questions being asked that the basics of the standards are still a major challenge for many.

It was also clear that some assessors (QSA) are still not understanding the intent of the standards. Many folks had questions related to assessors that were applying prescriptive standards elements to situations that they were not intended to address. This was evident in the type of questions, such as, "What defines a firewall?" and, "What is network access?" Assessments that are executed properly do not generate these types of basic questions. The best part of the training was interaction between the students. Those with more experience were able to share our thoughts, which, combined with the instructor's input, really helped the whole class.

The standards training went for two days with the Community Meeting beginning on the third day. Just as last year, the meetings were interspersed with many personal networking opportunities and vendor expo time.

This year's meeting occurred during the feedback period of the Standard's cycle, which meant that the general attendance had many opportunities to directly address the council. I did not do a formal review of feedback, but it seemed that the most popular topic was that of risk. Even as prescriptive as the PCI standard is, it still presents a checklist of items that MUST be completed to be compliant. Alternatively, a risk-based approach would allow the controls of the standard to be applied when and where they would have the most impact on threat. The Council's responses to this feedback indicated that they realize the issue and will be considering steps to make the standard more risk-based.

A risk-based approach would be good for everyone, but could be remarkably difficult for those without a strong security program already in place. The difficulty is primarily related to assessing the risk. It is a specific exercise that will generate different results for each environment, and dependence on assessor judgment will increase. Those that already use a risk-based approach would find a risk-based security standard a great improvement over the current prescriptive checklist format.

Engaging a third party to manage the controls, monitoring, and maintenance that secure a company's most important data assets can seem counter to the security of that data. I would venture that the opposite is actually true.

The employees a company hires are typically selected and trained for their productive abilities that have little to do with securing data. Even in IT, the technicians are judged first on their technology know-how and project experience, while security knowledge is either not considered or to be gained after hire, if at all.

Employees are selected and rewarded based on a "productive mindset" to achieve goals in the quickest and most effective method available, especially when teams are small and bringing innovative products to market quickly is the main goal. This mindset tends to disregard, or at least, fails to account for security and compliance measures. Teaching and instilling the practices required to obtain a "security mindset" takes time and resources, which are rarely available nor quickly obtained. For example, a productivity-based team can successfully bring an online product and e-store to market quickly, but would it also manage regular server updates and management of a web application firewall to protect the underlying systems? Probably not and understandably; it is not their expertise and anyone who has spent time in IT security consulting can confirm that a poorly executed or non-existent security program is as close as the nearest business.

Because of these issues, engaging with an external data security management service becomes a responsible business management decision. Handing the various security controls, audits, monitoring and patching to people trained for the purpose and with a "security mindset" allows companies to concentrate their talent where it is needed...building better products. And when the company is innovative, requiring agile thinkers using fast development models, taking advantage of the built-in security practices provided by a managed data security provider allows it to maximize the positive aspects of that model, while reducing the security risks imposed by it.

Further Reading

Security in a Reputation Economy - Bruce Schneier