PCI DSS requirement 11.3.1 indicates the need for annual network penetration testing, which includes an "internal" penetration test by an experienced security tester. GSI has always supported this activity by working with whomever our clients ask to perform the testing. This requirement can be met by using experienced internal staff or third-party professionals (see 11.3 Supplement), and we have seen both. Even though the selection of who does the testing is easy to make, the location in the network from which to perform the internal testing is another matter.

The environments GSI manages at PCI levels of security are particularly isolated with strong two-factor controls required for all administrative entry points. These environments are characterized by strict firewall change-control procedures and system hardening with accompanying audits. Another feature of these secure environments is the complete lack of desktop-level systems. Environments meant to house highly secure single-function payment processing systems do not require the abundance of services seen in systems that support desktop access for users. There are not any personal directory shares, email clients, productivity packages, or Internet user communication technologies present on these systems.

All of this culminates into a distinct lack of what the PCI DSS would determine to be an "internal" network.

So where does that leave us regarding internal penetration testing for requirement 11.3?

There are two scenarios that can play out:

One is for the client tester to be placed directly into the cardholder environment. This type of test assumes the attacker has compromised at least one system in the environment. The result of this test reflects how much information could be accessed post-compromise. Many QSAs will accept this as a proper internal test of the environment even though it is not strictly adhering to the intent of 11.3. In my opinion, it is not a valid test of risk exposure for the environment, because it completely disregards the security protections it is meant to test.

The second scenario is for the client's QSA to give them a "pass" on the internal penetration test requirement with the knowledge that, by definition, an internal penetration test is not possible. There is not an internal network to test from. We are starting to see more of this as QSAs better understand how the environments are managed at GSI. This decision is not arrived at lightly and even when the QSA gives a pass for an internal penetration test, it is only after the environment meets specific criteria. We must prove that administrative access is strictly protected by two-factor authentication and VPN connectivity. We also must show that there are not any desktop networks directly connected to access shares or services. At GSI, this is easily done with configuration diagrams and firewall configurations.

I have stated in a previous article that the PCI standards will inevitably lead to risk-based implementations. The ability for highly protected environments to forgo an internal penetration test is an excellent example of how a QSA can take the risk approach into their own hands and utilize the PCI standard in a flexible way to meet the intent, save clients money, and concentrate security where needed -- all at the same time.

Briefly: What is a Compensating Control?

The Payment Card Industry Data Security Standard (PCI DSS) roughly defines compensating controls as a method to meet the intent of a DSS requirement, while not implementing the control as written by the PCI Security Standards Council. This was a smart move on the part of the Council. They are saying that the prescriptive requirements may not be the best way for all situations and allow implementers to work outside the box, as long as the intent of the standard is being met.

Compensating controls have become necessary for situations where the combination of technical and business constraints prevent a control from being implemented. The typical reason for implementing compensating controls is the inherent expense involved with implementing the original DSS controls. This is not surprising considering the requirements for such things as automatic access control management, centralized logging, integrity monitoring, and all the vulnerability management technologies.

QSA's Perspective on Compensating Controls

QSAs (Qualified Security Assessors) really dislike compensating controls. It has been my experience that the dislike is due to the additional effort required to report compensating controls. For a QSA, the DSS is a list of requirements for which they test the controls. A compensating control does not have a clean pass/fail and must be fully documented in the Report on Compliance (RoC) in such a way to fully explain how it meets the intent of the requirement it replaces. Further, the QSA cannot simply use documentation provided by the merchant/service provider. The QSA must fully understand the new control so that they can make a judgment call as to whether it meets the original intent, as well as document it for the RoC. An assessment that involves multiple compensating controls will drag out much longer than one without them. Many assessment engagements use a fixed project pricing method, which means the longer an assessment takes, the thinner the profit margin for the assessment company.

Managed Hosting Impact on Compensating Controls

Compensating controls all by themselves are a wrench in the PCI DSS assessment process. Add in a third-party managed hosting provider and things get real sticky. Especially if the hosting provider does not fully support the PCI requirements, leaving the client to dig details out of a set of offsite tools that only partially describe an environment. A managed hosting provider utilized for systems requiring PCI DSS compliance will need a capability to deal with compensating controls. It means providing reasonable customization with the expertise to understand how that customization will affect a PCI DSS assessment.

Being a managed services provider, GSI must deal with the engineering, documenting, and implementation of compensating controls for client PCI DSS environments. It is not easy and there are many challenges. Any customization that pushes the boundaries of our standard operating procedures risks failure, and we must be vigilant with our audits and reporting to catch any discrepancies. Still, without doubt, it is worth having the capability to do it. Having that flexibility really tells the story when our clients continually pass PCI DSS assessments year after year.

The week of Sept 21st, I attended the PCI Council Standards Training and Community Meeting in Las Vegas. The training was offered after merchants and service providers asked for more education on the PCI standards and how the assessors are being directed. The training was a positive experience for me. It showed me that, compared to many other organizations, we (GSI) have a greater understanding of standards implementation, and how to be successful and compliant. It was clear by most of the questions being asked that the basics of the standards are still a major challenge for many.

It was also clear that some assessors (QSA) are still not understanding the intent of the standards. Many folks had questions related to assessors that were applying prescriptive standards elements to situations that they were not intended to address. This was evident in the type of questions, such as, "What defines a firewall?" and, "What is network access?" Assessments that are executed properly do not generate these types of basic questions. The best part of the training was interaction between the students. Those with more experience were able to share our thoughts, which, combined with the instructor's input, really helped the whole class.

The standards training went for two days with the Community Meeting beginning on the third day. Just as last year, the meetings were interspersed with many personal networking opportunities and vendor expo time.

This year's meeting occurred during the feedback period of the Standard's cycle, which meant that the general attendance had many opportunities to directly address the council. I did not do a formal review of feedback, but it seemed that the most popular topic was that of risk. Even as prescriptive as the PCI standard is, it still presents a checklist of items that MUST be completed to be compliant. Alternatively, a risk-based approach would allow the controls of the standard to be applied when and where they would have the most impact on threat. The Council's responses to this feedback indicated that they realize the issue and will be considering steps to make the standard more risk-based.

A risk-based approach would be good for everyone, but could be remarkably difficult for those without a strong security program already in place. The difficulty is primarily related to assessing the risk. It is a specific exercise that will generate different results for each environment, and dependence on assessor judgment will increase. Those that already use a risk-based approach would find a risk-based security standard a great improvement over the current prescriptive checklist format.

Several of the big names in the hosting and payments business have released "solutions" that offer PCI-DSS relief to Level 4 merchants (small businesses with small transaction volumes). Well, after reviewing almost 20 offerings, I can readily summarize this as "all hat, no cattle." A bunch of hot air.

One major hosting provider recommends that merchants just don't handle credit cards. Seriously? But what about the MILLIONS of merchants that have custom-coded shopping carts, ERP systems, and business POS tools that rely upon the back-end databases that hold their client and payment information? What about businesses that retain card data for recurrent payments?

IMHO, these folks are trying to pull the bait-n-switch on a relatively unsophisticated (from an IT capabilities perspective) group. 'Hey, host in our "cloud" and follow our recommendations (or at least think you are following them), and you can be PCI-DSS compliant.' Wrong, that is PCI-DSS avoidance. Not really an option for millions of businesses. 

These businesses need a REAL PCI-DSS compliant way to economically host their systems. We all know the hoops that a TRUE, VALIDATED, MANAGED PCI-DSS solution will require. It won't be cheap (no more $59 per month hosting with no firewalls!). But, there will be solutions on the market that will uphold even the most vigorous QSA audit, or even a REAL, HONEST Level 4 SAQ. Stay tuned for more from GSI!

Are you a skeptic in your passion for all things cloudy? Companies buying and selling cloud-based services have had to deal with an attitude shift to "where is my data?" fear and loathing. This angst is understandable, especially when protecting data that can make or break your company's reputation, business transactions, or even worse, data that potentially belongs to your clients.

GSI's complex managed hosting and PCI-DSS compliant hosting clients simply wouldn't/couldn't fathom not knowing where the bits live, and EXACTLY how we were protecting them, feeding them and watching over them every minute of every day. We went for true enterprise virtualization when we built our Matrix virtualization/cloud offering. Tools like VMWare Enterprise, NetApp Storage, Cisco and Force10 Networking, Dell/AMD Servers, etc., have proven themselves over and over again both in our shop and in nearly every IT organization we meet.

If one of our clients wants to know where their data is, we can point it out, both logically and physically. We can explain the security protocols, lockdowns, and testing that is performed to ensure the integrity of their data. We show them our PCI-DSS audit of our Matrix Virtualization platform and enable them to sleep at night knowing that the cloud won't "swallow" their corporate or client data.

We wanted to make sure that our cloud was anything but secret sauce. It's mustard, ketchup and some pickle relish (actually Enterprise Pickle Relish 2.1c).

<analogy alert> I see hosting in the cloud a bit like taking my eight-year-old niece and nephew on a trip to the mall. Most cloud offerings would say, "Go ahead, just drop them off and let them run around our semi-structured environment." Well, besides that fact that there are occasionally a few undesirable people at the mall, I just don't trust that my precious assets will be safe. When I ask for details and get "non-specific" answers on their security, activities, whereabouts and planned activities, I am definitely not going to let them have the run of the place.

However, I have no problem dropping them off at their school, because I know there is a structured security plan, trained professionals that cater to their needs, and at any time, if I need to, I can identify where they are and who they are playing with at recess.

GSI has been offering PCI-DSS compliant virtual machines for a couple of years already. We started by virtualizing environments for individual clients, and in January 2009, we announced our Matrix Virtualization platform, which has been fully validated as part of both our SAS70-Type II and PCI-DSS compliance assessments (yes, we are a PCI-DSS Level 1 Service Provider). PCI in the cloud can be done, and is being done.

Further Reading

GSI's Matrix Virtualization Platform

This is the first of three postings I will share pertaining to change management.

The importance of change management for the stable operation of data support systems cannot be denied. Data center operations literature, such as ITIL guidelines, have change management looming large as part of any solution. One of the metrics I have heard is that 80% of system outages can be attributed to a change event.

As a hosting services company, GSI takes change to any environment very seriously. Controlling that change can, literally, affect our bottom line. Even so, change management is one of the "behind-the-scenes" services and processes that is rarely advertised directly or attributed with value. A system outage will bring attention to a lack of change management, but never is change management mentioned when it prevents outages. Much like a utility provider being discussed when a home's power is on, but let the power go out and the first discussion is about the provider.

Since becoming Director of Security and Compliance at GSI, I have become familiar with GSI's role in securing our client's data. I realize that change management and the well-designed implementation of it, are a critical piece of my security software suite. Of course, the PCI DSS includes a requirement (6.4) for change management, but it could be more emphasized and has missed the true security value.

How important is implementation?

I will take a few lines here to briefly discuss GSI's implementation of our change management methods. The implementation, in many ways, is as important as the use of the result.

Who
Any operational staff or client can generate a change request. It is very important that accessibility to the change management system be open. Restricting it to special access would prevent its usage which defeats the purpose of its existence. Power to approve changes is purposefully kept to a small number of management staff, but submitting a change is open.

When
Everyday. A selected team at GSI reviews change requests every day. An hour of every day is set aside by operations managers and me to review change requests for approval. It can be tedious given that any client or operational staff member can submit requests, but the value of discussing, understanding and debating changes is well worth it. Scheduling the review for every day allows us to granularly adjust the amount of impact in data systems due to change. It also supports security efforts.

What
All hosts. The structure and restrictions for controlling change are applied to data systems with regards to their criticality. The change management support systems and procedures are built to accommodate that need. This flexibility allows GSI to apply levels of change management even when tolerance for the control is very low. Compliant systems are held to strict standards of control, and the procedure provides that capability, as well.

Obviously there is more to the entire system than described here, but you should get the notion that implementation must be robust.

Next week, I'll share another article where I talk about the role of change management in dealing with security event volume.

Further Reading

Change Management: Event Volume - Part 2 of 3
Change Management: Identifying Unauthorized Changes - Part 3 of 3