The PCI DSS assessment: what a QSA tests, and how it tests it

What a QSA is actually testing

The PCI DSS assessment answers one question: can a Qualified Security Assessor, working to the testing procedures the PCI Security Standards Council publishes with each requirement, find evidence that every applicable requirement is met across the whole cardholder data environment? Each requirement carries its own testing procedures, and they tell the assessor which methods to use: examine a document, interview a person, observe a process, or inspect a configuration or a system output. It works through the requirements one by one and records what it tested, saw and concluded.

A missing record is not a matter of opinion, and a quarterly scan either passed or it did not, so preparation means evidence that already exists in the form the testing procedure expects. Whether you need a ROC at all, or can validate with a self-assessment questionnaire, is covered in PCI DSS validation step by step: ROC, SAQ and AOC; this guide assumes a QSA is coming.

Before the assessor arrives: scope confirmation and document review

Confirming the scope

You present your own annual scope confirmation: the card data flows, the network diagrams, the inventory of system components, the locations and the third-party service providers. The QSA then challenges it. Does card data really stop where the diagram says? Are the jump hosts, logging servers and directory services that can affect the CDE counted as connected systems? Card data found outside the boundary grows the scope, and the fieldwork, on the spot. How to draw that boundary well in the first place is the subject of PCI DSS scoping and segmentation: shrinking the CDE.

Reviewing the documents

Most QSA companies ask for the document set several weeks before fieldwork: policies, configuration standards, targeted risk analyses, the incident response plan, the roles and responsibilities, the service provider register with each provider's AOC and responsibility matrix, and a year of periodic evidence such as scan reports, penetration test results and access reviews. The assessor reads them against the testing procedures and arrives with a list of what to verify in person.

How the assessor tests: interviews, observation and system inspection

Most requirements need more than one of these methods.

• Interviews. The people who run a control explain how it works, from the firewall rule review to how an account is removed, and the answers are compared with the documents.
• Observation. The assessor watches controls operate: a visitor badged in, a terminal inspected, a card number taken over the phone.
• Configuration and system testing. The assessor inspects settings and outputs on sampled systems: running services, password and lockout parameters, MFA, encryption protocols, anti-malware, time synchronisation and logging.
• Record examination. Change tickets, access approvals and periodic task evidence are sampled across the period.

When the policy, the person and the system tell the same story, a requirement closes quickly; when they disagree, the assessor keeps digging.

Sampling and segmentation verification

A QSA does not inspect every server, workstation or store. It selects a sample of system components and of business facilities, and it records how the sample was chosen and why it represents the population. A sample only works if the population is controlled the same way: standard, centrally managed builds justify a smaller sample, while hand-built systems or regions with their own processes force a wider one, and each exception found raises the question of how many more exist.

Segmentation gets particular attention because it is what keeps the rest of your network out of scope. The assessor examines the controls that separate the CDE from other networks and reviews the segmentation penetration test results required under requirement 11. If the assessor cannot confirm that segmentation works, the networks it was meant to exclude come back into scope.

Findings in the ROC, and fixing things during the assessment

For each requirement the assessor records one finding in the Report on Compliance.

Remediation during fieldwork is normal and expected, within limits. A rule without a justification or a leaver's active account can be fixed while the assessor is there; the assessor then decides whether the requirement is met. What you cannot fix is history. A missed quarterly scan or a skipped access review cannot be produced after the fact. Under version 4 reporting, the Council removed the separate finding that once flagged remediation and introduced a worksheet for assessors to note areas needing improvement, so ask your QSA how it records the fixes it verified.

Where a requirement cannot be met exactly as written, a compensating control or the customised approach may apply; both need preparation well before fieldwork and are explained in PCI DSS compensating controls and the customised approach.

Who attends, and an example assessment week

The assessor will ask for the executive sponsor, a compliance coordinator who owns the evidence list and the schedule, network and system administrators, security operations, developers and the change manager, the people who handle cards at the front line, facilities or physical security, HR for screening and training and vendor management. The executive who signs the AOC should attend the closing meeting. The questions each of them is likely to face are listed in PCI DSS assessor questions: what QSAs ask, and the evidence that answers them.

The table is an example for a mid-sized organisation with one main site, one hosting environment and a contact centre; your QSA sets the actual plan and days.

After fieldwork: the ROC and the AOC

After the closing meeting you close any open items with evidence, and the QSA writes the Report on Compliance, which passes through the QSA company's own quality review before it is issued. The Attestation of Compliance summarises the result and is signed by the QSA and by your executive officer. Both go to your acquirer, or to the card brands or your customers if you are a service provider.

The PCI DSS assessment: questions buyers ask