Vulnerability scanning uses tools to compare a system against a list of known weaknesses and report what is outdated, unpatched, or misconfigured. Penetration testing has a person act as an attacker, chain together several of the weaknesses found, and see how much access results at the end. The first shows which holes exist; the second shows whether an attacker can get in and how far the attack goes.
This article draws on three kinds of sources: the Regulations Governing the Classification of Cyber Security Responsibility Levels from the Laws & Regulations Database of The Republic of China (Taiwan), in the version amended on January 7, 2026 (ROC year 115) and checked on 2026-11-04; the testing guidance from NIST in the United States; and the POLISH team's experience supporting the Ministry of Education project.
Vulnerability scanning
- Identify the systemsOperating systems, applications, open ports
- Match against a known-vulnerability databaseOutdated versions, missing patches, misconfigurations
- List the findingsHigh false-positive rate; a person has to review the results
Penetration testing
- Find likely targetsOften starts from vulnerability scan results
- Chain multiple vulnerabilitiesSimulates a real attack
- See how far it goesHow much access is gained, and whether defenders can detect it
Paraphrased from NIST SP 800-115.
How vulnerability scanning and penetration testing differ
NIST SP 800-115 draws a clear line between the two. The document was published in September 2008, and tools and techniques have changed a great deal since, so this article cites only its definitions and limitations of the two approaches. The NIST page lists the document as Final and shows no withdrawal.
| Comparison | Vulnerability scanning | Penetration testing |
|---|---|---|
| Method | Uses tools to identify the operating systems, applications, and open ports on a host, then matches the results against the scanner's database of known vulnerabilities | Assessors simulate real attacks to find ways around a system's security controls, often launching real attacks against real systems and data |
| What it finds | Outdated versions, missing patches, misconfigurations | How much access multiple vulnerabilities provide when combined; how skilled an attacker needs to be; whether defenders can detect and respond |
| Main limitations | High false-positive rate; needs someone who understands network and operating system security to interpret results; sees only the surface of running systems; cannot describe overall risk | Labor-intensive, requires deep expertise, can damage systems, and should be done only after notification and planning |
The two work in sequence: a penetration test usually starts with port and service identification and vulnerability scanning to find likely targets. In addition, several low-risk vulnerabilities can combine into a high-risk one, which a scanner cannot see, and NIST considers penetration testing the more reliable way to find these. So a tender that requires both is not duplicating work, and the regulation lists the two separately, with different frequencies.
How often the regulation requires testing
Article 2 of the Regulations Governing the Classification of Cyber Security Responsibility Levels divides government agencies and specific non-government agencies into five levels, from highest to lowest: A through E. Article 6 states that an agency that maintains and operates information and communication systems established or developed by the agency itself or through outsourcing is Level C; the Level A and Level B circumstances are in Articles 4 and 5. Requirements for Levels D and E are not listed in this article.
The appendices list “vulnerability scanning” and “penetration testing” as two separate items under “security testing.” The same column also lists a network architecture review and several malicious-activity reviews, and a “cyber security health check” is listed separately. When a tender says “cyber security testing,” check exactly which items it means.
The frequencies for government agencies are below. Appendices 2, 4, and 6, for specific non-government agencies, set the same frequencies at each level.
| Responsibility level | Vulnerability scanning | Penetration testing |
|---|---|---|
| Level A | All core information and communication systems, twice a year | All core information and communication systems, once a year |
| Level B | All core information and communication systems, once a year | All core information and communication systems, once every two years |
| Level C | All core information and communication systems, once every two years | All core information and communication systems, once every two years |
The phrase in the table that is easiest to misread is “all core information and communication systems.” The frequency applies to core systems, not to every system an agency has. The vendor does not decide which systems count as core; according to the appendix note, for shared systems the agency primarily responsible for setting up, maintaining, or developing the system makes that call. After an agency's level is first determined, the level changes, or its core systems change, the appendix requirements apply starting the following year.
This table reflects the regulation as written. Which level's frequencies apply depends on the level determined for the agency; if the tender requirements specify something different, the contract text governs.
What came up in practice while supporting the project
The security testing on the Ministry of Education project was a requirement from the competent authority, and the team supported it. The team's role was system development and operations. Once results come back, someone has to be able to read the results, judge the findings, and fix what is found.
The scan scope is wider than the tender wording
The scans this project supports fall into these categories: web, penetration, source code, third-party components, and licensing. Reading only the words “vulnerability scan” in the tender, it is easy to assume a single tool run against hosts; in practice source code, third-party components, and licensing status are all in scope. Clarify the scope before estimating effort and scheduling.
False positives in scan results are appealed one by one
The host vulnerability scan results contained false positives. These items are handled through an appeal process, one item at a time. This matches the limitation NIST lists: scanners have a high false-positive rate and report vulnerabilities that do not exist, so someone has to interpret the results.
Genuine problems are a separate matter: the team is responsible for patching the medium- and high-risk items in the scan results.
- Host vulnerability scanProduces the results
- Someone interpretsHigh false-positive rate
- False positivesAppealOne item at a time
- Medium and high riskPatchBy the development team
The team's view is that both of these should be written into the contract. Judging false positives takes someone who knows the system, and patching takes someone who can change the code. These are usually the same operations vendor, and if the contract does not say so, no one knows who moves first.
Testing is only one part of the security support work
On the same project, the security work the team supports also includes the following:
- Matching against the National Institute of Cyber Security's VANS vulnerability notification system, and updating the high-risk software on its list.
- The cyber attack and defense exercise of the Executive Yuan's National Information and Communication Security Taskforce: helping fill out the self-assessment checklist and a survey on protective software installation.
- Restore drills for the business continuity plan (BCP).
- Adoption of the Government Configuration Baseline (GCB), with logs sent over syslog.
- Deployment of endpoint detection and protection software: an endpoint detection SOP, antivirus installation, and incident handling.
- Maintenance of the forms and audit documents for the information security management system (ISMS).
Dividing testing from development and operations
Results carry more weight as a reference when developing the system, testing it, and writing the report are done by different people. Where the system is developed or operated by the development team, testing is carried out by the agency or a testing firm, and the team helps read the report, sort out false positives, file appeals, and patch. This support can be written into the project scope or annual operations; see post-project operations and security support.
What to confirm with the vendor before signing an outsourcing contract
This list is the team's judgment, not statutory text. When a tender or the competent authority requires vulnerability scanning and penetration testing, settle these points with the vendor before the contract is signed.
- Who runs the scan: the vendor, a testing firm designated by the agency, or the competent authority handling it centrally. Whoever runs it determines who has the final say on the points below.
- Who receives the report: the agency, the vendor, or both. For the raw results and the processed version, state who can see which one.
- Who appeals false positives: judging whether a finding is a false positive takes someone who knows the system. Agree in advance on the appeal contact and on who holds the decision.
- Who is responsible for the patching deadline: the time allowed for patching medium- and high-risk items, and who bears the cost of delay, both need an owner.
- Which environment is tested: test or production, to be decided up front. NIST warns that penetration testing can damage systems and should follow notification and planning. If testing will be in production, agree in advance on the time window, who is notified, and how to recover.
The testing scope should also be written down; the Ministry of Education project split it into several categories. If the system includes a feature such as AI Q&A, state whether that part is in scope. For background, see the basics of RAG. Remaining questions on choosing a vendor are collected in choosing an outsourced software vendor.
If a tender requirements document is on hand, send the passages that mention vulnerability scanning and penetration testing to the development team through the contact page. The team can help separate what belongs to the testing party from what the operations party supports.
