In short
A well-filled CVE record is matched against your inventory automatically. An incomplete record slips under the radar or gets surfaced “just in case.” Grading a source's completeness means deciding, with full knowledge, how much credit to give each alert.
· 5 min read.
Why an incomplete record is costly
Take two records published on the same day for two flaws of similar severity.
The first specifies the vendor, the product, versions 2.0 to 2.14.1, a standardized product identifier, a CVSS vector and a CWE. A monitoring tool matches it against your inventory in a fraction of a second, and you get either an alert or nothing — an answer either way.
The second says “a vulnerability affects product X,” with no version. No tool can decide. There are two possible outcomes, both bad: discard it and take a risk, or surface it and add noise. That cost is what the grading aims to make visible.
The four criteria
Each CNA receives an overall grade based on the share of its records that correctly fill in the following four fields:
| Criterion | Question asked of the record | What it enables |
|---|---|---|
| Vendor | Is the affected vendor identified? | The first filter, even before the product |
| Product and version | Are the product and its version range present, in standardized form? | Automatic matching against an inventory |
| CVSS score | Is a complete vector provided? | Sorting by severity, and reading the vector |
| Weakness (CWE) | Is the flaw class filled in? | Trend analysis and long-term work |
These four criteria do not assess a record's accuracy — we have no way to verify that at scale. They assess its completeness, meaning how well a machine can process it. It is a modest measure, and that is exactly what makes it honest.
The grading scale
| Grade | Record completeness |
|---|---|
| A | Excellent — nearly all fields filled in, nearly always |
| B | Good — one field is regularly missing, usually the CWE |
| C | Fair — affected versions are filled in inconsistently |
| D | Poor — automatic matching often fails |
| F | Very incomplete — the record has to be reworked by hand |
Volume is not quality
The most consistent finding: some very high-volume CNAs show low completeness, while specialized CNAs that publish ten times less produce near-perfect records. The grade cancels out the size effect by working on proportions, never on totals.
This is not an indictment. A CNA handling thousands of community reports a year does not have the resources of a vendor's product team. The ranking is there to tell you what to expect, not to hand out gold stars.
Where this work stands
What it means for you, in practice
Even without a published ranking, the grid can be reused as is: when an alert leaves you puzzled, check the four fields in order. Vendor, version, vector, weakness. If two or more are missing, do not look for the error in your tool: go read the vendor's security advisory, which almost always contains what the record left out.
Key takeaways
- Four criteria: vendor, product and version, CVSS vector, CWE weakness.
- The grade measures completeness, not accuracy: a modest, verifiable measure.
- It cancels out the volume effect by working in proportions.
- A record missing two fields should be checked with the vendor, not in the tool.
FAQ
How is the grade calculated?
On the share of a CNA's records that correctly meet the four criteria, regardless of the volume published. A CNA that publishes 30 records and one that publishes 3,000 are compared on equal footing.
Does a low grade mean a careless CNA?
No. It flags records that are hard to process automatically. Several CNAs publish quickly and in bulk, which is valuable for coverage, at the cost of incomplete metadata. These are two different qualities, and you cannot always have both.
Is the CNA ranking published?
Not yet. The method is applied internally to weight the trust given to each source. We will publish the ranking once the history is deep enough for a public grade to be defensible.