When reviewing security products you often find they have some sort of Standard Security Certifications which should garantee a certain level of security.
Some certifications ensure adequate security controls are in place for audits, operational models, physical security, cryptography modules, etc.

The benefit of those certifications is that it should save you times and money to ensure some security requirements are met, they can also be used in contrat binding security controls, i.e.: you must comply with ISO XXXX.

There is however a drawback, an increasing number of vendors now hide behind those certifications and thus provide very little details about their security controls.
Likewise, many companies do not look further than a certification name on a paper to pass its security requirement reviews.

This is where the problem lies, how many Security professionals actually know what having such certifications actually means? to what part of the vendor’s product it applies to? Has the certification expired? Has it actually even been granted!?

Let’s take an example with the NIST Cryptography certification FIPS 140-2.
If a vendor claims its product to be FIPS 140-2 compliant, does that mean their cryptography implementation is secure enough?

Looking at the certification details we can quickly see there are 4 levels:
– The first level only provides a basic set of requirements at a software level.
– the Second level adds some physical security requirements
– The Third level goes further by requiring some physical detection/response controls.
– The fourth and last level looks at detecting and protecting against all possible physical threats

There was already a piece of information missing: What level was that FIPS 140-2 granted for?

Furthermore, one should check the certification was indeed granted for the product by checking on the NIST Module Validation Lists

Finally, the NIST website provides results details of all FIPS 140-2 certifications that were granted.
This is probably the most important part, because although a product may claim to be FIPS 140-2 certified only parts of its architecture may have been granted the certification.

The security implications are important. If a product uses a distributed architecture (client/server model) but the certification was granted only to the server part of its architecture, it means more investigation is required for the client part before one can label it secure enough.
Also of relevance is the version of the product for which the certification was granted, if the architecture has changed in newer versions, has it been reviewed again for FIPS 140-2 compliance?

To conclude, Standard Security Certifications are extremelly useful but they may provide a false sense of security if either the vendor choose to ommit some key details or if the security professional conducting the review does not fully understand what the standards mean.

What you see is not always what you get!