Recently I sat through a number of vendor pitches and asked questions which apparently prompted some people to ask me what sort of security questions should be incorporated into vendor evaluations. Software risk assessment is still a very unstructured, subjective and slow process as we still don't have definitive standards or metrics that are universally accepted. It is not uncommon to see meetings spend more time on technical dogfights about standards, probabilities, or rankings of various bugs and bug classes than than actually assessing risk. Also problematic is the risk of covering the same ground repeatedly when inexperienced but enthusiastic security analysts conduct ad hoc or unstructured threat modeling. I've spent approximately half my career as a software vendor, and half as a consumer, and it has been very illuminating to see the security issue from both sides of the table. This post is a synopsis of my experience with successful supply chain security management.
For vendors, it is important to engage sincerely with security teams. Customer security analysts can be very disruptive and inconvenient to the product sales cycle and it can be tempting to try to circumvent these obstacles. Such attempts may occasionally succeed, but in today's security landscape, they may also result in the vendor being singled out for special attention or scrutiny, making the security checkpoint process even longer. A more repeatable approach is to find someone among your staff who can serve as a product security manager, interfacing between customer security teams and product managers in order to understand and meet expectations of security policies, compliance, and outcomes.
Vendors may find that different sections of their customer base have larger and smaller security expectations, leading some vendors to conclude that security is a feature request to be prioritized. This approach will almost never succeed because 1) security teams, as sales execs sometimes say, "can say no to everything and yes to nothing" - they almost never have any budget authority and 2) this approach tends to signal that the vendor's security management process follows a "do the minimum" philosophy and is consequently under-resourced, relatively unsophisticated, and probably not highly effective.
When ferreting, as a customer, for supply chain and software risk I would, in addition to the usual questions, suggest the following;
It may be productive, at times, to drill down on any areas vendors steer discussion away from; these may be weak points waiting to be discovered. At the same time, it is important to avoid becoming entrenched on an issue and learn to recognize when a "rathole" is genuinely empty, or a proverbial horse is dead, and move on.
I'm also a bit of a fan of exploring problem spaces that appear, at first glance, to be empty as this has, on occasion, yielded interesting revelations. Few problem spaces are ever truly empty and those that appear so are sometimes arranged to look this way. Here again it is important to avoid becoming fixated on something that is not as interesting as it seems.
It is important to avoid taking an adversarial stance. Security efforts can succeed only when supply chain members work together. The job of the security team is to see the iceberg, not steer the ship; the duty of the security team is to identify risk and present alternatives. The question of risk appetite and residual risk tolerance, in the final analysis, is for the business stakeholders. Residual risk will never be eliminated completely without shutting down business initiatives and hobbling growth.
Avoid descending into competition for the status of smartest person in the room. Realistically, someone who has risen to be a product manager or software architect at a modern software manufacturer is quite possibly the smartest person in the room, at least among the engineers, and any such competition may be largely pointless. (This is not a self-reference; I would never claim to be the smartest person in the room, and nothing useful would result from such a claim).
Product security is driven by economic forces, as I will blog about next, and not a function of raw intelligence, so even the most brilliant software architects may have lingering security design flaws in their products worth considering. It is worth considering that a design flaw is often a very contentious thing; to be called a design flaw, an issue must generally be self-evident and obviously beyond question as to security impact.
These are the general classes of questions I ask, and have been asked, on the topic of software product security and supply chain risk.
Product security management. What does the vendor's SDL (security development life-cycle) look like? Do they undertake static or dynamic analysis, threat modeling, penetration testing (by whom)? Do they simply run a scanner twice a year? How many open and closed security bugs and design flaws do they track (most vendors will not release this, and if the answer is zero, they may be just getting started). How do customers report security issues and how are these managed? What, if any, standards, frameworks, or compliance regimes is the product designed to utilize or participate in? What, if any, third party audits or attestations exist?
Authentication and authorization: the fundamentals. How do clients authenticate? How is authorization handled? How are session fixation and brute force detected or prevented? How many auth bypass or privilege elevation bugs have been fixed (if the answer is zero, they have probably never tested). Are there any unauthenticated entry points? What do they do? Is there any code that trusts security assertions from a client, or any client side authorization code that could be tampered with?
Audit and logging. The fundamental issue here is when the CSO asks, "what happened, and who did it.." can they be answered? How rich and pervasive are logs and audit trails? Can unauthorized access be detected? Abuse or accidents by authorized users root caused? Are web services and external entry points instrumented well enough to detect abuse or attack? Can logs be transmitted to a SOC or SIEM tool for post-processing and analysis? What, if any, SIEM tools can parse the various event logs?
Data. What class of data would be present in the product or system? Does the product replicate databases, unstructured data or directory data from AD or LDAP, and how is this secured? How is data in motion secured? Data at rest? Could data loss be detected? Is TLS pervasive or are there sections that use HTTP, FTP, or the like? What encryption technologies are used and are they standards based or is some crypto code purpose-built? What, if any, auditing or testing of encryption implementation details has been performed and by whom?
Attack surface. How many entry points does the product have? What connectivity requirements do they have? Which, if any, may be Internet facing? Is there a notion of "secure by default" or is hardening left to the customer? Is there any automated security configuration management or auditing? How are patches and updates delivered, how often, and how are customers notified?
Cloud. If the product is hosted, or a cloud-like or SaaS product, how is security responsibility assigned? How are security configuration and policies managed, and by whom? How are security incidents detected and customers notified? How may security incidents have been detected and handled? (If the answer is zero, the experiential base may be short, or detection capabilities may not exist). Is there a notion of multi-tenancy and if so, how are tenant security boundaries implemented - application layer, database, or both? Has the implementation been tested or audited?
Finally, a sort of essay question: What are the principal security issues to be aware of?
No comments:
Post a Comment