Guidelines for healthy use of Open Source

Introduction

The purpose of this document is to provide concrete and actionable guidelines, hints and tips on how to achieve healthy use of third party libraries and frameworks in your application. This covers how to get started, how to stay in control, and how to act when health deteriorates. Where applicable, this document explains how Sigrid can be used to achieve this.

Structure and overview

The guidelines and best practices have been structured based on ‘jobs to be done’: the idea is to look at the actual tasks that stakeholders need to conduct, and provide them with help to do those tasks. Jobs can embed, or refer to, other jobs.

These jobs have been grouped in three categories: first some general guidelines for conducting healthy open source usage in development, that help to get started, and may need to be revisited or updated on a regular basis. Secondly the various types of tasks that are needed to ensure continued health of libraries (note that even when application code is not actively maintained for a while, the health of open source libraries may diminish!). And thirdly a set of practical tasks in handling libraries as a developer.

Be equipped for healthy open source usage

1. Define OSH-related policies
2. How to improve portfolio and system-level OSH
3. General guidelines for your application development

Ensuring your open source stays healthy

1. Scan the software for issues
2. Handling vulnerabilities
3. Handling license issues
4. Handling lack of freshness
5. Handling lack of activity

Handling libraries

1. Updating a library
2. Selecting a new library
3. When a library does not meet the requirements

About the SIG Open Source Health (OSH) model

The SIG Open Source Health model is described here in the documentation: OSH guidance for producers.

Be equipped for healthy open source usage

This section prescribes a typical way of working for ensuring healthy open source usage; in specific situations, you may adapt this approach, but it is best to follow a comply-or-explain approach.

1. Define OSH related policies for development

There are a number of policies on how to address open source libraries during development. For most of these policies, minimal requirements should be set for all teams; individual teams may agree on more stringent rules.

1. Define the usage of a package manager: choose the package manager(s) to be used, at least per system, preferably shared across the organization. Depending on the technologies that are used, you may need multiple package managers.

⚠️ The package managers need to be integrated in your CI/CD pipeline.

1. Set the thresholds for library risks that are (not) acceptable: this is applicable to all types of risks. Set these goals in the Sigrid objectives. SIG advises the following objectives:
   • No library vulnerabilities: having vulnerabilities of medium or higher risk is generally not acceptable as a goal, and since there are relatively few low-risk vulnerabilities in practice, a ‘clean sweep’ of all vulnerabilities is preferred.
   • No unacceptable licenses; for a typical context this means no licenses that come with obligations or restrictions for commercial usage (see the OSH Guidelines for producers for more details.). In Sigrid these are classified as no-risk, and include the MIT, BSD, and Apache licenses.
   • Ensure overall OSH quality rating is 4.0 stars ¹ or more

1. Define how frequent to check for risks such as vulnerabilities and other risks in open source libraries. Preferably check daily for vulnerabilities and quarterly for other OSH risks. See section 4. Scan the software for health issues for more details.

2. Define how fast new vulnerabilities have to be resolved; this will depend on the criticality. See the section on Handling vulnerabilities for details.

3. Declare which libraries should not be checked
   • This is useful when a library has properties that cause Sigrid to signal a risk, but that risk is a false positive.
   • Create an ignore-list. This list requires CISO approval. The ignore-list must be reviewed regularly (a few times per year): define when this review will happen.

   ⚙ Sigrid How to: To achieve this in Sigrid, the library to ignore must be added in the scope file

   ⚠️ Warning: Do note that if a library is put on the ignore-list since the reported vulnerability is a false positive, that does not necessarily mean that the other types of risk for that library should be ignored as well.

4. Optionally: Define a shared permitted-list: it can be useful (and in some organizations required) to have a shared list of libraries that are permitted.
   • This list can have an advisory role, functioning as a list of libraries that have already been checked, and are likely already in use. It can also have the role of a clearance list, where developers have only permission to use libraries from the permitted-list, and must seek approval for libraries that are not on that list.
   • For determining whether to include libraries, see the criteria defined in the section 10. Selecting a new library.

2. How to improve portfolio and system-level OSH

Especially when a system or portfolio is new in Sigrid, at the system and portfolio level there can be an abundance of OSH related issues that need to be fixed; this section provides some advice on how to tackle all those jobs incrementally, starting with the most critical and high-ROI topics first; Sigrid is designed specifically to help you focus on the highest priority issues.

1. In case no package manager is used, or not for all libraries (all technologies), it makes a lot of sense to start with the -extended- use of a package manager. This it will make all other improvement steps easier, faster, and less error-prone.
2. The next step is to focus on vulnerabilities, since these threaten your application security in the short term:
   • Start with a quick threat analysis to prioritize the systems that are most risk-prone: this means the system where attacks would have the most impact, and the highest likelihood of an attack. In particular for systems with business-criticality that is CRITICAL OR HIGH and systems containing privacy-sensitive data or transactions, the impact will be high. The likelihood is determined by the attack surface and exposure: so especially systems with a deployment type that is PUBLIC_FACING will have a higher likelihood.
   • Focus on removing all critical and high risk vulnerabilities first, continue with the remaining vulnerabilities.
3. Consider legal risks due to unacceptable licenses:
   • The highest priority are libraries that are used in an application without the proper rights; for example libraries that do not allow commercial use (if you are a commercial organization). Some of those require paying a license fee; which is the most straightforward means of addressing the issue.
   • A next category to consider are the copy-left licenses, which require the application that uses those libraries to be distributed with the same license (and e.g. also made open-source). Depending on your situation, using copyleft licenses carries the risk of not complying with license obligations, leading to consequences such as the obligation to distribute the source code and legal disputes. The main way of addressing legal risk due to unacceptable licenses is by replacing the library with another one.
4. For the other properties:
   • investigate the risks of heavily outdated and perhaps no longer maintained libraries: look at the product lifecycle, end-of-support date and maintenance activity to verify that there is a real need for replacing the library.
   • Do an impact analysis and plan the needed effort to mitigate the risks. This can be -a series of- updates, or complete replacement of a library.
   • ​​Open source libraries can be compared in terms of activity on tools like https://www.openhub.net/​
   • Looking at the bug reports of inactive libraries, and the fixed bugs or improvements of outdated libraries can give good information of the relevance of updating.

When many libraries require (multiple or major) version updates, the level of test coverage of a system can be used as an additional factor for prioritization: systems with high test coverage have a lower risk of running into defects at run-time due to incompatible updates.

3. General guidelines for your application development

There are a number of topics to consider that are not directly related to the libraries themselves, but to the way you organize the development of the application itself. The following guidelines should be considered as compliance rules for framework and library management:​

Keep application source code separate from frameworks/libraries​.

1. Do not change the source code of used frameworks/libraries: depending on the technology used, you often do not need source code at all, but will use binaries of the libraries.
2. Only a single version of each library or framework​ should be used directly: Also, do not have copies of the same library installed. It may well be that one or more of your libraries is importing another version of the same library that your application uses; such indirect use is mostly out of scope.

Regression tests and maintainability of the application code are key to updating frameworks/libraries​

If it is hard to update a library, chances are the problem lies in your codebase.​

1. Keep module coupling and component independence low, to make it easier to change code implementation (such as dealing with new versions of a library) while keeping the same behavior/requirements.
2. Develop, maintain and run regression tests. These help to identify breaking changes in updates.​​

Ensuring your open source stays healthy

This section describes guidelines, hints and tips on how to maintain healthy use of open source libraries in your application. Where applicable, we explain how Sigrid can be used to achieve this.

4. Scan the software for health issues

For timely handling of open source health risks, there are two concerns:

1. Risks that appear due to changes in the code: these need to be signalled as soon as possible (short feedback loops make it much more efficient to make changes); doing the scanning as part of the CI/CD pipeline using sigridci addresses this.

   The risks that are detected are best addressed immediately, before merging the new code.

2. Risks that appear due to changes in the ecosystem over time: these require regular scans, even when the application code does not change. For this category:

   • Vulnerabilities require frequent scanning: preferably daily, at least every 2 weeks.
   • The other OSH properties are less volatile and urgent, and monthly to quarterly scanning is sufficient for those.

   ⚠️ Warning: OSH analysis is conducted whenever Sigrid receives an update through Sigrid CI, or in the form of a new snapshot upload. Hence, for systems that are inactive, new vulnerabilities in the ecosystem are not visible in Sigrid.

A good time to triage scan results is during refinement for the next sprint: You need to decide to address the detected risks during the upcoming sprint, or possibly create a backlog item. In some cases, the detected risk is considered a false positive, or acceptable risk that can be ignored. The most common mitigation will be updating a library.

5. Handling vulnerabilities

When to remediate vulnerabilities

Security risks, and hence the urgency of fixing a vulnerability, of a certain framework or library should be determined based on at least the following aspects:

• The severity level of the detected vulnerabilities for the artifact​.
• The connectedness of the specific application: in particular the category public facing is the group of systems for which vulnerabilities need to be resolved most urgently. This information can be specified in the Sigrid metadata as the deployment type; all non-public facing categories (connected, internal and physical) are referred to below as local.

Additional considerations for prioritizing vulnerability handling can also be business criticality, lifecycle phase and the privacy sensitivity of the data that an application handles.

When a vulnerability is found, it must be remediated within a specified time period. The table below is a proposal how fast you should resolve vulnerabilities, depending on the risk level and the connectedness of the system:

How to remediate vulnerabilities

The primary means of remediating a vulnerability is to update the library: in most cases, vulnerabilities (especially critical ones) are only published once a patch is available in a new version of the library. See section 9. Updating a library for more details. Do check that the vulnerability is indeed solved in the newer version of the library.

If no such remediation is available, do a risk assessment which will have one of these outcomes:

• If the vulnerability does not pose any actual risk, ‘allow-list’ it: that means you allow the specific vulnerability for this library/application to be present. This requires approval from a CISO or other applicable role. This allowlist will be reviewed as part of a half-yearly measurement cycle.
• The risk can be mitigated in some other way. If, for example, the vulnerability is limited to a single method in the library that is not called by our application. You can then test for the use of that method and fail the pipeline in that case, to prevent future accidental risks.
• The library can be replaced, or no longer used, completely; see 12. Replacing a library for more details.
• In extreme cases, you may need to shut down the application until the vulnerability is resolved.

6. Handling license issues

SIG assesses whether a license is generally considered a risk for use within commercial software. Contact an IT lawyer to discuss license risks specifically for the code analyzed as well as the way it will be used.

Usually, license risks will appear whenever a library is scanned for the first time; either because the application is scanned for the first time, or the library has just been introduced.

Assess the license risk

• Libraries must have an acceptable license. This may be a paid license or an acceptable open-source license.
• Maintain a list of common licenses used in free and open-source software (FOSS); if a library has a license listed as acceptable, it can be used. Otherwise, see if an alternative is available, or contact the responsible in your organization to discuss whether the license is acceptable.
• Note that the actual risk of using a library with a certain license depends on the context:
  • e.g. when developing open-source software, licenses that require you to make your code available as open source are acceptable, where this is not acceptable for closed-source development.
  • It also depends on how you distribute a library:
    • Distribute the source code of the library, after making changes to it.
    • As linked libraries, not the actual source code.
    • Linked libraries are called through the network.
    • Libraries are used internally only.

The following table shows how various types of licenses are (not) suitable for different distribution policies, and explains how these common licenses are mapped to general risk levels. But do note that if your distribution model is clear, and the value listed for the particular license-distribution model is ‘ok’ in the table, then your actual licensing risk is minimal:

Possible actions

Depending on the circumstances, one or more of the following actions can be taken to remediate detected licensing issues

• Ensure that libraries with commercial licenses are properly registered and paid for.
• [When applicable] Add a license to the shared list of acceptable licenses (this may involve an approval process).
• [When applicable] Adjust the distribution model of the application to avoid violating the terms of the library license. For example, distribute linked libraries instead of (modified) source code.
• Stop using a library with unacceptable licensing conditions: in practice this means 12. Replacing a library.

7. Handling lack of freshness

Lack of freshness occurs when there is a newer version of a library available, but that version is not used in the application.

Development teams are responsible for keeping libraries up-to-date to a recent version: this may be part of How to remediate vulnerabilities, to make sure that bug fixes and improvements are incorporated, for compatibility with other libraries, or to ensure that future updates will not be too complicated or require a large effort all at once.

The remedy for lack of freshness is always Updating a library, possible exceptions are:

• the newer version has a vulnerability for which a fix is not available (very rare)
• the newer version is not compatible with other libraries.

8. Handling lack of activity

Lack of activity in the development of a library is not an urgent problem, but it is a long-term concern, in particular since it precludes detecting and patching security vulnerabilities. This issue cannot be resolved by application developers, except by Replacing a library.

Handling your libraries

9. Updating a library

There can be several reasons for updating a library:

• Urgent reasons: when a vulnerability is detected in a library that has been resolved in a new version, or a bug has been resolved in a newer version.
• Hygiene: ensure that the version of a library that you use does not fall behind too much, since that will make your life as a developer harder.

Updating to a newer version will always also improve the freshness rating. Using a package manager, updates may be installed automatically, or require updating the version constraints in the configuration file (sometimes called ‘manifest’) of the package manager.

The effort involved in updating a library can be estimated based on the release notes, and also semantic versioning: where a patch or minor version update should require very little effort, whereas for major version updates the effort can be substantial.

Scheduling library updates:

• Small library updates can be updated as part of regular maintenance.
• Larger updates (e.g. major versions or frameworks) should be planned explicitly.

Ground rule: never postpone updating

• The longer you postpone updating, the bigger the eventual pain. As your system grows and evolves, the costs and risks of upgrading an old library increase. Such an accumulation of maintenance debt may lead to a much larger effort than in the case of smaller, incremental updates.​
• if a new, stable version comes out: don’t wait, start testing. If it’s really core and really important, already start testing with release candidates.
• Do not adopt the “If it ain’t broke, don’t fix it” strategy​
  • This strategy implies that you do not update unless you have to. You stay with the current version of the third-party library until you notice something wrong in your application, no matter how often the vendor publishes an update. ​
  • Whilst easier in the short term, with this strategy you will end up with a system that depends on outdated and unmaintained libraries, where you cannot use some other libraries since they require a newer version of that library which you cannot upgrade and at some point. You may lose the ability to fix some issues at all.​
• Only when a new version breaks the behavior of the application, postponing may be warranted.

10. Selecting a new library

The main criterion for selecting a new library is when a non-trivial amount of commonplace behavior is needed within the application: implementing such behavior from scratch is typically more time-consuming and error-prone than predicted, hence reuse from an (open source) library may be the better option.

Often, libraries are part of an ecosystem, or work within a certain application framework, such as Eclipse, or Apache, where it makes a lot of sense (consistency, frictionless compatibility) to pick a library from the same ecosystem, unless it violates any of the other recommendations provided in this document (e.g. you adopted a library that is no longer maintained, etc.).

See section Reviewing a library for a detailed checklist of properties to consider before selecting a new library.

A more extensive discussion of selecting (including reviewing) open source libraries can be found in this talk.

11. When a library does not meet requirements

There are several possible cases where a library does not support the needs and requirements:

1. It’s Open Source Health has unacceptable risks.
2. Functional mismatch: a library is missing features that cannot easily be added on top, or the implementation of the library is based on assumptions or choices that are incompatible with the ones in the application.
3. Bug in library implementation: typically detected after a library has been adopted, so the cost of switching is non-negligible.
4. Compatibility break: a library does not (or no longer) work well together with another library or the application itself, due to changes in the APIs of involved components.

The basic rule is that library implementations should not be modified or customized: ​One of the main benefits of libraries and frameworks, is that they provide functionality without the duty of maintaining it. After customizing a library implementation, you lose this benefit while being dependent on the changes that the community makes.​

How to address failing requirements:

1. First, check whether a newer version of the library may solve the issue, then consider updating (Updating a library); you may also wait a bit until a fix is being released, especially when the issue is being worked on.
2. If the issue is a bug or lacking feature, you can file an issue at the maintainer of the library.
3. If the issue is a bug or lacking feature, you can also look at the implementation of the library and develop a fix:
   • You may be able to wrap the relevant library call(s) with extra code that corrects or hides the bug.
   • Alternatively, you can temporarily use the modified library while merging back the bug fix into the library (be aware of the license, and make sure you have clearance from your employer to do so). Once the community has accepted the fix, you can update and remove the local code​.
4. Consider whether another library that implements similar functionality is available, and the costs of adopting that library are acceptable. Check Replacing a library for more details.
5. If no other solution is feasible and a modification is absolutely required (or: the costs of alternative solutions are very high), the source code can be forked and put into a designated area in a version control system. In this case carefully consider the possible legal ramifications, e.g. should you make the modified version open source as well. The modification should be documented so that it can be re-applied whenever a newer version of the library is made available.

12. Replacing a library

There can be multiple reasons that require discarding a library and replacing it with another; see 11. When a library does not meet requirements for such situations.

In most cases, rebuilding a common functionality is not the best option: just assume that doing that will take much more time than expected, and will also require you to maintain the code in the future. So looking for an alternative library is most likely the best choice, unless there is only very specific and limited behavior that you need now (and in the foreseeable future).

See section 10. Selecting a new library for guidelines on how to pick a new library.

One major concern when replacing a library with a new one is that a new library will most likely come with a new API; this means that all the locations in the application that use that library may need to be identified and adjusted. This can be more than just the identifiers of method calls, but also the data types that are passed back and forth to the library API can be different, which may impact the calling code substantially. An approach in this case can be to wrap the new library and in this way provide an interface that is equal, or more similar, to the previous library.

13. Reviewing a library

Whenever choosing a new library or updating to a new version, consider the following review criteria:

1. What are the known vulnerabilities?
2. Is the license acceptable?​ (and/or is the library in the shared permitted-list). See also 6. Handling license issues.
3. Is an updated version compatible with the previous version?  ​(release notes should indicate any breaking changes)
4. Is the code quality (esp. maintainability) of the library acceptable (>3.0 stars)?
5. How mature is the version?​ (e.g. an x.0 version tends to be a bit more immature). Are there still -relevant- open issues? Use a stable version unless there is a real reason not to do so. An example might be that a Release Candidate fixes a vulnerability, and you do not want to wait for the stable version to come out.
6. Are there enough users of the library? (check for example the number of downloads, or amount of GitHub stars).