Using Sigrid CI

This page is about interacting with Sigrid CI from a user perspective. If you’re looking for instructions on how to integrate Sigrid CI in your development platform, locate the “Sigrid CI: Pipeline integration” in the menu. See the option reference if you want to know which configuration options are available.

Sigrid CI does two things:

1. Sigrid CI publishes code to Sigrid. This integration makes it easier to keep your Sigrid up-to-date, since changes to your code are automatically reflected in Sigrid.
2. Sigrid CI provides feedback on your changes. This allows development teams to consume this feedback as part of the normal development process.

In a typical development process, step 1 would take place whenever you merge something into your main/master branch, and step 2 would apply to your pull requests. If you’re using a different process, this documentation contains guidelines on where and how Sigrid CI can be used in different development processes.

The remainder of this page is about how Sigrid CI gives feedback.

What does Sigrid CI give feedback on?

Sigrid CI gives software quality feedback for multiple quality aspects. The aspects included in the feeddback depend on your Sigrid license:

• Maintainability feedback
• Open Source Health feedback
• Security feedback (Beta, not enabled by default)

If you do not want feedback for one of these aspects, you can explicitly define the --capability option in the Sigrid CI configuration.

How does Sigrid CI give feedback?

Sigrid CI gives you feedback on different quality aspects. You can find more information on our feedback for Maintainability and Open Source Health in the sections below. However, there are also some shared principles we apply across Sigrid CI:

• Focus on objectives: All Sigrid CI feedback relates your changes to the objectives you’ve defined in Sigrid. Objectives provide context. It’s not reasonable to expect the same level of quality for a legacy system as for a brand new mission-critical system. Objectives allow you to provide this context, so that every system gets a target that is both reasonable and achievable. Using objectives also means there is a common thread between what you see in your pipeline versus what you see in the management dashboard, which helps to make different stakeholders in the IT organization have a shared perspective.
• Strive for good, not for perfection: It’s not realistic to expect teams, who often already have a high workload, to be able to fix every single issue right away. This is another area where defining objectives help you to set a baseline on what you consider good-enough quality.
• Don’t forget about positive feedback: Sigrid CI feedback is split into categories that should be familiar to anyone who has participated in an agile retrospective: What went well and What could be better. We intentionally start with the positive feedback, to put the emphasis on how people improved technical debt. This helps to create a culture of quality awareness, and helps to point out that we see and appreciate people’s efforts, and that we don’t just want to flood them with endless lists of stuff they need to work on.

Maintainability feedback

Maintainability feedback follows the feedback structure introduced in the previous section. When it comes to your objective, Sigrid CI will let you pass as long as you’re moving towards your objective. As long as you keep making incremental improvements in the right direction, you’ll get there eventually. This approach means Sigrid CI is generally “nicer” than tools that require you to fix every single issue right away.

The reason Sigrid CI is “nice” is to encourage refactoring during normal development. Let’s say you’re working on a normal, functional ticket. You find the file you’re changing is full of technical debt. Ideally, you would try to perform some minor refactorings while implementing your ticket. Obviously, we don’t expect you to fix everything, that would not be reasonable. The behavior we’re going for is known as the boy scout rule, where you should leave your code (or your campsite) cleaner than you found it.

Feedback is based on your maintainability objective. If you have not configured an objective, Sigrid CI will use a default target of 3.5 stars, which is what we recommand as a reasonable default for modern systems.

Open Source Health feedback

Sigrid CI gives feedback on security vulnerabilities and license risks in open source libraries. Sigrid also checks other aspects of using open source libraries, such as freshness (i.e. how often you update), but those are not part of Sigrid CI.

So why not give feedback on all aspects? In a word: Urgency. People generally security vulnerabilities or license issues much more urgent than those other aspects. Updating a library can wait (but please not too long), fixing a vulnerability cannot wait. Obviously, you should still manage those other aspects, as explained in our guidelines for healthy use of open source libraries.

Sigrid CI does not require you to update every single open source library to address every single issues. Which vulnerabilities and licenses are “allowed” versus “not allowed” is decided based on your objectives. This means you can use different objectives dependent on the (type of) system. For example, you can decide to prevent high or critical severity vulnerabilities in public-facing systems, but only prevent critical vulnerabilities for internal systems.

Unlike the feedback for maintainability, you really do need to address every single vulnerability and every single license issue that’s blocking your objective. This is because of a difference in urgency between maintainability and security: Maintainability is more of a “chronic” problem, while security threats are “acute” and really do need to be mitigated right away.

Sigrid CI separates vulnerable open source libraries into two categories:

• Vulnerable libraries for which a fix is available: These have a straightforward mitigation: Update the open source library, ideally to the latest version, but at minimum to a version that is no longer vulnerable.
• Vulnerable libraries for which no fix is available: These cases are more difficult to manage, since there is no obvious solution or mitigation. These cases typically require more discussion on the exact details of the vulnerability and possible follow-up actions. This is explained in more detail, also from a process perspective, in our guidelines on using open source.

Feedback is based on your Open Source Health objectives. If you have not configured an objective, Sigrid CI will use a default target of no critical security vulnerabilities. You will not receive feedback on license issues unless you have explicitly defined an objective for this.

Adding Open Source Health feedback to an existing Sigrid CI configuration

If your Sigrid license includes Open Source Health, Sigrid CI will automatically give feedback on both Maintainability and Open Source Health. If you do not want feedback on Open Source Health, even though you have a license for it, you can explicitly add --capability maintainability to only receive feedback for Maintainability.

If you use GitHub you need one extra step: In your pipeline configuration, look for the line message-path: sigrid-ci-output/feedback.md, and change this to message-path: sigrid-ci-output/*feedback.md. Adding the asterisk allows you to get feedback on all Sigrid capabilities, not just maintainability.

Security feedback (Beta)

Sigrid CI provides security feedback based on your objectives. This helps you to “shift left” by reporting security findings early in the development process, making it easier and faster to address those findings.

When you encounter security findings during code reviews, there are three ways how you can deal with them:

• Address the finding in the pull request: It’s always the best course of action to just address the finding within the pull request. This prevents the finding from ever going into the main/master branch, which is generally considered a best practice in shift-left thinking.
• Merge the pull request, manage the finding via Sigrid: In some cases, the pull request author and reviewer might agree it’s not feasible to address the finding right now. In those situations, it’s OK to merge the pull request. This will cause the security finding to appear in Sigrid’s Security dashboard, where it can be tracked.
• Merge the pull request, mark the finding as a false positive in Sigrid: Like any automated check, Sigrid can produce findings that are false positives. In those situations, if the pull request author and reviewer agree the finding is actually a false positive, it’s OK to merge the pull request. After your changes have been merged and Sigrid has been updated, you can mark the finding as a false positive in Sigrid’s security page. False positives are automatically excluded from future Sigrid CI feedback.

Feedback is based on your security objective. If you have not configured an objective, Sigrid CI will use a default target of no critical security findings. We believe this to be a reasonable default for most systems.

Why does Sigrid CI only give feedback on new security findings?

Sigrid CI’s intended workflow is to “shift left”: If you catch security issues early in the development process, it becomes easier and faster to address them. Therefore, Sigrid CI will give you feedback on new security findings, in the code that you added/changed. This feedback is intended to be used in the context of a pull request.

Including all security findings in the pull request feedback is not a good fit in terms of workflow. It is not practical or reasonable to expect people to reuse their pull request to start fixing issues in completely unrelated areas in the code. Moreover, this tends to become overwhelming if the system has lots of existing findings.

If you have a system with a large number of existing security findings, we recommend you use the security dashboard to make an explicit decision on which findings to address right now and which findings can be addressed later. This can then be planned as its own effort. You can then use Sigrid CI in parallel to avoid the introduction of new security findings in new code.

During the Beta phase, Sigrid CI will give feedback on all new findings. That might include situations where your code is not new, but new security findings have been discovered in existing code. We will decide on the final behavior based on feedback we get during the Beta phase, because arguments go both ways: (A) People prefer to only receive feedback on things they did, but (B) new security threats are continuously discovered, and we need to make people aware.

Adding Security feedback to an existing Sigrid CI configuration

Sigrid CI feedback for Security is currently in Beta, and not enabled by default. If you want to enable Security feedback, you will need to change your configuration to explicitly enable it.

The Beta version of Sigrid CI feedback for Security requires a Sigrid Security license.

• All platforms: You need to add the option --capability maintainability,osh,security to the Sigrid CI step in your pipeline configuration.
• GitHub: In addition to the above, you need one extra step: In your pipeline configuration, look for the line message-path: sigrid-ci-output/feedback.md, and change this to message-path: sigrid-ci-output/*feedback.md. Adding the asterisk allows you to get feedback on all Sigrid capabilities, not just maintainability.

How do you deal with feedback from Sigrid CI?

Feedback from Sigrid CI is intended to be used in the context of a pull request. Pull requests typically involve two participants, the author and the reviewer, and both can use this feedback. When working on the pull request, the author can use the feedback to make some corrections before asking for a review. When the pull request enters the code review, the reviewer can use the feedback as part of his or her review.

Should you block your pipeline?

If you ask us: No. Any form of automated feedback will occasionally lead to results that are unfair, and Sigrid is no different. As explained above, we ideally want Sigrid CI feedback be used as input for a code review, with the reviewer making the final call on whether to approve the review.

Still, we concede there are situations and organizations where you really do want to make Sigrid CI a mandatory check. Therefore, we do provide the option to fail the pipeline if Sigrid objectives are not met. You can configure this to make the pipeline fail on any Sigrid objective, but you can also make this behavior more nuanced by failing the pipeline on certain types of objectives. The latter is configured using exit codes, as explained here.

Contact and support

Feel free to contact SIG’s support team for any questions or issues you may have after reading this documentation or when using Sigrid.