# OpenSSF Scorecard Meeting Archive — 2021

## Dec 16, 2021

Updates:

* Azeem  
  * Scorecard might be getting an intern\!  
* Jeff  
  * Working on doc with long/short term plan for Allstar \+ Scorecard integrations.  
  * Not planning on implementing: [https://github.com/ossf/allstar/issues/22](https://github.com/ossf/allstar/issues/22)  
  * Next integration: check for Scorecard GHA  
  * Further integration: interpret GHA results  
* Laurent   
  * Separating checks from the score policy  
  * Partnered workflows for GitHub market place [\#1302](https://github.com/actions/starter-workflows/pull/1302)  
  *   
* Naveen  
  *   
* (Chris)  
  * (11/29) Adding line numbers to results: [\#886](https://github.com/ossf/scorecard/issues/886) and [\#725](https://github.com/ossf/scorecard/issues/725)  
* David A. Wheeler \- distracted by Great MFA Distribution project & log4j  
  * Any more thoughts about taking data on scorecards from .yaml file? SECURITY-INFO.yml, e.g., [https://docs.google.com/document/d/1Hqks2J0wVqS\_YFUQeIyjkLneLfo3\_9A-pbU-7DZpGwM/edit\#](https://docs.google.com/document/d/1Hqks2J0wVqS_YFUQeIyjkLneLfo3_9A-pbU-7DZpGwM/edit#)  
    * The big issue was how to score it. Could give lower number than 10 (“project says they have it, but we can’t confirm it), or separate numbers (ignore / consider claims from project). Could also add an “asterisk” next to numbers that are from project claims. But it’s important to have an idea on how to do this, or this file is much less useful.  
  * We have a dependabot warning on scorecard project itself :-)  
  * LICENSES: Should we score lower if license doesn’t match known OSS license? [https://github.com/ossf/scorecard/issues/1369](https://github.com/ossf/scorecard/issues/1369)  
* 

Open issues:

- [x] ~~Email to Michael Scovetta, cc David A. Wheeler \- [Michael.Scovetta@microsoft.com](mailto:Michael.Scovetta@microsoft.com)~~  
- [x] ~~Create scorecard-webapp repo \- Naveen Srinivasan~~

* No meeting on Dec 30th   
* V4 release  
  * Docker image creation [\#15](https://github.com/ossf/scorecard-action/issues/15)  
  * [Release notes](https://docs.google.com/document/d/1tIBXs801G-d2zLWzr9qBq6uKAXvshMs9wMTpDVwHmMU/preview?resourcekey=0-UlykB-9xEM5wj2aQPHTTMg). Please add list of enhancements there.  
  * OSSF metrics not up to date, e.g. [https://metrics.openssf.org/search?q=systemd%2Fsystemd](https://metrics.openssf.org/search?q=systemd%2Fsystemd) claims \`last updated 8 months ago\`. The the page says \`Last refreshed 2021-09-13\` [https://metrics.openssf.org/grafana/d/default/metric-dashboard?orgId=1\&var-PackageURL=pkg%3Agithub/systemd/systemd](https://metrics.openssf.org/grafana/d/default/metric-dashboard?orgId=1&var-PackageURL=pkg%3Agithub/systemd/systemd). Is data stale? We want to say we've scaled to 1M in blog post and release notes and point to OSSF metrics dashboard. Can this be done for Jan? Note: should use the new data format.  
  * “We used to update Google cloud buckets, now we don’t”  
  * They should be automatically updated somehow  
*   
* Create a scorecard-webapp repo (azeems)  
  * Repo name?  
  * Golang webapp?  
* Improving code quality (azeems)  
* Reducing number of open issue (azeems)  
* Dockerfile pinning from same team (laurent)

## Dec 2, 2021

* Laurent: GH action on PR: disable?  
  * Laurent: GH action: badges vs runs

**Please sign yourselves**

- David A. Wheeler

Score for pinning dependencies: start with 10, \-1 if any/all of the actions of the platform (forge & CI system) aren’t pinned, \-1 for each additional unpinned dependency, minimum 0\. E.g., CIFuzz is hard to pin, but don’t really want to make an exception because it SHOULD be pinnable (they merge code & configuration, and that really should be separate). This way, the problem is noted without us having to constantly make exceptions. We didn’t do percentages; a project with 50 unpinned dependencies is riskier than one that has 1, even if their percentages are the same.

David: Let’s make sure we don’t lock requirements to GitHub. There’s GitLab & other forges.

Naveem demo’d an early-hacked tool to list the transitive dependencies & what’s highest-risk. He’ll switch to prepared statements in his SQL first. This could be contributed to OpenSSF.

FYI: Security-Threats WG has discussed OSSF-SECURITY.yaml, currently early in discussion \- [https://docs.google.com/document/d/1Hqks2J0wVqS\_YFUQeIyjkLneLfo3\_9A-pbU-7DZpGwM/edit](https://docs.google.com/document/d/1Hqks2J0wVqS_YFUQeIyjkLneLfo3_9A-pbU-7DZpGwM/edit)  
Goal: Provide a text file that tries to justify answers for Scorecard & CII Best Practices Badge & other analysis tools like it. Willing to support? Issue: Do you trust the text? Possible solution: cap the score.

- Azeem: I like having a common format, not sure how much we could trust it. Using it might reduce users’ trust in scorecard.  
- Jeff Mendoza: Posting in BitQuery is trusted, needs to be very trustworthy. For actions & badges might accept it.  
- David Wheeler: Could report both scores (automated \+ self-assertion)  
- Jeff: Some redundancy with future scorecard configuration (disable a check), vs setting this declaration (leave check enabled, declare compliance).  
- 

## Nov 15, 2021

Attendees:  
**Please sign yourselves**

- Naveen Srinivasan
- Jeff Mendoza

Updates:

* (Azeem)  
  * Mostly working on [https://github.com/ossf/scorecard/issues/1240](https://github.com/ossf/scorecard/issues/1240)  
* Jeff  
  *   
* (Laurent)   
  * Finished GH action support for pull requests  
  * Drafting a doc for scorecard badges  
  * Started a PR for raw e2e results, will write a pager for implementation details  
* (Naveen)  
  * Move docker builds to ko \- [https://github.com/ossf/scorecard/pull/1214](https://github.com/ossf/scorecard/pull/1214)  
  * Working on enabling scorecard to sign releases using cosign and keyless  
    * [https://github.com/ossf/scorecard/issues/1201](https://github.com/ossf/scorecard/issues/1201)  
    * Opened a PR in cosign to address one of the pending items which is a blocker [https://github.com/sigstore/cosign/pull/1001](https://github.com/sigstore/cosign/pull/1001). This has been merged  
    * Rekor has a issue in uploading large binary [https://github.com/sigstore/rekor/issues/481](https://github.com/sigstore/rekor/issues/481)  
    * Fix the bug [https://github.com/ossf/scorecard/pull/1272](https://github.com/ossf/scorecard/pull/1272)  
    *   
* (Chris)  
  * (11/29) Adding line numbers to results: [\#886](https://github.com/ossf/scorecard/issues/886) and [\#725](https://github.com/ossf/scorecard/issues/725)

Open issues:

* Updated meeting time Thursdays 4PM EST.  
* Jose from GH is here. Discuss GH Actions.  
  * Send Jose a link to the action to get it verified “blue check mark” in market place. Need to rename action  
  * Custom logo in the marketplace. Need an SVG logo from our side.  
    * Scorecard logo:  
    * [https://github.com/ossf/scorecard/blob/main/artwork/openssf\_security.svg](https://github.com/ossf/scorecard/blob/main/artwork/openssf_security.svg)  
    * [https://github.com/ossf/scorecard/blob/main/artwork/openssf\_security\_alt.svg](https://github.com/ossf/scorecard/blob/main/artwork/openssf_security_alt.svg)  
    * OpenSSF logo:  
    * [https://github.com/ossf/artwork](https://github.com/ossf/artwork)   
  * Can setup a custom workflow like a partner GitHub action. \[2 weeks in advance\]  
  * [https://partner.github.com/resources](https://partner.github.com/resources) explains the steps  
* For v4 release  
  * Fix branch protection [https://github.com/ossf/scorecard/pull/1252](https://github.com/ossf/scorecard/pull/1252)   
  * (low prio) Bug \`Resource not accessible\` [https://github.com/ossf/scorecard/issues/1097](https://github.com/ossf/scorecard/issues/1097)   
  * GH action:  
    * Test actions more thoroughly (Appu, ask evverx@)  
    * GH action doc: Asra/Kara  
    * Action verified: laurent  
    * Docker image generation: Naveen  
    * Add line numbers/filenames to more checks (Chris)  
  * [https://github.com/ossf/scorecard/issues/1128](https://github.com/ossf/scorecard/issues/1128) make token permissions defined at top level mandatory.  
    * [https://github.com/ossf/scorecard/issues/1257\#issuecomment-968353572](https://github.com/ossf/scorecard/issues/1257#issuecomment-968353572) agrees it's better.  
  * Vendor dependencies for hermetic builds. [https://github.com/ossf/scorecard/issues/1188](https://github.com/ossf/scorecard/issues/1188) [https://github.com/ossf/scorecard/pull/1208](https://github.com/ossf/scorecard/pull/1208)   
    * Punting on this for now  
* Enable signed commits for this repository. [https://github.com/ossf/scorecard/issues/223](https://github.com/ossf/scorecard/issues/223)  
  * Only do this for merge  
* Should scorecard be forcing users to use a zap.Logger? [https://github.com/ossf/scorecard/issues/1273](https://github.com/ossf/scorecard/issues/1273)  
  * Keep the issue open and see if we can get to it soon

## Nov 1, 2021

Updates:

* (Azeem)  
  * 1M repos now  
* (Chris)  
  * Working on parsing issues.  
  * Will use actionlint to parse workflows.  
  * Will also look into adding line info to PinnedDependency check.  
* (Jeff)  
  * AllStar overview: install GH app on repos or orgs. If repos are out of compliance, can file issues and potentially in future fix it for you.  
  * Allstar is using Scorecard as a library. Only calls BinaryArtifact check.  
  * Scorecard does not have much input config as of now. Having this helps with applying fine-grained policies.  
  * AllStar check for Scorecard GH action.   
  * AI(jeff): one-pager on Scorecard are library or GH Actions.  
* (Laurent)  
  * Finishing up on GH action  
* (Naveen)  
  * Reproducible Builds GoReleaser  
  * Fixed the Broken e2e tests  
    * RepoClient e2e is still pending.   
  * Fixed the e2e tests which weren't running and now it is running part of CI  
  * Released 3.1.1 with arm builds for Darwin

Issues that [need discussion](https://github.com/ossf/scorecard/issues?q=is%3Aopen+is%3Aissue+label%3A%22needs+discussion%22):

* Frido: Project Thoth collaboration \- [https://thoth-station.ninja/](https://thoth-station.ninja/)  
  * Analysis and recommendations for Python packages  
  * OSS project  
  * Uses prescriptions \- [https://github.com/thoth-station/prescriptions](https://github.com/thoth-station/prescriptions)  
  * Uses Scorecards. Fetches data from BQ and to create prescriptions.   
  * Eg for Flask repo \- [https://github.com/thoth-station/prescriptions/tree/master/prescriptions/fl\_/flask](https://github.com/thoth-station/prescriptions/tree/master/prescriptions/fl_/flask)  
  * Having difficulty in using Big Query  
* Need decision on \`--repo=file://\` or \`--local\`  
  * Go with local option for now  
* OSSF to discuss guidelines when recommending tools in our documentation (no span, open-source, etc): David suggested we write a doc and he will circulate to his lawyer if tools we recommend are proprietary. For oss tools, it's fine. We can say "Here are tools we are aware of that can help: X, Y, Z".  
  * Laurent to create an issue and submit a doc discussing this. Will defer to David Wheeler once we have consensus internally.  
* Token permissions [https://github.com/ossf/scorecard/issues/1129](https://github.com/ossf/scorecard/issues/1129)   
  * Outcome: keep opinion out of Scorecard. Create an umbrella bug for separating policy and evaluation.   
  * AI(naveen): create issues for these.  
* Scorecard should sign releases binaries with cosign using keyless option. [https://github.com/ossf/scorecard/issues/1201](https://github.com/ossf/scorecard/issues/1201)  
  * AI(naveen): Involve Asra, Priya and confirm who the entity will be. And how closely will this map to SLSA builds?  
* Record Scorecard scans into Rekor [https://github.com/ossf/scorecard/issues/1200](https://github.com/ossf/scorecard/issues/1200)  
  * Need more discussion with the Rekor team. May consider it for v5 not for v4.  
  * AI(naveen): update issue with comments  
* GH actions: upload in our action or leave this in the workflow?  
  * Leave as is, in the workflow. We can add a different action later to do it all at once, without breaking existing users

## Oct 18, 2021

Updates:

* (Azeem)  
  * Created Token server and multiple cron workers running now.   
* (Laurent)   
  * Some additional ideas to improve communications/engagement with the community, see [doc](https://docs.google.com/document/d/1rgk7-jnXVpLVvB84YlJ75fqQX32RLn-FI_y5Ko2CeV8/edit).  
  * Crowdsourcing cron job to client workflows, see draft [here](https://docs.google.com/document/d/1s9xkRM2EasyEf6tBO8a6vGieMNxN9h3I00UoCQOrnw8/edit?resourcekey=0-CuJDKI0Q0A0P6udmVAhSog#).  
  * SLSA with GiHub, see draft [here](https://docs.google.com/document/d/1JbLFhpCpy6Y607g72W-BekAoBC6djaKz3gbcMcLWhwQ/edit?resourcekey=0-eRujJ1O_5m4_Bt0Z0KugsA#).  
* (Naveen)  
* (Chris)

Issues that [need discussion](https://github.com/ossf/scorecard/issues?q=is%3Aopen+is%3Aissue+label%3A%22needs+discussion%22):

* Milestone for v4 planning [\#1121](https://github.com/ossf/scorecard/issues/1121)  
  * Resolution reached on \#1121  
* Token permissions [https://github.com/ossf/scorecard/issues/1129](https://github.com/ossf/scorecard/issues/1129)   
  * We shouldn’t be recommending commercial tools. Ask the bigger community what the guidelines here are.  
* Pinning versions: [https://github.com/ossf/scorecard/issues/1014\#issuecomment-943007111](https://github.com/ossf/scorecard/issues/1014#issuecomment-943007111)  and [https://cloud.google.com/blog/topics/developers-practitioners/best-practices-dependency-management](https://cloud.google.com/blog/topics/developers-practitioners/best-practices-dependency-management) [https://github.com/ossf/scorecard/issues/427](https://github.com/ossf/scorecard/issues/427) I'm going to start a doc with Rhys from renovabote: inventory and description of lock files options for each package manager. If you know someone from dependabot and other package managers, let me know.  
* PGP signing of commits [https://github.com/ossf/scorecard/pull/1123](https://github.com/ossf/scorecard/pull/1123)  
  * laurentsimon@ to attend the next "Best practices for open source developers" meeting for discussion  
  * Need to think more on it, not much clarity  
* Scorecard meet calendar issues  
  * azeems@ to ping \#general Slack channel  
* AI(Laurent): Ask Asra to join bi-weekly 

## Oct 4, 2021

Updates:

* (Azeem)  
* (Laurent)   
  * GH action for push/cron triggers. You can try it out by following [\#1074](https://github.com/ossf/scorecard/issues/1074)  
  * We need to add file and line numbers to all checks ([\#886](https://github.com/ossf/scorecard/issues/886), [\#725](https://github.com/ossf/scorecard/issues/725), etc)  
  * We're ready for a v3 release:  
    * It provides numeric scoring and risk categories instead of plain Pass/Fail; aggregate scores; and a new repo interface to ease integration with other code versioning systems. We have scaled scanning to 150k GitHub repos (critical ecosystems dependencies from deps.dev) and we continuously make the results publicly available.  
    * Contributor ack: chrismcgehee, nanikjava, sprabery, neil465, nathan-415, amamoose, asraa ,  loosebazooka, johanbrandhorst, oliverchang, evalphobia, notanton, ben-moss, olivekl (rough estimate)  
    * LTS support  
* (Naveen)  
* (Chris)

Issues that [need discussion](https://github.com/ossf/scorecard/issues?q=is%3Aopen+is%3Aissue+label%3A%22needs+discussion%22):

* Rest API instead of BQ table [\#1056](https://github.com/ossf/scorecard/issues/1056)  
* Naveen: Improving discussion for scorecard in general. See this [doc](https://docs.google.com/document/d/1J576To36LUb7O01fQJVO_ysRH-H0fulWHCm13gONA-w/edit?resourcekey=0-mkbr-VNWS16HyAZXjha6mw#). (I don't know how to make if world-readable, so request access)

## Oct 4, 2021

Updates:

* (Azeem)  
* (Laurent)   
  * GH action for push/cron triggers. You can try it out by following [\#1074](https://github.com/ossf/scorecard/issues/1074)  
    * Some checks fail on PR event.   
    * For v0 only use checks which use local repo content and not GitHub APIs.  
    * Enable BranchProtection in PRs?  
      * Start with source code only and then consider implementing BP check.  
  * We need to add file and line numbers to all checks ([\#886](https://github.com/ossf/scorecard/issues/886), [\#725](https://github.com/ossf/scorecard/issues/725), etc)  
  * We're ready for a v3 release:  
    * It provides numeric scoring and risk categories instead of plain Pass/Fail; aggregate scores; and a new repo interface to ease integration with other code versioning systems. We have scaled scanning to 150k GitHub repos (critical ecosystems dependencies from deps.dev) and we continuously make the results publicly available.  
    * Contributor ack: chrismcgehee, nanikjava, sprabery, neil465, nathan-415, amamoose, asraa ,  loosebazooka, johanbrandhorst, oliverchang, evalphobia, notanton, ben-moss, \+ (rough estimate)  
* (Naveen)  
* (Chris)

Issues that [need discussion](https://github.com/ossf/scorecard/issues?q=is%3Aopen+is%3Aissue+label%3A%22needs+discussion%22):

* Naveen: Improving discussion for scorecard in general. See this [doc](https://docs.google.com/document/d/1J576To36LUb7O01fQJVO_ysRH-H0fulWHCm13gONA-w/edit?resourcekey=0-mkbr-VNWS16HyAZXjha6mw#). (I don't know how to make if world-readable, so request access)  
  * How can the OSS community be better involved in the Scorecard project?  
  *   
* Pinning in vendored directories [\#1095](https://github.com/ossf/scorecard/issues/1095), Linear history and review dismissal [\#1027](https://github.com/ossf/scorecard/issues/1027)  
* Support for local folder instead of GitHub online repo. We can support \--repo=file://path/to/source-code, \--repo=path/to/source-codeor create a new option \--source-code=/path/to/code  
* REST API:  
  * As a v0 maybe export BQ data to GCS and expose REST API on top of that.  
  * A full-fledged REST API is in roadmap further down the road.  
* 

## Sep 27, 2021

Updates:

* (Azeem)  
* (Laurent)   
  * Cron JSON, bypassable code-review/BP, policy file (GH actions \= 1st use case)  
* (Naveen)  
* (Chris)

Open Issues:

* (Christie \+ Kim) Demo of using Scorecard for Go dependencies  
* Naveen \- Demo of using OSV for Go dependencies \- [https://github.com/naveensrinivasan/stunning-tribble](https://github.com/naveensrinivasan/stunning-tribble)

* (Laurent) Versioning and letting our users know [https://docs.google.com/document/d/1femVdKsCWAOHJ1cyIskqd-1k9wTDKhrLnXRrOcLeVhc/edit](https://docs.google.com/document/d/1femVdKsCWAOHJ1cyIskqd-1k9wTDKhrLnXRrOcLeVhc/edit), see also [https://github.com/ossf/scorecard/pull/1001\#discussion\_r706943610](https://github.com/ossf/scorecard/pull/1001#discussion_r706943610) thread  
  TL:DR: we need to define at least a few public facing API/formats (cron JSON) and use these to announce new versions, changelogs, breaking changes, update steps; then adopt a proper channel to tell our users. Defining an SLO on new versions should also be explicitly stated: e.g., any breaking change should have support for N months minimum.   
* (Laurent) Thinking about what scorecard badges for GitHub repos would look like: how about badge levels based on score (e.g., score \>= 3 \-\> bronze; score \>= 5 \-\> silver, etc)? do we want more granular badges short-term, long-term? (e.g., a certain branch protection flag is enabled or disabled). Note that for the majority of checks, the distinction between score-based and granular badges is not really important \- it does provide some benefits for branch protection.  
* Pinned deps check: is it too large? Shall we split it into smaller ones:  
  * Docker pinning \- note that would make such check result binary (0 or 10\)  
  * Workflow pinning \- same note  
  * Etc  
* https://github.com/ossf/scorecard/issues/1027  
* [https://github.com/ossf/scorecard/issues/895](https://github.com/ossf/scorecard/issues/895) and [https://github.com/ossf/scorecard/issues/1005](https://github.com/ossf/scorecard/issues/1005) broken/bypassable check

## Sep 20, 2021

Open Issues:

* (Christie \+ Kim) Demo of using Scorecard for Go dependencies  
* Naveen \- Demo of using OSV for Go dependencies   
* (Laurent) Versioning and letting our users know [https://docs.google.com/document/d/1femVdKsCWAOHJ1cyIskqd-1k9wTDKhrLnXRrOcLeVhc/edit](https://docs.google.com/document/d/1femVdKsCWAOHJ1cyIskqd-1k9wTDKhrLnXRrOcLeVhc/edit), see also [https://github.com/ossf/scorecard/pull/1001\#discussion\_r706943610](https://github.com/ossf/scorecard/pull/1001#discussion_r706943610) thread  
  TL:DR: we need to define at least a few public facing API/formats (cron JSON) and use these to announce new versions, changelogs, breaking changes, update steps; then adopt a proper channel to tell our users. Defining an SLO on new versions should also be explicitly stated: e.g., any breaking change should have support for N months minimum.   
* (Laurent) Thinking about what scorecard badges for GitHub repos would look like: how about badge levels based on score (e.g., score \>= 3 \-\> bronze; score \>= 5 \-\> silver, etc)? do we want more granular badges short-term, long-term? (e.g., a certain branch protection flag is enabled or disabled). Note that for the majority of checks, the distinction between score-based and granular badges is not really important \- it does provide some benefits for branch protection.  
  * Related: [https://github.com/ossf/scorecard/issues/1027](https://github.com/ossf/scorecard/issues/1027)  
  * Chris: not having scores might break users when a code change changes score calculation  
  *   
* Pinned deps check: is it too large? Shall we split it into smaller ones:  
  * Docker pinning \- note that would make such check result binary (0 or 10\)  
  * Workflow pinning \- same note  
  * Etc  
* [https://github.com/ossf/scorecard/issues/1005](https://github.com/ossf/scorecard/issues/1005) broken/bypassable check  
  * Closed?  
* [https://github.com/ossf/scorecard/issues/895](https://github.com/ossf/scorecard/issues/895) and   
  * azeems@ to look into it

## Sep 7, 2021

Updates:

* Azeem  
  * 100k cron repos.  
  * RepoClient interface implementation complete.  
* Laurent   
  * Cleanup JSON format and added schema validation/unit tests  
    * David A. Wheeler: Will this impact the OpenSSF metrics dashboard?  
    * No, it shouldn’t. I talked with Michael Scovetta & all should be fine.  
* Naveen  
  * sigstore cosign had a breaking change, looking into it.  
* Chris

Open Issues:

* Cleanup JSON format and added schema validation/unit tests  
* JWT potential vuln. \- github.com/dgrijalva/jwt-go  
  * Coming due to github.com/bradleyfalzon/ghinstallation  
* [https://github.com/ossf/scorecard/issues/942](https://github.com/ossf/scorecard/issues/942)  
  * Naveen working on it already  
* [https://github.com/ossf/scorecard/issues/967](https://github.com/ossf/scorecard/issues/967)  
  * Good first issue  
* [https://github.com/ossf/scorecard/issues/966](https://github.com/ossf/scorecard/issues/966)  
  * Scorecard should look for whether certain tools are run and not specifically run those tools.  
  * Running this can cause false positives which needs to be handled by Scorecard  
  * Look for whether or not there are security scans  
  * It’s easy to detect most CI pipelines & check if they do certain things, just by looking at configuration files within the project. Most projects use a small number of CI pipeline systems, which are configured by a small set of config files:  
    * .github/workflows  
    * .circleci  
    * .travis.yml  
    * .gitlab-ci.yml  
  * An alternative is actually running the scans & putting in results \- that’s a different topic \- push that option off to the future (if ever), not this year at least.  
  * Currently in scope of Scorecard is to check configs but in future we might look into  
* [https://github.com/ossf/scorecard/issues/965](https://github.com/ossf/scorecard/issues/965)  
  * This is an idea for hardening the Scorecards project itself  
* [https://github.com/ossf/scorecard/issues/950](https://github.com/ossf/scorecard/issues/950)  
  * Maybe wait until OpenSSF as an organisation decides the policy here and follow the same policy for Scorecard.  
  * SBOM \- Scorecard can get started, but signing / cosign needs a bigger discussion.  
  * Signing \- start identifying how to sign packages for OpenSSF? Okay to use cosign? Any particular preferences?  
* [https://github.com/ossf/scorecard/issues/859](https://github.com/ossf/scorecard/issues/859)  
* Webhook for prod cron job. Volunteers?  
* Naveem will look at CII Best Practices badge site to see what it’s missing on Scorecard & talk with David A. Wheeler about that  
  * https://metrics.openssf.org/grafana/d/default/metric-dashboard?orgId=1\&var-PackageURL=pkg:github%2Fcoreinfrastructure%2Fbest-practices-badge  
  * https://github.com/coreinfrastructure/best-practices-badge  
  * Naveem will email [dwheeler@linuxfoundation.org](mailto:dwheeler@linuxfoundation.org)  
  * E.g.: CII Best Practices badge \*DOES\* have frozen dependencies \- just look for “Gemfile.lock”.

## Aug 23, 2021

Updates:

* (Azeem)  
* (Laurent) SARIF support  
* (Naveen)  
* (Chris)

Open Issues:

* Protobuf is being rebuilt for every check run on PRs, which takes time and delays merging. Can we pull a protobuf binary directly, or use apt-get, or something else?  
  * Naveen to work on this.  
* Go.sum lists the same dependency multiple times (different metadata/hashes). Can this be cleaned?  
  * This is expected since difference  
* e2e tests status [https://github.com/ossf/scorecard/issues/861](https://github.com/ossf/scorecard/issues/861)  
  * New contributor working on it. If not, Naveen will take it up.  
* Note from David A. Wheeler: Hi, I teach classes at George Mason University (GMU) on Monday evenings (4:30-10pm US Eastern Time), starting August 23\. This is the time when the Security Scorecards meetings meet. It'd be awesome if there was a different time things can be scheduled to. If not, I understand, but please understand I can't join in that case. Thanks\!\! \<[dwheeler@linuxfoundation.org](mailto:dwheeler@linuxfoundation.org)\> Also: Is there a way to contact the group other than issues? E.g., a mailing list? If so, let me know…  
  * Azeem to try and reschedule.

## Aug 9, 2021

Updates:

* Scorecard checks/sarif (Laurent)  
* Fixes (Naveen)  
* Line numbers in remediation results (Chris)  
* [Friction log](https://docs.google.com/document/d/1dPnjpS8SQo8GfqVBkmxwhPCJz4j9YRS7Sr51u4lnysE/edit?resourcekey=0-DBA9jpoXbRB5RSdCvI-xXA#) by Kara

Open Issues:

* E2E Integration tests \- Assign to Naveen  
* Bubble up results into OSV  
* [https://github.com/ossf/scorecard/issues/792](https://github.com/ossf/scorecard/issues/792)  
* Security-Policy: add a check for allstar being installed?  
* ShiftLeftSecurity  
* [https://github.com/ossf/scorecard/issues/759](https://github.com/ossf/scorecard/issues/759)

## Jul 26, 2021

Attendees:

* David A. Wheeler (Linux Foundation)

Updates:

* Cron job/performance (Azeem)  
  * E2E release testing is setup. Next step is to setup a release procedure.  
* RepoInterface (Azeem)  
  * More checks now use RepoInterface APIs  
  * Working on replacing Active check  
* Scorecard checks (Laurent)  
  * Score-based result integrated, some nits to fix. Divide by 0 blocking cron job  
* Naveen  
  * Active check using graphQL \- [https://github.com/ossf/scorecard/issues/639](https://github.com/ossf/scorecard/issues/639)  
  * Scorecard V2 update: [https://github.com/ossf/scorecard/issues/711](https://github.com/ossf/scorecard/issues/711)  
  * [https://github.com/ossf/scorecard/issues/630](https://github.com/ossf/scorecard/issues/630)  
  * Removed gitcache  
    

Open issues:

* Weekly cron job blocked on [https://github.com/ossf/scorecard/issues/748](https://github.com/ossf/scorecard/issues/748).  
* Constantly breaking e2e tests \- [https://github.com/ossf/scorecard/issues/741](https://github.com/ossf/scorecard/issues/741)  
  * A 20%-er who might take up mocking RepoClient  
  * Maintain some fake repos like “ossf/fakerepo” for testing  
* Code-Review and PullRequest merge \- [https://github.com/ossf/scorecard/issues/731](https://github.com/ossf/scorecard/issues/731)  
  * everyone agrees?  
* CI-Test check looks for CI test, ie whether some presubmit checks are run. I'm not sure we still need it. no other checks we already look for specific CI tests, like CI-Fuzz, SAST, etc. We should be looking for the corresponding config for other CI systems and add it to other checks. The check, as implemented today, is mostly informational.  
* Table does not understand \\n. \-\> need fix.  
* Show check description/remediation steps in new command \--explain or \--show-details  
* Need monitoring of check errors in cron job to validate/refine check implementation. Relevant previous PR [\#597](https://github.com/ossf/scorecard/pull/597)  
* Frozen-Deps \-\> Pinned-Dependencies  
  * Ok for everyone?  
* Use graphQL for SecurityPolicy check [https://github.com/ossf/scorecard/issues/753](https://github.com/ossf/scorecard/issues/753)

## July 12, 2021

Attendees:

* David A. Wheeler (Linux Foundation)

Updates:

* Cron job/performance (Azeem)  
  * Update to use graphQL   
* RepoInterface (Azeem)  
  * Same as above  
* Scorecard checks (Laurent)  
  * Started a PR for adding check details (why check is passing/failing \+ remediation) \- \#650.  
  * Add better error handling. Will write a doc for error handling guidelines.   
* Gitcache efficiency (Naveen)  
  * Code doesn’t seem to be using the Gitcache.   
  * We can remove it. Naveen to work on it.   
* Misc (Naveen):  
  * Working on a graphQL PR for archived repos.   
  * Fixed some bugs  
* Dependabot PRs (Naveen)   
  * No update  
* Remediation (Chris)

Open Issues:

* How should Scorecard results be interpreted?  
  * Oliver: we can have categories like \- critical, ideal to have etc.   
  * David: CII badge has 3 levels: passing, silver, gold. Having passing in itself is a bug indicator of secure practices being followed.   
  * [https://bestpractices.coreinfrastructure.org/en/criteria/0](https://bestpractices.coreinfrastructure.org/en/criteria/0)  
  * David: might be useful to think of this in terms of risk.   
  * Azeem: merge both ideas and have severity categories and risk scores.  
  * Oliver: checks do not currently have threat models. Having this may help categorize checks.  
* Remediation and Error details:  
  * [https://github.com/ossf/scorecard/issues/629](https://github.com/ossf/scorecard/issues/629)  
  * [https://github.com/ossf/scorecard/pull/662](https://github.com/ossf/scorecard/pull/662)  
  * [https://github.com/ossf/scorecard/pull/650](https://github.com/ossf/scorecard/pull/650)   
* Scorecard releases \- [https://github.com/ossf/scorecard/issues/651](https://github.com/ossf/scorecard/issues/651)

## June 28, 2021

Attendees:

* David A. Wheeler (Linux Foundation)  
* Naveen Srinivasan  
* Azeem Shaikh  
* Oliver Chang  
* Laurent Simon  
* Matt Rutkowski (IBM)

Updates:

* Fixed integration test issues (hooray\!)   
* Cron job/performance (Azeem) \- not much of an update. Still planning to scale up eventually.  
* RepoInterface (Azeem) \- will change interface for get files list, etc., so it’s not GitHub-specific.  
* Scorecard blog post is about to come out.  
* Scorecard checks (Laurent)  
  * https://github.com/ossf/scorecard/pull/611  
  * Working to detect pipe-to-shell as a security concern  
  * Developers might use files other than .txt, e.g., looking for other information.  
  * Hashing itself of just a package in Python doesn’t lock all the transitive dependencies necessarily. No way to detect that just with hashes. Need to check that out. E.g.., Hash pinning of all transitive dependencies is fine.  
  * Python: requirements.txt doesn’t have this information. It was never designed to handle locking, it doesn’t enforce it, you have to do it yourself. Perhaps get people to use pipenv instead, it supports lockfiles.  
  * David W: I suggest that the goal is simply “if you’re using Python you use something that locks transitive dependencies” & allow any reasonable solution. E.g., support:  
    * [https://python-poetry.org/](https://python-poetry.org/) \- poetry uses pyproject.toml and poetry.lock files. Presencee of “poetry.lock” says there IS a package lockfile for Python.  
    * pipenv uses Pipfile and Pipfile.lock. Again, presence of Pipfile.lock says there are package locks.  
    * pip-tools provides pip-compile and pip-sync commands. Here, requirements.in lists your direct dependencies, often with loose version constraints and pip-compile generates locked down requirements.txt files from your .in files. More complex \- need to check if the requirements.txt file is generated with transitive locks.  
    * If you just have requirements.txt, but don’t meet any of the other requirements, then you have Python that is NOT dependency locked & that’s a risk.  
    * [https://stackoverflow.com/questions/52665596/equivalent-of-package-json-and-package-lock-json-for-pip](https://stackoverflow.com/questions/52665596/equivalent-of-package-json-and-package-lock-json-for-pip)  
    * More info on Python dependency locking:  
      * [https://lincolnloop.com/blog/python-dependency-locking-pip-tools/](https://lincolnloop.com/blog/python-dependency-locking-pip-tools/)  
      * [https://dev.to/fridex/micropipenv-the-one-installation-tool-that-covers-pipenv-poetry-and-pip-tools-3ee7](https://dev.to/fridex/micropipenv-the-one-installation-tool-that-covers-pipenv-poetry-and-pip-tools-3ee7)  
  * Also: Check that there’s a language virtual environment (e.g., Python, Go). There could be some double-checks for consistency to make sure that the package versions are consistent (e.g., someone might not have enabled their virtual environment) \- check that files are consistent with the locks. (E.g., “go install” without pinning)  
* Gitcache efficiency (Naveen)  
  * No update here  
  * Will work on this next week.  
* Remediation (Chris)  
  * No update

Open Issues:

* [Needs discussion issues](https://github.com/ossf/scorecard/issues?q=is%3Aissue+is%3Aopen+label%3A%22needs+discussion%22)  
  * restrict github token for scorecard repo  
* Synk test failures  
  * Removed now.   
* ZeroConfidenceErrors and RetryErrors \-  
  * AI([azeems](https://who.corp.google.com/azeems)): Create issue and assign to Naveen   
* ![][image1]  
    
* BinaryArtifact checks on repos like [https://github.com/Smithsonian/OpenAccess](https://github.com/Smithsonian/OpenAccess)  
* Managing dependabot PRs.  
  * With integration tests fixed, it should be ok to merge.  
  * Setting weekly PRs may skip 0-day vulnerabilities  
  * Review process for dependabot PRs:  
    * Naveen to write a README step-by-step to review PRs.  
* How can we maintain code quality?  
  * Maintain list of fake repos to test the checks on  
  * Emulator for PubSub exist. We can use that.  
  * https://www.testcontainers.org/modules/gcloud/

## June 14, 2021

* Welcome Chris\!  
* Donate your tokens to Scorecard.  
* Improve implementation of CheckIfFileExists and CheckFileContent \- [https://github.com/ossf/scorecard/issues/578](https://github.com/ossf/scorecard/issues/578)  
  *   
* Should we delete Gitcache and HTTP cache \- [dashboard](https://cloud.google.com/monitoring/dashboards/builder/6d72d8f8-4fab-4904-9f29-62484c85d147?project=openssf&dashboardBuilderState=%257B%2522editModeEnabled%2522:false%257D&timeDomain=6h)?  
* curl | bash and other download: new check Non-Pinned-Download or in Frozen-Deps? I think it's part of pinning  
* Integration of 3P tools for custom checks results  
  * Possible options:  
    * Compile out-of-band and add it to the scorecard via go:embed. Easy, but kinda ugly. We won't be able to receive dependabot's PRs.  
    * Create a new github project with a shim around shellsheck, e.g. go-shellcheck-wrapper that we regularly update to use a newer version of shellcheck. This would still require the use of go:embed, I think so still kinda ugly. We won't be able to receive dependabot's PRs.  
    * Have scorecard check that shellcheck is installed. It may be installed on the system or in a dedicated directory, e.g. $HOME/.XXX/ where XXX is the package manager: for shellcheck it's Haskel's cabal. If the tool is installed, run the check, otherwise return an error (and the steps to install) when the check is enabled.   
      * We can provide an option in scorecard to install dependencies, e.g. scorecard \--install-deps=CheckName  
      * Download a binary pinned by hash. We won't be able to receive dependabot's PRs.  
    * For sandboxing, we may need something stronger than have the tool installed on system... but any sandboxing utility will require additional installation for container runtime and setup  
    * Use a docker container as a weak security boundary and to ease installation of 3P tools. As most tools work on files rather than stdin, this option also helps keep workstation clean.  
    * Long-term, if we have many 3P tools we need, we can think of having config files for each 3P  
* Integration tests for cron job \- [https://github.com/ossf/scorecard/issues/510](https://github.com/ossf/scorecard/issues/510)  
* Use regex for quick (though imperfect) analysis, or do deeper AST analysis (which requires a lot more to get going)? Probably start with some regexes  
* Updates/blockers  
  * Azeem  
  * Naveen  
    * Integration test broken \- not currently clear why. Will investigate this week.  
  * Laurent  
  * Chris  
* How can we get people to make changes to produce higher scorecard results?  
  * David A. Wheeler: The best approach is to make it really easy for the project to accept the change. Turn it into a pull request/merge request. Don’t “just enable whining” \- provide a change that actually implements the change. Some changes aren’t just technical, but are social \- social changes are harder.  
* Scorecard scaling: Good news, expect to add scanning of 30K more OSS projects

## May 24, 2021

* Updates/blockers  
  * Scaling Scorecard (Azeem)  
    * Scorecard dashboard \- [link](https://pantheon.corp.google.com/monitoring/dashboards/builder/6d72d8f8-4fab-4904-9f29-62484c85d147?project=openssf&dashboardBuilderState=%257B%2522editModeEnabled%2522:false%257D&timeDomain=6h)  
    * Ideas for reducing token usage  
    * Cloudbuild  
  * Scorecard rollouts (Naveen)  
    * ok-to-test  
    * Binary artifacts/scorecard UX  
  * Remediation (Laurent)  
* Open issues:  
  * ?  
* Shared calendar  
  * Move to the OpenSSF calendar meet after this meeting

## May 17, 2021

* Open issues needing discussion:  
  * Frozen build/install dependencies \- [bug](https://github.com/ossf/scorecard/issues/429)  
    * Docker file dependencies are left.  
    * Dependabot might support Dockerfile updates.  
  * [GitHub token scopes](https://github.com/ossf/scorecard/issues/432)  
    * We have a solution for Scorecard  
    * Adding a Scorecard check to do this might be a future improvement to do.  
  * [Ok-to-test action](https://github.com/ossf/scorecard/issues/316)  
    * Naveen to take this up  
  * [Binary artifacts in source repo](https://github.com/ossf/scorecard/issues/400)  
    * Up for grabs  
  * Scorecard UX \- [HTML](https://github.com/ossf/scorecard/issues/342), [GitHub Actions](https://github.com/ossf/scorecard/issues/430), [SARIF/CI integration](https://github.com/ossf/scorecard/issues/193) , [badges](https://github.com/ossf/scorecard/issues/271)  
    * Up for grabs

## May 10, 2021

* Discuss Scorecard and Remediation collaboration  
  * Attendees:  
    * Azeem  
    * Jeff  
    * Abhishek  
    * Laurent  
    * Oliver  
* API \- db or lib  
  * Remediation will run on private repositories, so cannot use Scorecard DB. Will use Scorecard as a library. AI is to confirm that the available Scorecard API is enough to unblock Remediation work.  
  * Remediation clients will input a policy/config which will be used to run customize and run Scorecards. The schema of this config is TBD and will decide how granular Scorecards API/lib should be to support the Remediation usecase.  
  * Remediation requires support to run checks on local files using Scorecard. Need development effort on the Scorecard side to enable this.   
* CI/CD system  
  * If Scorecard can be enabled as a GitHub action, it is easy for Remediation to use it.  
* List of checks/priority  
  * AI: Figure out milestones to be reached in terms for new checks to be added or improving check accuracy.

## May 10, 2021

* Discuss Scorecard and Cosign/Sigstore/BinAuth  
  * Attendees:  
    * azeems@  
    * dlorenc@  
* BinAuthz  
  * Enforces policies on what to run on Kubernetes.  
  * Not very well developed yet. Maybe wait for v2, if the issue is not critical.  
* Sigstore/Cosign  
  * Provides an easy way for developers to sign their images and store both image and signature in the same registry.  
  * Developers publishing binaries/images directly could easily adopt Cosign to make installing and running these binaries more secure. But, this is upto the developer publishing binaries, does not help our case.  
* Rollout policies/guidelines   
  * Using Cloud Build to do automated rollout might be the safest/sanest option available, without maintaining more services.  
  * No way to restrict local binary pushes.  
  * For floating point dependencies: provide a “make install” command which downloads all dependencies pinned to a certain hash/digest.

## May 6, 2021

* Discuss Scorecard rollout, non-pinned dependencies  
  * Attendees:  
    * azeems@  
    * laurentsimon@  
    * ochang@  
* Oliver: maybe not always push master branch?  
* Oliver: tag commits that we want GCR to pull.  
* Any way to avoid manual pushes? GCP BinaryAuth maybe?

## Apr 26, 2021

* Intros  
* Focus areas for contributors  
  * AI([azeems](https://who.corp.google.com/azeems)): Create issues to discuss  
  * Some open issues:  
    * [https://github.com/ossf/scorecard/issues/78](https://github.com/ossf/scorecard/issues/78)  
    * Sigstore  
    * Test and deploy   
    * Check quality [https://github.com/ossf/scorecard/pull/347\#discussion\_r616364482](https://github.com/ossf/scorecard/pull/347#discussion_r616364482)  
* OpenSSF TownHall \- [https://docs.google.com/presentation/d/1s4Y5pr3drWlx29-Lkio7zWkegJzugWQCqplvV3GiIH0/edit?pli=1\#slide=id.p](https://docs.google.com/presentation/d/1s4Y5pr3drWlx29-Lkio7zWkegJzugWQCqplvV3GiIH0/edit?pli=1#slide=id.p)   
  * Try in Q3  
* Snyk test

[image1]: <data:image/png;base64, >