close
Skip to content

Import data from OpenSSF Malicious Packages - #1412

Closed
shravankshenoy wants to merge 7 commits into
aboutcode-org:mainfrom
shravankshenoy:feat/1409/create-openssf-importer
Closed

Import data from OpenSSF Malicious Packages#1412
shravankshenoy wants to merge 7 commits into
aboutcode-org:mainfrom
shravankshenoy:feat/1409/create-openssf-importer

Conversation

@shravankshenoy

Copy link
Copy Markdown
Contributor

Fixes #1409

Changes Made

Created OpenSSF importer openssf.py and added files for testing the importer

Results

Ran the importer locally using command below with debug mode on
docker-compose exec vulnerablecode ./manage.py import vulnerabilities.importers.openssf.OpenSSFImporter

Importer ran successfully and imported 17182 advisories as can be seen in the image below

image

Other Consideration

  1. Initially I named the file ossf.py but it was tantalizingly close to oss_fuzz.py and could cause confusion, hence changed the name to openssf.py. Let me know if any better name is possible
  2. Out of the 4 ecosystems in the repository (crates.io, npm, pypi, rubygems), univers supports 3 (npm, pypi, rubygems), but I am not sure if it supports crates.io
  3. For the time being, I have added the 4 ecosystems in the list, but if required I can create a separate classmethod called supported_ecosystems and add all the ecosystems there as a list/dictionary, similar to how it is done in Vulntotal. Do let me know what is best approach
@classmethod
    def supported_ecosystem(cls):
        return {           
            "pypi": "PIP",
            "gem": "RUBYGEMS",
            "npm": "NPM",           
        }
  1. Have add a try-except block in for loop. Let me know if that has to be removed.

Signed-off-by: Shenoy <shravanshenoy1998@hotmail.com>
Signed-off-by: Shenoy <shravanshenoy1998@hotmail.com>
Signed-off-by: Shenoy <shravanshenoy1998@hotmail.com>
Signed-off-by: Shenoy <shravanshenoy1998@hotmail.com>
Signed-off-by: Shenoy <shravanshenoy1998@hotmail.com>
Signed-off-by: Shenoy <shravanshenoy1998@hotmail.com>

@TG1999 TG1999 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@shravankshenoy Thanks++, added some comments for your consideration

Comment thread vulnerabilities/importers/openssf.py Outdated
@@ -0,0 +1,9 @@
{
"aliases": ["MAL-2023-1077"],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be used as an alias? What other options we have?

cc @pombredanne

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@TG1999 yes, as an alias sounds fine

@mjherzog mjherzog changed the title Import data from OpenSSF Malacious Packages Import data from OpenSSF Malicious Packages Feb 5, 2024
Signed-off-by: Shenoy <shravanshenoy1998@hotmail.com>
@shravankshenoy

Copy link
Copy Markdown
Contributor Author

I have made the changes that had been requested. Let me know if anything else needs to be done.

@shravankshenoy

shravankshenoy commented Feb 12, 2024

Copy link
Copy Markdown
Contributor Author

While working on PRs #1405 and #1417 I realized that in this PR I have not followed the purl-spec properly. For example crates.io should be mapped to cargo and rubygems should be mapped to gems. I will work on making these changes.

@TG1999
TG1999 requested a review from ziadhany July 9, 2024 15:39
@TG1999

TG1999 commented Jul 22, 2024

Copy link
Copy Markdown
Contributor

@shravankshenoy Thanks! PR can not be merged in this state, I tried to run the importer and it's throwing errors. Please run the importer and improver once.

@TG1999 TG1999 closed this Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add malware from https://github.com/ossf/malicious-packages

3 participants