Conda ecosystem support #932
Labels
Comments
6 tasks
|
I'll be opening a Draft PR when I have more done in my syft fork for conda support |
|
Thanks @rigzba21 for the attention here! Admittedly I don't have a lot of experience with the Conda ecosystem, but I'm excited to see the PR and learn more about the different constructs a cataloger might have to interact with to provide support. Feel free to tag the tools team on the PR and we can help with getting the builds kicked off for CI. I've also tentatively added this as a topic to our next community meeting on Link to the community meeting where we discuss new ideas/features for syft/grype: |
|
Yep, thanks @rigzba21, I had thought a bit about trying this before but never got around to it. One thing that will be interesting here is syft already identifies packages installed by conda as pip packages because they end up with the same METADATA entries per https://github.com/anchore/syft/blob/main/syft/pkg/cataloger/python/parse_wheel_egg_metadata.go. There is extra json for conda installed stuff that isn't taken into account though. Anyways, ths one will definitely be interesting and a great addition I think! |
|
@spiffcs, I've joined the Slack channel and added the community meeting to my calendar. @westonsteimel, thank you for pointing me to a good starting point! |
|
Closing my old draft PR which attempted to add a separate conda cataloger, in favor of the smaller-scoped work of extending the python cataloger as discussed in the last community meeting notes Conda Support
|
|
Is Conda support still being looked at? In our images we have Syft only finding openssl that the base image had installed and not the one in the Conda custom environment Syft only finds: |
While I'd love to keep working on this (admittedly, I have not looked at this in a while), I don't currently have the time to focus on this. |
|
I'll have time to pick this back up again this quarter 😄. |
|
I'd be interested in helping get this across the finish line, if another body would help. I should have some time coming up in early June that I can work on this, if this is still happening? Is there a list of specific things that still need to be fleshed out and finished? |
|
Hi @razzlestorm! When I started this a while ago, I was looking at potentially adding a new cataloger . I also see that this issue conveniently has a new label, "new-cataloger." 😄 I'd love to collaborate on this now that I have more free time. |
|
FWIW -- it looks like there were some open questions from the April 14 community meeting: https://docs.google.com/document/d/1ZtSAa6fj2a6KRWviTn3WoJm09edvrNUp4Iz_dOjjyY8/edit#heading=h.3ybu7ev6q4bx |
|
@razzlestorm @rigzba21 Thanks for keeping this thread alive! If you have cycles this summer for looking at this cataloger then absolutely we would love the help: The document Keith linked should have our OSS meeting on it - we have one this Thursday if you want to come hang out and talk about the scope of what would be a good first pass. Otherwise happy to talk async on this thread to get the ball rolling =) |
|
Definitely! I'll come hang out. EDIT: Unfortunately, I won't be able to make this, sorry! Apparently a last-minute meeting has been scheduled that I need to attend and the only time slot is that time. I've marked these slots as already being booked but hey, what can you do? I'm happy to coordinate async after your meeting, and make the next one (it's already booked in my calendar and hopefully nobody will unbook it). In general I've had a brief chance to look over things, and it does seem like it makes sense to extend the python cataloger rather than creating a new one just for .conda packages. I'll take a look at everything in a bit more in-depth over the long weekend here. |
|
I am linking Contributing New Catalogers - A Guide for Hacktoberfest and Beyond. Thank you, @spiffcs, for the writeup! I have some time this Nov/Dec to pick back up and look at this again. |
I'm new to this project, but I don't see why conda would be part of the python cataloger. Conda can be used for python packages, but it is really a user-space alternative to OS-level packaging and it is primarily useful because it can manage shared libraries and binaries alongside things like Python or R packages that depend on them. |
|
Recently, I got involved in Anaconda license topic, so I am checking it. So, if we can use Syft to get the list of packages managed by Conda and where they come from (like with |
|
Here's a list of open questions from the community meeting referenced above:
Here are some answers from Jonathan Velando from that same doc:
For folks looking for an artifact to use for a basis for exploration: I can confirm that And there is some pretty rich info present: Example JSON from /opt/conda/conda-meta/psutil-5.8.0-py38h497a2fe_1.jsonI think this heavily hints at:
|
|
From the live stream: we want to not affect existing grype matches today. Let's let syft find both overlapping python and conda packages and mark the ownership-overlap relationship between them. When we start supporting matching for conda explicitly in grype we can decide to dedup matching at that point. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi all!
What would you like to be added:
Conda ecosystem (language agnostic) support
Why is this needed:
The conda ecosystem provides an amazing way to work with dependencies in terms of environments and is used heavily in the Data Science and Scientific Computing communities.
Additional context:
I'm happy to submit a PR for this feature/capability and looking forward to diving into the syft src!
The text was updated successfully, but these errors were encountered: