-
Notifications
You must be signed in to change notification settings - Fork 104
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Should we endorse Pixi? #615
Comments
I've just started experimenting with pixi and there are definitely a lot of things to like. The biggest issue that I've seen so far is just the difference in workflow/philosophy from conda. Pixi is directory-based, whereas conda/mamba has a global registry of environments. With conda I use the same environment for several projects, and will also do stuff like work in a scratch directory to fire up some jupyter-lab instance to do some quick analysis, or just open up a terminal, run Pixi requires you to I think one thing to ask is if there is a world where you could use pixi to generate lock files and those lock files would be usable directly from conda/mamba to build environments. Then pixi could be used in place of conda-lock, without the requirement of adopting the full pixi workflow. |
Thanks @synapticarbors for the feedback, it's really valuable, especially since I haven't found the time to investigate these details myself! Regarding your question, someone asked something very similar here and received and answer. In order to get the x-link, the answer directs here: prefix-dev/pixi#800 |
I'm trying to develop some ideas for how to proceed with this, while being honest about my lack of time for maintenance. README and migration guideI propose adding a notice to the README, something like:
We might also consider some sort of migration guide separate from the README and make roughly the following points:
Missing featuresThen there's the question of what functionality is missing in the gap between pixi and conda-lock? Some things that come to mind:
What should we do?It's plausible that this one or both of these features arrive soon in pixi. On the other hand, given what we currently have in conda-lock, I wonder if we could quickly provide solutions ourselves.
I'm very curious what people think, and if anyone's willing to tackle any of this? |
Just as a note, I recently wrote a prototype tool to render a I've been talking to some of the pixi devs about developing it into the start of the |
I think in general, having a call-out for Alternatives would be sufficient, probably with ❌/✅ feature table. For example, As to integrating directly with I don't know if re-implementing the logic for its
This looks great as inspiration, and Otherwise: I have found the wheel-related stuff to be underwhelming (and the magic it does vs
Before adding anything here, or copy pasting any code into new packages, perhaps these morsels could be fully consumed. |
Thanks so much @bollwyvl for your input!!! I'm struggling a bit to unpack everything. I'd really appreciate it if you could help me understand your viewpoint.
Why do you prefer Python-only code? Is this a security/auditability thing?
In my head I've actually been envisioning I'm confused by the "Otherwise: ..." paragraph about conda-constructor functionality. It seems like you wish there were a CLI interface for extracting info, but I don't understand how this relates to
I've been thinking that py-rattler may be a good way to interface with the underlying data structures from Python.
Which morsels, and what specifically do you mean by "consumed"?
I really love this idea, and I wish I had the time to do it myself. Here I get what you're saying, so I can explain it for readers who are missing context. This is basically a complaint that |
The doorhandle counts. The recommended install option (
If
Move everything other than bog-standard, no-magic-comments, multiple Within this repo, have a folder structure like...
The top-level |
As someone who just spent some time today experimenting with a migration from conda-lock to Pixi for a project, I just wanted to give some feedback that my biggest motivation was because Pixi does minimal updates to lockfiles. conda-lock not supporting this (#370) is one of the biggest pain points I have in using it. |
Yes @jayqi, I agree that the minimal updates are very nice. Can you provide any feedback regarding the actual migration process? For example, did you encounter any annoyances when transitioning from an environment-based workflow to a project-based workflow? It'd be great to be able to provide a migration guide. |
It's worth noting that pixi is just a binary. The script at https://pixi.sh/install.sh downloads it and puts it on the path. That's it. You can easily do that manually from their releases page. It's probably a matter of opinion but I am totally fine with pixi or similar being the "root" that installs Python+pip instead of the other way around. Also, the concern about controlling where environments are stored that @synapticarbors raised is, I think, completely resolved by pixi's detached-environments feature. Personally I would be in favor of a notice like @maresb proposed. |
Thanks a lot @zmbc for the input! I was looking at py-rattler and it seems to me like most if not all the things we'd need to reimplement conda-lock are already there and wouldn't even require a pixi binary. |
I'll preface all of my comments with: any new-start project I've started recently is based on
Kind of the whole point of using a tool like For some more context raised on their repo: prefix-dev/pixi#1765
... still creates an unpredictable environment directory prefix, which then, clearly, can't then be moved.
But then would > mamba create --dry-run -n foo python | wc
53 177 2885
> mamba create --dry-run -n foo "conda-lock=0.*" | wc
79 332 5029
> mamba create --dry-run -n foo "conda-lock=1.*" | wc
119 572 8947
> mamba create --dry-run -n foo conda-lock | wc
120 584 9057 |
I just generated a This could be considered a first step towards enabling pixi as a backend. |
I am finally giving
|
Thanks a lot @wholtz for the feedback! I really appreciate it. The Prefix team is extremely responsive, and I'm sure they'd also appreciate your feedback. (You can usually get an immediate response on Discord, if you have that.) |
|
@pavelzw - I believe it is true if you have both conda and pypi dependencies:
And the environment does includes python v3.12 via conda-forge. |
With regards to the first point @wholtz you are right! We need to have a python available for any sdists that needs to be built. The best idea to get around this is currently: prefix-dev/pixi#1340 which would be a pretty big feature, but also pretty awesome I think. With regards to authentication we are kinda tied here to what uv supports, that's why there is a PR open to add custom middleware so then we can add any authentication however we like it: astral-sh/uv#3502. For example, to add an environment variable support we would need to find the way that that best matches the uv authentication methods which are listed here for different registries: https://docs.astral.sh/uv/guides/integration/alternative-indexes/. Myself, I don't really like an We could think of allowing environment variables in the toml files something along the lines of In the mean time, wouldn't installing keyring with the backend that you need be an option inside the container? |
While I have not yet had the chance to transition to Pixi in any of my projects, I'm convinced that this is the future of lockfiles in the Conda ecosystem. Moreover, they have a full-time team moving things forwards.
I think that conda-lock will still play a role with supporting a large number of existing projects since the effort of retooling can be hard to justify. But I'd hope that new projects would look primarily at Pixi.
Dually, I'd be very curious to hear from people who have tried both, which things does conda-lock do that are are not yet adequately supported by Pixi?
The text was updated successfully, but these errors were encountered: