215 questions
0
votes
0
answers
16
views
Java Heap Space in PySpark while running using Rapids GPU
I have an issue when running a Spark submit. I'm not sure whether the problem lies in my Spark script or my resources (but I suspect it's related to the resources).
I have 9 main processes:
A ...
0
votes
2
answers
79
views
how do I append the output of a dask_cudf apply function to the original dask_cudf?
I am applying a function (e.g. letter frequency) to a dask_cudf dataframe that consists of a single column of words of fixed length.
I am trying to merge the output or append the output into the ...
0
votes
0
answers
35
views
How to migrate WSL2 to Jupyter Notebooks for cuDF?
I finally successfully installed rapids onto my WSL2 for the purpose of using cuDF to multithread my GPU in python, so now the remaining task is to be able to use a Jupyter Notebook instead of the ...
- 43
0
votes
1
answer
35
views
cudf ModuleNotFoundError after installing RAPIDS in MSL2
I successfully installed RAPIDS in my WSL2 window, but importing cudf is not working. According to the RAPIDS website https://docs.rapids.ai/install#conda , as well as some other sources, I should be ...
- 43
0
votes
2
answers
1k
views
cuDF installation failure: "Didn't find wheel for cudf-cu12 24.6.1"
I'm trying to install cuDF to gain the glorious powers of multithreading my NVIDIA GPU in my python scripts. However, I can't install cuDF.
To install cuDF, i went to https://docs.rapids.ai/install ...
- 43
0
votes
0
answers
100
views
Rapids Cudf library not fast enough
I have a server with 1xNvidia T4 and a 32 Cores CPU. Trying to migrate from pandas to cudf.
I am performing multiple row wise transformations on my data but I could find the CPU computing faster than ...
- 1
1
vote
0
answers
99
views
DbScan from sklearn vs Dbscan from Rapids
I'm trying out the Rapids version of DbScan that works with GPUs.
I wrote a short code in Python that finds clusters in 3D random generated points, and when I was playing with the code, I realized ...
- 493
0
votes
1
answer
125
views
Using %load_ext cudf.pandas throws AttributeError
I am trying to use cudf.pandas on a notebook on Kaggle and running into a long error message when enabling GPU on GridSearch. The main issue being an AttributeError on a DatFrame.
The code works fine ...
- 23
1
vote
0
answers
327
views
"fatal error: spdlog/fmt/bundled/core.h: No such file or directory" when trying to build executable for libucdf program
I am trying to write a basic program to read a csv into a cudf with libcudf. I have installed libcudf by installing the RAPIDS conda environment. I am trying to use this command to build my executable:...
1
vote
1
answer
204
views
Reading full image with RAPIDS cucim
I'm trying to understand how to read an image as an array using cucim. My image is a float64 .tiff with shape=(657, 668) from a CT-scan.
The closest function I could find was read_region (I couldn't ...
- 361
0
votes
0
answers
116
views
How to Distribute Dask-CUDA Workload Across Multiple GPUs?
I'm working on a project where I need to evenly distribute data processing tasks across multiple GPUs using dask_cudf. Despite my current setup, the workload seems to be handled by only one GPU. I'm ...
2
votes
1
answer
74
views
How to convert cuGraph directed graph to undirected to run MST?
I'm trying to build MST from a directed graph by converting it to an undirected one. I followed cuGraph example here but getting NotImplementedError: Not supported for distributed graph.
I tried doing ...
- 23
2
votes
1
answer
291
views
trying to install cudf but cannot find a version that matches with pd what should be the version of pandas and cudf or where can i find the doc
(base) mitul@pop-os:~$ work
(rapids-24.02) mitul@pop-os:~$ python
Python 3.10.13 | packaged by conda-forge | (main, Dec 23 2023, 15:36:39) [GCC 12.3.0] on linux
Type "help", "copyright&...
0
votes
1
answer
390
views
A way to use Nvidia Rapids in windows directly?
I would Like to know that is there a way to install Nvidia Rapids directly in windows 11 and use rather than using it with wsl2 or docker? Or is there a way to connect jupyter lab host to dataspell?
I ...
1
vote
1
answer
198
views
tSNE cuML taking a long time to run/ not running but returning no error
I recently installed RAPIDs in an conda environment using WSL2 on my Windows laptop. I used the following command to install it:
conda create --solver=libmamba -n rapids-24.02 -c rapidsai -c conda-...
- 65
0
votes
0
answers
132
views
Dask Dataframe using memory from a single GPU instead of all available in the cluster
I have a script running on an EC2 instance that reads vector embeddings from s3 and dumps them into a list variable; from there, it creates a dask dataframe that will be used in a Dask KMeans ...
0
votes
0
answers
143
views
std::bad_alloc: out_of_memory: CUDA error
I have this code:
import cudf
import cuml
import cupy as cp
from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split
from cuml.model_selection import ...
- 1
0
votes
1
answer
298
views
cuML UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan
i am trying to train a RF regression using gridsearchcv. I change all file types to float32 and i still get these warnings that i'm not sure how to solve.
my code:
combined_df=cpd.concat([train_df,...
- 10.2k
0
votes
1
answer
280
views
Explain Dask-cuDF behavior
I try to read and process the 8gb csv file using cudf. Reading all file at once doesn't fit neither into GPU memory nor into my RAM. That's why I use the dask_cudf library. Here is the code:
import ...
- 734
0
votes
0
answers
101
views
CUDA FileNotFoundError: [Errno 2] No such file or directory: when using 'import cudf' WSL2, Jupyter Notebook
I am attempting to run cudf in a Jupyter Notebook in a conda rapids23.12 environment. I painstakingly went through all the steps to install the necessary drivers and packages, however when I run '...
0
votes
1
answer
775
views
Stuck with cudf in WSL2 - Installed all dependencies but after loading it, its not working at all
Im using WSL2 and for some reason cudf.pandas is not using my gpu at all, in fact it stays at 0% usage with large datasets. I've installed rapids and cuda running the following codes succesfully:
%...
1
vote
1
answer
356
views
I'm having trouble installing rapids.ai on my windows 10 desktop
I have installed WSL2. I have tried both the pip installation and conda installation.
i'm getting a package not found error: rapids=23.10*
I'm completely lost and am hoping someone will guide me ...
0
votes
0
answers
42
views
which is the correct version of akka-actor-typed to use with Spark 3.5.0?
I keep receiving this error,
"Exception in thread "main" java.lang.NoClassDefFoundError: akka/actor/typed/ActorRef",
when I try to deploy my application.jar in one active Spark ...
- 16
0
votes
2
answers
522
views
TypeError in cudf.pandas
I am using the Rapids library for Pandas. On top of my code:
import cudf.pandas
cudf.pandas.install()
import pandas as pd
I'm using https://clickhouse-driver.readthedocs.io/en/latest/ and client....
-2
votes
2
answers
2k
views
rapids cannot import cudf: Error at driver init: Call to cuInit results in CUDA_ERROR_NO_DEVICE (100)
To install RAPIDS, i have already installed WSL2.
But i still got the following error when import cudf:
/home/zy-wsl/miniconda3/envs/rapids-23.12/lib/python3.10/site-packages/cudf/utils/_ptxcompiler....
- 771
0
votes
0
answers
75
views
How to use RAPIDS module and MetaTrader5 module in the same python project
I want to use the RAPIDS and MetaTrader5 modules in the same project in a Windows 11 environment.
The RAPIDS module requires the use of WSL2 in Windows 11, and the MetaTrader5 module needs to be used ...
- 1,026
0
votes
1
answer
706
views
Solving environment: failed when install RAPIDS using conda
In order to install RAPIDS, I get the command from the site below and run it, but the following error occurs.
https://docs.rapids.ai/install
conda create --solver=libmamba -n rapids-23.08 -c rapidsai -...
- 1,026
1
vote
0
answers
542
views
Can I run the cuML RandomForestClassifier with a sklearn MultiOutputClassifier wrapper on a dataset for a multilabel classification task?
I am currently working on an Excel datasheet where I have rows with several features for which I want to predict multiple labels.
The features are either 0 or 1 because they describe whether ...
- 13
0
votes
1
answer
237
views
Multiple Spark Executors on single GPU
We are trying to improve the Spark Job processing performance by introducing GPUs to the nodes. But after enabling Spark3 with GPUs we are seeing downtrend in spark job performance, due to limited ...
- 1,202
0
votes
0
answers
103
views
Feature Selection, Outlier Removal, Target Transformer for Dask-ML pipelines
While FS, OR, TT have well-established components in "classic" scikit-learn pipelines, documentation of dask-ml and RAPIDS totally omits them.
What are the best practices to implement ...
- 2,370
2
votes
1
answer
3k
views
RAPIDS pip installation issue
I've been trying to install RAPIDS in my Docker environment, which initially went smoothly. However, over the past one or two weeks, I've been encountering an error.
The issue seems to be that pip is ...
- 320
0
votes
1
answer
474
views
NVidia Rapids: Non-Euclidean metric in cuml UMAP
I am trying to use GPU (A100) to perform UMAP for speedup. I am facing problem as Euclidean metric does not seem to work for me at all but correlation/cosine are promising. However, the code I am ...
- 1,144
1
vote
1
answer
879
views
How to parallel GPU processing of Dask dataframe
I would like to use dask to parallelize the data processing for dask cudf from Jupyter notebook on multiple GPUs.
import cudf from dask.distributed
import Client, wait, get_worker, get_client
from ...
- 31
0
votes
1
answer
885
views
Why can't I install cuML on wsl?
Installing rapids and cuML is not working
I have Cuda installed (Cuda compilation tools, release 11.8, V11.8.89) in a Python 3.10.11 env in JN in VSC on wsl2 on a desktop running Windows 11, with the ...
0
votes
1
answer
40
views
Troubleshooting cudf.tokenize(): 'Length Mismatch' error with non-space delimiters
Cudf Tokenize Element Length Mismatch
This is the expected result for tokenize(' ') on space character:
0 Due
0 to
0 being
0 ...
1
vote
1
answer
75
views
NVidia Rapids filter neither works nor raises warn/errors
I am using Rapids 23.04 and trying to select reading from parquet/orc files based on select columns and rows. However, strangely the row filter is not working and I am unable to find the cause. Any ...
- 1,144
1
vote
2
answers
1k
views
Google Colab: cannot install cudf
I need help. I am using Google Colab with Python 3.10.11 and I have a Colab with CUDA Version: 12.0 , Nvidia driver version 525.85.12 and I am following this tutorial on how to install cuDF https://...
- 55
1
vote
1
answer
256
views
RAPIDS cuML linear regression running slower than statsmodels.api equivalent?
This is my first time posting on here so my apologies if this is the wrong place to ask or if I'm missing info. Basically I have the following code for a linear regression model using statsmodels and ...
- 11
0
votes
1
answer
86
views
Rapidsai (DGA Streamz): ERROR- module dask has no attribute distributed
I have been trying to run the dga detection streamz on the rapidsai clx streamz docker container for the last few days without any resolution.I'm following the instructions on the rapids website: ...
- 5
2
votes
0
answers
200
views
how to convert 'dask_cudf' column to datetime?
How can we convert a dask_cudf column of string or nanoseconds to a datetime object? to_datetime is available in pandas and cudf. See sample data below
import pandas
import cudf
# with pandas
df = ...
- 2,304
0
votes
0
answers
578
views
create conda enviroment with cuml and tensorflow-gpu dependencies got error
I try to create conda env with the following environment.yml file
name: myenv
channels:
- rapidsai
- conda-forge
- nvidia
dependencies:
- python=3.10
- cudf=23.04
- cuml=23.04
- cugraph=...
- 353
0
votes
1
answer
275
views
Install older version of Rapids AI using Docker
how do I install an older version of Rapids AI using Docker such as 22.06? The newest version 23.02 doesn't work on any VAST AI (https://vast.ai/) machine.
- 95
0
votes
0
answers
282
views
Latest version of RAPIDS cuML in Kaggle notebooks
First of all, I am fairly new to running models on GPU, so sorry in advance for stupid questions.
I use RAPIDS cuML to GPU-accelerate some algorithms, but I noticed I cannot use the latest version (23....
- 1
1
vote
0
answers
272
views
dask_cudf/dask read_parquet failed with NotImplementedError: large_string
I am a new user of dask/dask_cudf.
I have a parquet files of various sizes (11GB, 2.5GB, 1.1GB), all of which failed with NotImplementedError: large_string. My dask.dataframe backend is cudf. When the ...
- 1,208
1
vote
0
answers
192
views
dask_cudf dataframe convert column of datetime string to column of datetime object
I am a new user of Dask and RapidsAI.
An exerpt of my data (in csv format):
Symbol,Date,Open,High,Low,Close,Volume
AADR,17-Oct-2017 09:00,57.47,58.3844,57.3645,58.3844,2094
AADR,17-Oct-2017 10:00,57....
- 1,208
0
votes
1
answer
103
views
Correctly zipping two columns with different data types in cuDF
I have the following DataFrame in cuDF:
Context Questions
0 Architecturally, the school has a Catholic ...
- 400
0
votes
1
answer
188
views
Using Rapids with kmeans imputation in python
I was just wondering if anyone has been able to successfully use rapids with knn imputation. I know cuml.impute was avaliable perviously but is seems like it has not been removed. If anyone has a ...
1
vote
1
answer
936
views
RuntimeError: Cluster failed to start with dask LocalCudaCluster example setup
I am new to Dask and I run into problems when executing the example code:
from dask.distributed import Client
from dask_cuda import LocalCUDACluster
cluster = LocalCUDACluster()
client = Client(...
0
votes
1
answer
249
views
How do I use RAPIDS on python if it is not working
Hi guys I am trying to acces gpu accelerated regression trees through cuml from rapids. However I can't seem to be able to download rapids as instructed> I am trying to dwonload through anaconds ...
3
votes
2
answers
4k
views
RAPIDS installation issue
I have executed the pip install commands from RAPIDS into Google Colab ipynb jupyter notebook. They are:
pip install cudf-cu11 dask-cudf-cu11 --extra-index-url=https://pypi.ngc.nvidia.com
pip install ...
- 33