All Questions
Tagged with multiprocessing performance
24 questions
1
vote
0
answers
56
views
Bi-Directional A* function in MATLAB
I am currently working on my thesis for 3D path planning involving external elements, and I have completed about 1/5 of the total trials that I am doing. For each 1/5 of the trials I am increasing the ...
- 11
4
votes
2
answers
2k
views
Multi-threading Class in C++
I wrote a C++ class which can take a vector of functions and their respective arguments and execute them in parallel and return a vector of results from its respective functions.
...
- 411
0
votes
1
answer
150
views
Training a Reinforcement Learning algorithm
Problem:
I'm trying to reduce the computation time and optimize specific functions of a Reinforcement Learning algorithm in the training phase and observed that one block of code takes too much ...
- 103
4
votes
1
answer
557
views
Prime factorization algorithm by using the multiprocessing module of Python
I may introduce a Python code of prime factorization which is from my personal project that I'm working on.
...
- 59
0
votes
1
answer
61
views
streaming / low memory groupby in julia
I am working on a groupby-aggregation function that will work without RAM overflow issues. Essentially, I want it to run as fast as possible, while not necessarily loading the entire data structure ...
- 123
2
votes
1
answer
516
views
My code creates a soundboard that can assign a sound to a key on a keyboard while listening to keyboard strokes in parallel
My code works but I am looking for an efficient way to implement the idea with out having to repeat the same code block inside the function Recin() (record Input function is stated in the code) in ...
- 23
2
votes
1
answer
123
views
3
votes
1
answer
261
views
benchmarking requests to localhost
I am writing a benchmarking tool from scratch in Python.
However I can't get the performance of other benchmarking tools like wrk or ...
5
votes
3
answers
176
views
Analysing a Huge Codebase with Python
I've written a Python program to analyse a huge C++ code base in excess of millions of lines of code. The job of the program is simply to search for all C and C++ style comments and extract all the ...
- 1,378
3
votes
2
answers
2k
views
hash image url and if valid store image url
I am trying to get all image urls which are valid and unique.To make sure i wont end up using the same image again i calculate hash of the image and store it in a text file and read the text file ...
- 185
3
votes
1
answer
58
views
Monitor a web page for when a desired price is met
I have a code that keeps checking a webpage with a price on it and when the price meets a number I set it to, the thing is purchased. I am looking for ways to increase the speed of it.
I have added ...
- 31
3
votes
1
answer
576
views
Simple sharding of delimited files to more sophisticated
A while back I came across the issue of having a large delimited file where I wanted to simply parallelize my python code across each line of the file. However, doing so all at once either took up too ...
- 281
2
votes
0
answers
1k
views
Image processing using multiprocessing in Python
I have an image stack. I am trying to use multiprocessing to do some process on each image in stack to get new image, and then bitwise_or with the old image.
Since ...
- 139
10
votes
1
answer
1k
views
Python 3.7 UltimateBruteforcer
So I made a bruteforce program for sha-X hashes and MD5 using wordlists in Python.
I know Python is probably one of the worst languages for this, but it's only for learning purposes.
I think this was ...
- 319
14
votes
2
answers
792
views
Translate nucleic acid sequence into its corresponding amino acid sequence
Goal of the program
The goal of the program is to translate a nucleic acid sequence into its corresponding amino acid sequence. The nucleic sequences
have to be formatted in a specific format called <...
- 588
4
votes
2
answers
465
views
Parsing contents of a large zip file into a html parser into a .csv file
I have some zip files somewhere in the order of 2GB+ containing only html files. Each zip contains about 170,000 html files each.
My code reads the file without extracting them,
Passes the ...
6
votes
1
answer
437
views
Add an edge to a graph to minimize the average shortest path length
This program is used to find the nodes in a grid network, between which, if an edge is added, the average shortest path length of the entire grid reduces by the most.
"Average shortest path length" ...
- 61
4
votes
1
answer
770
views
Creating a graph representing all combinations of 4-bit binary strings
Update: Important notes to understand the problem can be found in the comments under the response of colleague Gareth Rees's.
I have an algorithm that creates a graph that has all representations of ...
6
votes
3
answers
286
views
Class scheduling tool with image output
I have a multilayered dictionary that contains information about classes. I am using this to code an automatic schedule builder that I will eventually add to a separate Tkinter application I made that ...
- 231
4
votes
2
answers
8k
views
Multiprocess, multithreaded read write on a single file
I've got a job to create a library that would be able to support multithreaded and multi process read and write to a single file. On that single file we would store C# models in an array in JSON ...
- 123
7
votes
2
answers
1k
views
Extracting keywords from 3 billion CSV records
I am processing around 3 billion records using this piece of code. It's pretty slow; it would be really helpful if you could suggest better ways to do this.
...
- 363
9
votes
2
answers
4k
views
Mandelbrot fractal with MPI
I'm not very familiar with MPI. I've written this little piece of code to draw the Mandelbrot fractal processing each row of the image in parallel. It works but it's really slow. I've written code to ...
- 454
5
votes
1
answer
258
views
Parallelizing scrypt key-derivation function
To review my use of multiprocessing, I don't think it is at all necessary to understand the algorithm, but it's the scrypt key-derivation function.
This uses ...
- 878
4
votes
1
answer
685
views
Batch program to xcopy from host PC to remote destination with multi-processing
This is a batch program to xcopy from host PC to remote destination with multi-processing.
Kepler, Python-3.x are my environment.
...
- 143