All Questions
140 questions
0
votes
0
answers
29
views
Memory problem when serializing zipped files in PySpark on Databricks
I want to unzip many files in 7z format in PySpark on Databricks.
The zip files contain several thousand tiny files.
I read the files using binary File and I use a UDF to unzip the files:
schema = ...
- 974
0
votes
0
answers
40
views
FTP server - caged
I try to access the FTP server to get the CAGED files. But as an error, it shows that the files are corrupted. When I try to access the message “Corrupt input data”. So I'm unsure whether, in fact, ...
1
vote
0
answers
89
views
Python py7zr extracting .7z archive differently to Linux command line 7zip
I have created a .7z archive using the usual basic Windows UI. It is my understanding this defaults to relative paths for any archives created. When looking in the archive post creation, all I see is ...
- 4,102
0
votes
0
answers
36
views
How to efficiently extract individual files from 7z archives efficiently in Python? [duplicate]
I have downloaded 424 files from Nexusmods using a Python script I wrote. They have a total size of 21.5GiB and they are all either .zip, .rar or .7z files (some .zip files are renamed to .dazip files)...
- 2,984
0
votes
1
answer
237
views
Reading a multi-part 7z file using Python is failing due to memory issues
I am using a loop to read multi-archive 7z files with this code.
import py7zr
import multivolumefile
zip_path = f"{ARCHIVE_PATH}/test.7z"
with multivolumefile.open(zip_path, mode='rb') as ...
- 1,233
0
votes
1
answer
104
views
How to run a one-off Python unzipping script (19Gb .7z archive) on GC?
I need to extract files from the archive on a one-off basis. The Python code that can handle the task is very simple:
from py7zr import unpack_7zarchive
import shutil
shutil.register_unpack_format('...
- 3,549
0
votes
0
answers
234
views
How to import external library py7zr into AWS Glue Job?
I am working with AWS Glue. A 7z file must be extracted from S3 and subsequently transformed and loaded the lines of a resulting .txt into an RDS database. I need to use the py7zr library to be able ...
1
vote
0
answers
93
views
On Windows, when using Python's subprocess to run 7-Zip, the terminal can not output the compression progress
In a Windows system, when using Python's subprocess to run 7z.exe, the terminal does not display the compression progress. However, when I run the command directly in the Command Prompt (CMD), I can ...
- 21
1
vote
0
answers
316
views
How do I solve "invalid header data" for py7zr?
I am attempting to create a generic backup program in python using py7zr. So far, it seems to be creating archives successfully, but they cannot be extracted. I have created the following test script ...
- 11
0
votes
1
answer
502
views
How to read a .txt file inside a 7z folder using python?
I'm trying to print() the contents of a .txt file (called helloWorld.txt) and this .txt file is inside a .7z folder.
The folder is called FolderWorld.7z
Here's what I've tried:
import py7zr
...
- 11
0
votes
0
answers
776
views
FileNotFoundError: [Errno 2] No such file or directory: '7z' in Python
subprocess.call(["7z", "e", "Data/test/Robert_Hilder_history.txt.7z"], stdout=open(os.devnull, 'wb'))
I got an error
FileNotFoundError: [Errno 2] No such file or ...
- 45
0
votes
0
answers
40
views
python 7zip script works for a while then closes randomly
This python script goes into all subdirectories and zips all the folders individually into a .zip using 7zip. Then it deletes the orginal folder and logs the directory it has deleted in a text file.
...
- 31
0
votes
0
answers
87
views
7zip go down 1 directory in all folders then zip up all folders seperately
With the python script below I'm trying to make it so it goes down 1 directory to all folders within the set "path_to_dir", then zip up all the folders inside them individually.
For example ...
- 31
3
votes
1
answer
622
views
read a .7z file in memory with Python, and process each line as a stream
I'm working with a huge .7z file that I need to process line by line.
First I tried py7zr, but it only works by first decompressing the whole file into an object. This runs out of memory.
Then ...
- 59k
1
vote
1
answer
1k
views
Python: Reading 7zipfile without extracting it
I have a zip directory similar to this one:
folder_to_zip
- file_1.csv
- folder_1.zip
1. file_2.csv
2. file_3.csv
3. folder_2.zip
**.**file_4.csv
...
user19480211
1
vote
1
answer
218
views
Open 7zip with no crc in python [closed]
I want to open an 7zip file in python and used the py7zr library but getting following error:
CrcError: (3945015320, 1928216475, '1_Microsoft Outlook - Memoformat (3).tif')
I tried the following code:...
- 39
0
votes
0
answers
229
views
How I can compare two folders with multi-volume .7Z archive files, and get the output files/folders where the difference is more than 1GB
I have 2 paths where each one them has multi-volume .7Z archive files.
Path A contains 4 files: example1.7z.001, example1.7z.002, example1.7z.003, example1.7z.004 (Total size of all is 15 GB). Once ...
- 1,388
2
votes
1
answer
164
views
why does the list switch allow xcopy to work?
I have a batch script that uses xcopy to copy a large number of files. For a while, the following setup worked fine:
set source=<path_to_source_files>
set dest=<path_to_destination>
xcopy ...
- 51
1
vote
1
answer
1k
views
How to extract a mult-part zip file in python?
Suposse that I have some files that I downloaded from a server and they are zipped with 7zip in multiple parts, the format is something like this myfile.zip.001, myfile.zip.002, ..., myfile.zip.00n. ...
1
vote
1
answer
232
views
How to extract many files from multiple 7z using Python?
I need to extract 700k jpg files that are dispersed among 50 7z files. I have a txt file that has one row for each file I need. The row contains the target 7z file and location and file name.
This is ...
- 113
0
votes
0
answers
2k
views
Bruteforce 7z archive with a password list
I have a 7z file and a password list.
I want to bruteforce that file with that list.
I use Windows 10, Python 3.11.1 and 7z 22.1.0.0
My code:
path_7z: str = "C:/Program Files/7-Zip/7z.exe"
...
- 2,929
0
votes
1
answer
4k
views
Python Compressed file ended before the end-of-stream marker was reached. But file is not Corrupted
i made a simple request code that downloads a file from a Server
r = requests.get("https:.../index_en.txt.lzma")
index_en= open('C:\...\index_en.txt.lzma','wb')
index_en.write(r.content)
...
- 13
1
vote
1
answer
586
views
How to compress folders in 7z format using python
I have a nested folder structure like:
- Folder1
---->Test1
-------->a
-------->b
-------->c
- Folder2
---->Test2
-------->d
------...
1
vote
1
answer
1k
views
Python py7zr can't list files in archive - how to read 7z archive without extracting it
I tried to list all files inside 7z archive (I don't want to extract them).
I followed the documentation of the creators of py7zr.
My code look like this:
def checkArchive(archivePath):
for filename ...
3
votes
1
answer
573
views
Random access in a 7z file
I have a 100 GB text file in a 7z archive. I can find a pattern 'hello' in it by reading it by 1 MB block (7z outputs the data to stdout):
Popen("7z e -so archive.7z big100gb_file.txt", ...
- 46k
0
votes
0
answers
112
views
Create a 7z .sfx file for different architecture with python
I'm trying to create a compressed 7z sfx archive using python on a x86-64 Ubuntu 20.04 machine, and transfer it to a ARM EABI5 Linux machine, but I cannot find the appropriate functions or flags to ...
- 300
1
vote
2
answers
2k
views
How can I only compress selected individual files in py7zr?
I have a list that includes the paths of the files (in my case movies), which is called collectMovies and has followed data structure:
['C:\\Users\\UserName\\Desktop\\Movies\\pso-cash-sd.mkv',
'C:\\...
- 1,586
2
votes
5
answers
3k
views
Write in-memory data to an encrypted ZIP or 7Z (without using an on-disk temp file)
I have 2 GB of data in memory (for example data = b'ab' * 1000000000) that I would like to write in a encrypted ZIP or 7Z file.
How to do this without writing data to a temporary on-disk file?
Is it ...
- 46k
2
votes
1
answer
514
views
Downloading encrpyted/compressed 7z file from slack url
Having an issue trying to download a encrypted/compressed 7zip file. I see the binary data being returned but still getting an error that the file is not downloaded. The information is being grabbed ...
- 103
1
vote
0
answers
1k
views
Show progress bar when compressing with py7zr
I'm using py7zr to compress a very large file.
I'm using this syntax from the example:
with py7zr.SevenZipFile('target.7z', 'w') as z:
z.writeall('./base_dir')
Being the file very large, I would ...
- 4,004
0
votes
1
answer
449
views
Cannot run system command
I am trying to zip a folder using python. The following command works from command prompt:
7z a C:\\Temp\\ C:\\Temp\\
But the following only show an error code of 1:
os.system("7z a C:\\Temp\\ C:...
- 375
0
votes
0
answers
479
views
How to extract 7z file with python in Ubuntu environment?
I want to extract .7z file using python.
Previously in Windows I use pyunpack to extract it, and works. But when I run my script on Ubuntu, It return an error during extraction.
This is the error:
...
- 892
-1
votes
1
answer
2k
views
How to extract multiple 7z file using Python?
I would like to extract multiple .7z files using Python.
I've tried this, but it only extracted one file. I already put in a loop.
Below is what I've tried.
import os.path
import glob
from pyunpack ...
- 892
0
votes
1
answer
699
views
Can i include linux executables in my android application?
I have created a python app that calls subprocess to 7z.exe On windows and 7z on linux and it works fine.
subprocess.run('7z x ...')
What about android will the linux executable version of 7z
work ...
user13425180
0
votes
1
answer
214
views
remove '.part1' from Output directory in 7z
I have 2 files test.part1.rar and test.part2.rar
When extracting them using command
7z x -o* test.part1.rar
The output directory is a folder with test.part1 name
I want the output directory to be a ...
user13425180
1
vote
0
answers
374
views
Python code to create password protected zip in-memory for HttpResponse
I would like to generate a password-protected Zip file for my csv data in memory and return to webpage via Django HttpResponse.
So far I can generate plain zip file to HttpResponse as follows:
from ...
- 1,522
0
votes
1
answer
237
views
using python to extract archives of different types to a specific folder with using a subprocess
I'm trying to extract archives of different types (such as: *.7zip, *.V3P, *.Q4P) to a specific folder with using a subprocess.
import subprocess
archname = "E:\\temp\\payments.zip"
dir7z = ...
- 1
1
vote
0
answers
310
views
create self-extracting archive (sfx exe) in-memory with python
I need to create a self-extracting archive exe file in-memory and upload it to GCS.
I know 7zip can do that but as far as I see, py7zr doesn't support SFX files.
Does someone know how to do that?
- 31
1
vote
1
answer
417
views
pysftp + py7zr decompress is hanging on archive
A little context : a client is putting 7-zip archive on a remote sftp server and i process them.
My issue is that on some 7-zip file my program is hanging and the decompress function is never ending ...
- 61
0
votes
1
answer
70
views
"incorect command line error" when compressing file via subprocess
I need to compress a batch of files via subprocess and 7z.exe. Their names start with "-".
It seems that 7z cannot accept that. There is no problem with other files starting with "+&...
- 39
1
vote
1
answer
663
views
Using python subprocess and 7-zip to extract .tgz file
I am trying to extract a .tgz file by calling 7-zip using the subprocess function (helpful stack overflow post here)
I need to call subprocess twice. First to unzip from .tgz --> .tar and second to ...
- 89
0
votes
0
answers
119
views
how to unzip / extract tgz.part-af file with python?
I want to unzip it. The file size is almost 5GB
import tarfile
fname = x_geotiff.tgz.part-af
opener, mode = tarfile.open, 'r:gz2'
tar = tarfile.open(fname, "r:gz") # gz,gz2,bz,bz2,* do not ...
-4
votes
1
answer
162
views
How to use 7z from the python scripts
I have following syntax in my python script.
x = Popen("\"C:/Program Files/7-zip/7z\" a -tzip " + new_file + " general/*")
Can anyone please explain the meaning of this ...
1
vote
2
answers
1k
views
Tracking 7zip progress on Windows 10 with Python
I understand 7zip has some issue, where it masks its progress from code that tries to call it (not sure why).
I saw here that -bsp1 flag should show the hidden progress, but still nothing in Python:
...
- 2,808
2
votes
3
answers
3k
views
Unzipping .7z File on Google Colab
I have a Zip file named 'mathoverflow.net.7z' in my google drive which I have loaded to colab using the given code. But, when I try to unzip it I get an error. Please suggest a way to rectify this.
...
- 937
2
votes
1
answer
2k
views
Extract 7z files on the fly in S3 with boto3
I have a really large 7z file in s3 bucket say s3://tempbucket1/Test_For7zip.7z that runs into several tens of GB. I do not want to download it, unzip it and re upload it back to s3. I want to use ...
- 2,191
1
vote
1
answer
775
views
Combine 7z chunks on an AWS S3 Bucket
I have compressed a file into several chunks using 7zip:
HAVE - the following saved in an AWS S3 Bucket:
pic.7z.001
pic.7z.002
pic.7z.003
pic.7z.004
pic.7z.005
Each chunk is 700 MB.
WANT - the ...
- 13
1
vote
1
answer
452
views
GitHub Actions Pynsist NSIS YAML
I am trying to create a GitHub action that will use Pynsist (a python package together with NSIS) to produce a build folder with the setup file my project on my github repository. I wondered if anyone ...
- 113
0
votes
0
answers
112
views
Replicate 7zip gzip with advanced options in python
I want to compress file with gzip and use python to get result identical to 7zip gzip compressed file, but python gzip does not provide advanced options given by 7zip.
7zip gzip has two options that ...
2
votes
1
answer
4k
views
Python 7zip (py7zr) not registering file type
I have a script to extract all the contents of an .exe file however, the register_archive_format and register_unpack_format functions don't seem to work as expected. Here's a short version of my ...
- 287