Python gzip module

28/12/2020

In this lesson, we will study about how we can make use of Python gzip module to read & writing into the compressed files in Python. The biggest feature this module provides to us is that we can treat compressed files as normal File objects which saves us from the complexity of managing files and their lifecycle in our code and allows us to focus on core business logic of the program.The gzip module provides us almost the same features as the GNU programs like gunzip and gzip.

Writing Compressed Files with open()

We will start with a basic example where we can create a gzip file and write some data into it. For this, we need to make a file and open it with write mode so that data can be inserted into it. Let’s look at a sample program with which we can write data into a gzip file:

import gzip
import io
import os

output_file = ‘linxhint_demo.txt.gz’
write_mode = ‘wb’

with gzip.open(output_file, write_mode) as output:
with io.TextIOWrapper(output, encoding=‘utf-8’) as encode:
encode.write(‘We can write anything we want to the file.n’)

print(output_file,
‘contains’, os.stat(output_file).st_size, ‘bytes’)
os.system(‘file -b –mime {}’.format(output_file))

Here is what we get back with this command:

Writing to zip file

If you now take a look at the folder structure where you executed this script, there should be a new file named with what we provided in our program above.

Writing multiple lines into a compressed file

We can also write multiple lines or actually any number of lines in our gzip file in a very similar fashion as we did in the previous example. To make this example different, we will make use of itertools module as well. Let’s look at the sample program:

import gzip
import io
import os
import itertools

output_file = ‘linxhint_demo.txt.gz’
write_mode = ‘wb’

with gzip.open(output_file, write_mode) as output:
with io.TextIOWrapper(output, encoding=‘utf-8’) as enc:
enc.writelines(
itertools.repeat(‘LinuxHint, repeating same line!.n’, 10)
)

os.system(‘gzcat linxhint_demo.txt.gz’)

Let’s see the output for this command:

Writing multiple lines

Reading Compressed Data

We can also read the compressed file we created in the last example using the gzip module with a very simple call to open function:

import gzip
import io
import os

file_name = ‘linxhint_demo.txt.gz’
file_mode = ‘rb’

with gzip.open(file_name, file_mode) as input_file:
with io.TextIOWrapper(input_file, encoding=‘utf-8’) as dec:
print(dec.read())

Here is what we get back with this command:

Reading a gzip file

Reading Streams

Due to the fact that text files can be very big in size, it is smart to open these files in a stream rather than loading the complete file in a single object which occupies a lot of system’s memory and in some cases may even cause the process to crash completely. Let’s look at a sample program which read the given compressed file in a stream:

import gzip
from io import BytesIO
import binascii

mode_write = ‘wb’
mode_read = ‘rb’

non_compressed = b‘Repeated line x times.n’ * 8
print(‘Non compressed Data:’, len(non_compressed))
print(non_compressed)

buf = BytesIO()
with gzip.GzipFile(mode=mode_write, fileobj=buf) as file:
file.write(non_compressed)

compressed = buf.getvalue()
print(‘Compressed Data:’, len(compressed))
print(binascii.hexlify(compressed))

in_buffer = BytesIO(compressed)
with gzip.GzipFile(mode=mode_read, fileobj=in_buffer) as file:
read_data = file.read(len(non_compressed))

print(‘nReading it again:’, len(read_data))
print(read_data)

Let’s see the output for this command:

Reading gzip file in a Stream

Although program was a bit long, we actually just used Python modules open the file and stream the content on to the console with a buffered reader object.

Conclusion

In this lesson, we looked at how we can make use of Python gzip module to compress and decompress files in Python. The biggest feature this library provides to us is that we can treat compressed files as normal File objects.

Read more Python based posts here.

ONET IDC thành lập vào năm 2012, là công ty chuyên nghiệp tại Việt Nam trong lĩnh vực cung cấp dịch vụ Hosting, VPS, máy chủ vật lý, dịch vụ Firewall Anti DDoS, SSL… Với 10 năm xây dựng và phát triển, ứng dụng nhiều công nghệ hiện đại, ONET IDC đã giúp hàng ngàn khách hàng tin tưởng lựa chọn, mang lại sự ổn định tuyệt đối cho website của khách hàng để thúc đẩy việc kinh doanh đạt được hiệu quả và thành công.
Bài viết liên quan

Creating REST API in Python

REST or Representational State Transfer is a software development style used mainly in API or Application Programming Interface...
29/12/2020

Python OS Module

In this lesson on the Python ‘OS’ module, we will study how this module allows us to perform common OS related...
28/12/2020

Scheduling tasks Using Python

Some tasks in our life are really time taking and we get bored doing those tasks repeatedly. In order to avoid those boring...
29/12/2020
Bài Viết

Bài Viết Mới Cập Nhật

Reliable IPv4 and IPv6 Subnet Rental Services: The Perfect Solution for Global Businesses
23/12/2024

Tìm Hiểu Về Thuê Proxy US – Lợi Ích và Cách Sử Dụng Hiệu Quả
11/12/2024

Mua Proxy V6 Nuôi Facebook Spam Hiệu Quả Tại Onetcomvn
03/06/2024

Hướng dẫn cách sử dụng ProxyDroid để duyệt web ẩn danh
03/06/2024

Mua proxy Onet uy tín tại Onet.com.vn
03/06/2024