{"id":279,"date":"2023-03-27T18:05:22","date_gmt":"2023-03-27T18:05:22","guid":{"rendered":"https:\/\/www.endurantdev.com\/blog\/?p=279"},"modified":"2023-03-31T12:09:46","modified_gmt":"2023-03-31T12:09:46","slug":"both-async-i-o-modules-supported-to-unzip-asynchronously-in-python","status":"publish","type":"post","link":"https:\/\/www.endurantdev.com\/blog\/both-async-i-o-modules-supported-to-unzip-asynchronously-in-python\/","title":{"rendered":"Both async I\/O modules supported to unzip asynchronously in Python"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">Finally, I got some time to update the <code><a href=\"https:\/\/github.com\/ueni-ltd\/async-unzip\" target=\"_blank\" rel=\"noreferrer noopener\">async-unzip<\/a><\/code> Python module to add support for <code><a href=\"https:\/\/github.com\/Tinche\/aiofiles\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">aiofiles<\/a><\/code>. One friend asked about this, as his project is <code>aiofiles<\/code>, and he doesn&#8217;t understand my love for <code><a href=\"https:\/\/github.com\/mosquito\/aiofile\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">aiofile<\/a>.<\/code> It is elementary; I prefer a thread-free file I\/O operation in Linux with <a href=\"https:\/\/pypi.org\/project\/caio\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">caio<\/a>\/<a href=\"https:\/\/pagure.io\/libaio\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">libaio<\/a> native asynchronous async I\/O operations by <code>aiofile<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So, <code>async-unzip<\/code> was built for some operations with <code><a href=\"https:\/\/github.com\/samuelcolvin\/arq\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">arq<\/a><\/code> to not get <a href=\"https:\/\/docs.python.org\/3\/library\/asyncio-exceptions.html#asyncio.TimeoutError\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">AsyncioTimeout<\/a> issues on extended sync operations with unzipping big files. Also, I wouldn&#8217;t say I like massive memory usages, like a project on enormous NPI and other healthcare data imports in a <a href=\"https:\/\/pharmacy-near-me.com\/\" target=\"_blank\" rel=\"noreferrer noopener\">pharmacy near me<\/a>. So, <code>async-unzip<\/code> helped with that too.<br>I will have another post to compare <code>aiofile<\/code> and <code>aiofiles<\/code> later, but, just to confirm from the start, they will always be slower than the standard way to work with files. We had another aim here \ud83d\ude09<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So, it took some time for debugging to find that <code>seek<\/code> call is synchronous in <code>aiofile<\/code> and requires <code>await<\/code> in <code>aiofiles<\/code>, and I finally got out of crazy Zlib errors.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">So, version 0.3.6 of <code>async-unzip<\/code> was released, feel free to take it from PyPi by :<\/p>\n\n\n\n<blockquote class=\"wp-block-quote is-layout-flow wp-block-quote-is-layout-flow\">\n<p class=\"wp-block-paragraph\"><code>pip install --upgrade async-unzip<\/code><\/p>\n<\/blockquote>\n\n\n\n<p class=\"wp-block-paragraph\">P.s. Was quite surprised by results of libaio usage, on my Intel-based server on Debian Linux the time of data import with the usage of async-unzip and asynchronous file-reading operations without threads decreased from 56 to 44 minutes. However, I need to cut the part of actual files I\/O operations to generate some synthetic, but close to be real test with possibly parallel work of the tasks inside one process.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Finally, I got some time to update the async-unzip Python module to add support for aiofiles. One friend asked about this, as his project is aiofiles, and he doesn&#8217;t understand my love for aiofile. It is elementary; I prefer a thread-free file I\/O operation in Linux with caio\/libaio native asynchronous&#8230;<\/p>\n","protected":false},"author":1,"featured_media":284,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-279","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-technology"],"amp_enabled":true,"_links":{"self":[{"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/posts\/279","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/comments?post=279"}],"version-history":[{"count":2,"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/posts\/279\/revisions"}],"predecessor-version":[{"id":285,"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/posts\/279\/revisions\/285"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/media\/284"}],"wp:attachment":[{"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/media?parent=279"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/categories?post=279"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.endurantdev.com\/blog\/wp-json\/wp\/v2\/tags?post=279"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}