Python and boto3 knocking my broadband out
I knocked up a script to download a number of large zip files from s3. Super simple with boto3 and Python but it was so effective at using my 100Mb/s network connection that everything else got knocked off the network including Chrome on the same PC. Luckily boto3 1.19.5 and later now support a bandwidth limit for s3 trasnfers so after adding that I was able to keep it at a sensible level. import boto3 from boto3.s3.transfer import TransferConfig print('boto3 version') print(boto3.__version__) # Download an S3 object - max_bandwidth is in Bytes per sec so 1024**2*5 is 5 MiB/s download_config = TransferConfig(max_bandwidth=1024**2*5) s3 = boto3.client('s3') s3_client = boto3.client('s3') s3_bucket = 'your_bucket_name' filename = 'your_file' with open(filename, 'wb') as f: print(f'Downloading {filename}') s3_client.download_fileobj(s3_bucket, filename_file, f, Config=download_config) See https://boto3.amazonaws.com/v1