2,366 questions
0
votes
0
answers
40
views
Audio Recorded on Chrome and Uploaded to S3 Using Presigned URL Plays in All Browsers Except Safari
I'm developing a web application that allows users to record audio and upload it to AWS S3 using presigned POST URLs. The recorded audio is played back using an element with the S3 link as the source....
0
votes
3
answers
100
views
Duplicity SSLEOFError: EOF occurred in violation of protocol
I have duplicity running for years, stable and without problems. Now I encountered the following problem when I backup my files to Amazon S3.
All files in the format "duplicity-full....
- 431
0
votes
0
answers
29
views
Error uploading file: An error occurred (InvalidArgument) when calling the PutObject operation: None
I have this create function using Boto within the Flask and it has been working for months perfectly, but suddenly it just stopped working, and I can't seem to figure out why.
@posts_blueprint.route('/...
- 197
0
votes
1
answer
77
views
boto 2.49.0 unable to load ECS Fargate task role credentials from instance metadata
I am working on getting a legacy application up and running. It is using Python 2.7, I have managed to get it Dockerized and deployed to ECS using Fargate.
However the legacy code is using boto 2.49.0 ...
- 55
3
votes
0
answers
274
views
EOF occurred in violation of protocol (_ssl.c:2426)
I am trying to get inference from a deployed pretrained model on Sagemaker notebook environment. While executing the below line of code,
response = predictor.predict(serialized_data)
I am receiving an ...
- 31
0
votes
0
answers
405
views
Amazon Bedrock - security token included in the request is invalid
I'm trying to invoke a ML model with amazon bedrock, I verified that I used the correct credintials to connect to my AWS accout, i imported boto3, I even tried to list my S3 buckets and it worked. I ...
0
votes
0
answers
23
views
Different shaped responses for generate_presigned_post
My application's deployed across 2 regions, and calling generate_presigned_post from the two regions returns different response shapes. The first returns:
{
"url": "https://app-1-...
- 8,157
0
votes
2
answers
1k
views
retrieveAndGenerate Syntax Error: Unknown parameter generationConfiguration or retrievalConfiguration (Claude-v3, Amazon Bedrock)
I am trying to retrieve and generate response from knowledge base use claude-v3 model. To do so I followed the boto3 documentation and Blog Post on Amazon and wrote this method:
def ...
- 74
0
votes
1
answer
972
views
ERROR - Runtime.ImportModuleError: Unable to import module 'main': cannot import name 'DEPRECATED_SERVICE_NAMES' from 'botocore.docs'
I'm supporting a lambda project (python 3.8) and when I run the command "sam local invoke LambdaFunction --event events/get_users.json", it gives the following error
`Invoking main....
- 1
1
vote
1
answer
158
views
botocore >= 1.28.0 slower in multithread application
The official Boto3 docs recommends creating a new resource per thread: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/resources.html#multithreading-or-multiprocessing-with-resources
...
- 989
0
votes
1
answer
152
views
How to set timeout and retry strategy when I enrypt/decrypt data using python aws_encryption_sdk?
I'm using aws_encryption_sdk to encrypt and decrypt data. Once we hit an issue and our process stuck for ~24 hours. One of the hypotheses was that there was a problem with string value decryption, ...
- 197
1
vote
1
answer
293
views
How to save logs, to S3Boto3Storage, with Django Logging
I need to save my logs to a file on dedicated storage (using S3BotoStorage). I haven't found any working solution yet. Does anyone know how to make it work? I need to incorporate something similar to ...
1
vote
1
answer
194
views
Asyncio with boto and for range?
I'm trying first time the asyncio botocore implementation. However, I'm quite sure I'm not getting the expected asynchronicity, likely due to my own lack of experience with it. :)
The goal of the ...
- 1,299
0
votes
0
answers
30
views
AWS::SNS::Subscription not created by moto by CloudFormation.create_stack
I have following CloufFormation template
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
ServiceName:
Default: test
Type: String
Resources:
DynamoDBTable:
Properties:
...
- 3,801
0
votes
2
answers
85
views
Lookup Key from S3 Bucket using Boto
I've taken a script written by Paul Davies about reingesting Splunk Logs from the AWS Cloud.
The
When my logs have failed to process in Kinesis Firehose they get placed in a backup S3 bucket. The ...
- 129
1
vote
2
answers
967
views
get-log-events returns only a few results of many
I am logging to CloudWatch from a python lambda. In the AWS console I can see several hundred log entries, but if I try to use get-log-events from either a Python Boto call, or through the AWS CLI I ...
- 23
1
vote
2
answers
777
views
AWS DynamoDB Delete Item
I'm working on deleting DynamoDB item through Lambda.
I have a simple table setup and Partition key is Date (Number) and I don't have sort key.
I tested below command in AWS CLI (via CMD) and it is ...
- 1,191
1
vote
1
answer
222
views
How to get verification code/change password code in boto3?
I'm writing tests in Moto and Pytest for a confirm sign up route that receives a verification code and a email and uses boto method confirm_sign_up to confirm user.
My question is: After I created a ...
0
votes
1
answer
232
views
Cannot SSH on to AWS EC2 instance created using Boto
I have been creating AWS EC2 instances using Ubuntu and Boto for months. They start-up, I issue commands from Python etc. This has been working fine.
I create the instances using this:
instances = ec2....
- 17.2k
1
vote
0
answers
583
views
How to get all error logs from aws cloudwatch using boto3
I am trying to get all the error logs from aws cloudwatch, to do this I tried looping through all log group names and running this code:
client.start_query(
logGroupName=group, # log group name
...
- 37
3
votes
2
answers
2k
views
S3:HeadObject returns 403 on MinIO instance proxied by Cloudflare
When sending HeadObject request to MinIO instance proxied by Cloudflare, 403 is returned instead of 404 (for non-existent files). GET, PUT and DELETE operations can be performed with no issue.
For ...
- 806
0
votes
2
answers
91
views
boto equivalent of aws client command
I have this command that works as expected.
But I will like to use boto instead.
aws cloudwatch get-metric-statistics \
--region ap-south-1 \
--namespace AWS/RDS \
--metric-name DBLoad \
...
- 32.1k
1
vote
0
answers
227
views
Boto 2.49.0, support for TLS 1.2?
I have an old application that has a very old version of boto2 (2.6.0) that doesn't have the support for TLS 1.2
So instead of the costy re-writing of all the application in python3, using boto3, I ...
-1
votes
1
answer
116
views
Grant s3 user access in bucket level policy
I have minio with bucket named "bucket" and IAM user named "user1"
I'll try to grant access to this bucket with Bucket Level Policy
client = boto3.client('s3', endpoint_url='...
- 71
1
vote
0
answers
232
views
Error when using pandas .to_csv function to upload to S3. "Anonymous access is forbidden for this operation"
I'm getting and error "Anonymous access is forbidden for this operation" when trying to use the df.to_csv function.
df.to_csv("s3://testdatateam1/test6.csv",
...
- 23
0
votes
1
answer
312
views
Download and upload data from AWS S3 using python [duplicate]
I would like to upload and download data from AWS S3 bucket using python code.
The files are present under a folder in AWS S3 location.
Example :
Bucket - aws-xxx-use1-00-d-s3b-xxx-yyy-xxx
Folder - ...
0
votes
1
answer
1k
views
AWS Boto3 sts get_caller_identity - catching exceptions if credentials are invalid
With a Python app, using Boto3 v1.26.59 (and botocore of same version) about the first thing done is to try to get the username of the user. We have Identity Center (SSO) users. With aged credentials (...
- 2,852
1
vote
0
answers
490
views
Multithreading in AWS boto pagination Python not working
I tried multithreading with paginator but instead of running on multiple thread, it is running on a single thread.
response: <class 'botocore.paginate.PageIterator'>
page.get('Contents') return ...
- 110
0
votes
1
answer
725
views
AttributeError: 'dict' object has no attribute 'Instance' python
i am trying to start the EC2 Instance with PrivateIP where i provide as argument "PrivateIP" and below code will describe the ec2 and get InstanceId. Once InstanceId received, it starts the ...
- 63
0
votes
1
answer
996
views
S3 tagging and access control policies not working for limiting the tags keyset on an object
Trying to restrict tags to only a given set of keys that can be attached to the objects. Using bucket level policies to define this condition. However, the logic is not working. Bucket policy (https://...
- 1,920
0
votes
0
answers
16
views
Skipped ec2 installation error:fatal: [localhost]: FAILED! => {"changed": false, "msg": "boto required for this module"} [duplicate]
I installed boto. But when I run playbook it's skipped ec2 server creation and asking boto required.
[ec2-user@ip-172-31-11-186 ~]$ cat playbook.yml
---
- hosts: localhost
remote_user: ec2-user
...
0
votes
1
answer
54
views
Reading Reponses SMS using SNS AWS
I'm using AWS SNS service and I'm able to sent sms using that service, I was wondering if there is any way that I can read responses for my SNS SMS sent?
Appreciate the help!
- 79
1
vote
1
answer
254
views
API for creating behaviour in cloudfront
I have created a CloudFront Distribution through the console , Am trying to create a origin and behaviour in that distribution using boto3
I cant find any api to create origin and behaviour in ...
- 23
0
votes
1
answer
1k
views
trying to use boto copy to s3 unless file exists
in my code below,
fn2 is the local file and "my_bucket_object.key" is a list of files in my s3 bucket.
I am looking at my local files, taking the latest one by creation date and then looking ...
- 99
1
vote
0
answers
127
views
Return filename of specific pattern which was last modified from s3 bucket
I am writing a python script where I have to fetch just the filename (Eg. 'abc.csv') from a directory in a S3 Bucket which was last modified. I have managed to fetch the filename from the bucket but ...
- 11
0
votes
1
answer
737
views
Cannot upload the files into AWS S3 bucket with ACL pubic-read using python
I am trying to upload the file into s3 bucket but I am getting access denied error when I am trying to upload the files with ACL public-read
It is working fine with ACL private
class S3:
def ...
- 287
0
votes
0
answers
53
views
pip show boto3 - missing classifiers
I am comparing boto3 of two Amazonlinux instances(server1 and server 2). I can see classifiers are missing in server 2 .
Server 1:
[root@xxxxx]: pip show boto3
---
Metadata-Version: 2.0
Name: ...
- 41
1
vote
1
answer
2k
views
Find the cost of each resource in AWS
I have a python script to get the cost of each service for the last month, but I want the cost for each resource using the tag mentioned for that resource. For example, I have got the cost for RDS ...
- 23
2
votes
0
answers
351
views
S3 boto - list objects in parallel
I want to quickly (in parallel?) list directory/prefix with 100K+ objects.
The pagination works (see below) but I'm unable to get NextKeyMarker until I download the last page - it breaks the idea of ...
- 1,547
0
votes
1
answer
893
views
Python : XML file downloaded from S3 full of string escaping characters
I have a number of XML files that I have added to S3 (localstack sever). I can view these files through Cyberduck and they are valid xml files. However, when I download the objects, the XML data is ...
- 7,043
-1
votes
1
answer
4k
views
How I can upload file to folder in bucket in S3 using Python?
I have local S3 and want to upload some files into folders in mu bucket.
For example:
my_bucket
- folder1
- folder2
I have to upload file file1.json to folder1 and file1.json to folder1
I try do ...
- 41
0
votes
1
answer
412
views
Return empty columns in athena boto response object
When executing a select query against an athena table via boto3, the response object given is in the syntax:
{
"UpdateCount":0,
"ResultSet":{
"Rows":[
...
- 476
1
vote
0
answers
244
views
AWS boto3 Quicksight "AllowedDomains" parameter non-existent
my problem is that im calling the boto3 quicksight API with this function call:
response = quickSight.generate_embed_url_for_registered_user(
AllowedDomains=[
'...
- 11
1
vote
0
answers
98
views
Can I read a trained linear model from s3, without reconstructing a local copy?
In order to run a dask pipeline on a coiled cluster that uses a previously trained linear model in each task, I believe I need to read the model directly from S3. Reading the model within a task did ...
- 11
0
votes
1
answer
99
views
run 2 instructions sequentially and execute the second instruction based on a condition in Python Lambda
I have a requirement to execute 2 tasks sequentially in a Lambda Function in Python.
But as the execution is asynchronous I have to check the status of the first task regularly and run the second task ...
- 7
0
votes
1
answer
602
views
Differentiate between website endpoint from REST API endpoint for AWS S3
I have an input provided by a user, that would be used as the endpoint url for bucket operations for an S3 bucket.
Is there a way to differentiate if the url is a REST API endpoint or a website ...
- 399
0
votes
1
answer
763
views
How do you effectively move and partition files in s3 using boto3?
There are around 10k files in an s3 location which got exported from dynamodb PITR export to s3 option. These files aren't partitioned in any way and it is within a single folder which is a problem ...
- 49
0
votes
1
answer
497
views
Return all items from dynamodb ordered by date boto
I am attempting to return all items from a dynamodb table with results ordered by date. The tables structure is:
ID
Date
Category
Email
Errors
IsRead
Message
With ID as the primary key and date ...
- 476
0
votes
1
answer
764
views
import S3 from s3 library
I installed s3 library on my python 3.8 environment using
pip install s3
which installed s3 version 3.0.0
I then try to run
from s3 import S3
But I would get an error of
Traceback (most recent call ...
- 2,883
7
votes
2
answers
11k
views
Best way to read aws credentials file
In my python code I need to extract AWS credentials
AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID
which are stored in the plain text file as described here:
https://docs.aws.amazon.com/sdkref/latest/...
- 235