All Questions
415 questions
0
votes
0
answers
29
views
Error uploading file: An error occurred (InvalidArgument) when calling the PutObject operation: None
I have this create function using Boto within the Flask and it has been working for months perfectly, but suddenly it just stopped working, and I can't seem to figure out why.
@posts_blueprint.route('/...
- 197
0
votes
2
answers
1k
views
retrieveAndGenerate Syntax Error: Unknown parameter generationConfiguration or retrievalConfiguration (Claude-v3, Amazon Bedrock)
I am trying to retrieve and generate response from knowledge base use claude-v3 model. To do so I followed the boto3 documentation and Blog Post on Amazon and wrote this method:
def ...
- 74
0
votes
1
answer
232
views
Cannot SSH on to AWS EC2 instance created using Boto
I have been creating AWS EC2 instances using Ubuntu and Boto for months. They start-up, I issue commands from Python etc. This has been working fine.
I create the instances using this:
instances = ec2....
- 17.2k
0
votes
2
answers
91
views
boto equivalent of aws client command
I have this command that works as expected.
But I will like to use boto instead.
aws cloudwatch get-metric-statistics \
--region ap-south-1 \
--namespace AWS/RDS \
--metric-name DBLoad \
...
- 32.1k
-1
votes
1
answer
116
views
Grant s3 user access in bucket level policy
I have minio with bucket named "bucket" and IAM user named "user1"
I'll try to grant access to this bucket with Bucket Level Policy
client = boto3.client('s3', endpoint_url='...
- 71
0
votes
1
answer
725
views
AttributeError: 'dict' object has no attribute 'Instance' python
i am trying to start the EC2 Instance with PrivateIP where i provide as argument "PrivateIP" and below code will describe the ec2 and get InstanceId. Once InstanceId received, it starts the ...
- 63
0
votes
1
answer
996
views
S3 tagging and access control policies not working for limiting the tags keyset on an object
Trying to restrict tags to only a given set of keys that can be attached to the objects. Using bucket level policies to define this condition. However, the logic is not working. Bucket policy (https://...
- 1,920
0
votes
1
answer
54
views
Reading Reponses SMS using SNS AWS
I'm using AWS SNS service and I'm able to sent sms using that service, I was wondering if there is any way that I can read responses for my SNS SMS sent?
Appreciate the help!
- 79
1
vote
1
answer
254
views
API for creating behaviour in cloudfront
I have created a CloudFront Distribution through the console , Am trying to create a origin and behaviour in that distribution using boto3
I cant find any api to create origin and behaviour in ...
- 23
1
vote
0
answers
127
views
Return filename of specific pattern which was last modified from s3 bucket
I am writing a python script where I have to fetch just the filename (Eg. 'abc.csv') from a directory in a S3 Bucket which was last modified. I have managed to fetch the filename from the bucket but ...
- 11
0
votes
0
answers
53
views
pip show boto3 - missing classifiers
I am comparing boto3 of two Amazonlinux instances(server1 and server 2). I can see classifiers are missing in server 2 .
Server 1:
[root@xxxxx]: pip show boto3
---
Metadata-Version: 2.0
Name: ...
- 41
1
vote
1
answer
2k
views
Find the cost of each resource in AWS
I have a python script to get the cost of each service for the last month, but I want the cost for each resource using the tag mentioned for that resource. For example, I have got the cost for RDS ...
- 23
1
vote
0
answers
244
views
AWS boto3 Quicksight "AllowedDomains" parameter non-existent
my problem is that im calling the boto3 quicksight API with this function call:
response = quickSight.generate_embed_url_for_registered_user(
AllowedDomains=[
'...
- 11
0
votes
1
answer
99
views
run 2 instructions sequentially and execute the second instruction based on a condition in Python Lambda
I have a requirement to execute 2 tasks sequentially in a Lambda Function in Python.
But as the execution is asynchronous I have to check the status of the first task regularly and run the second task ...
- 7
0
votes
1
answer
602
views
Differentiate between website endpoint from REST API endpoint for AWS S3
I have an input provided by a user, that would be used as the endpoint url for bucket operations for an S3 bucket.
Is there a way to differentiate if the url is a REST API endpoint or a website ...
- 399
0
votes
1
answer
763
views
How do you effectively move and partition files in s3 using boto3?
There are around 10k files in an s3 location which got exported from dynamodb PITR export to s3 option. These files aren't partitioned in any way and it is within a single folder which is a problem ...
- 49
7
votes
2
answers
11k
views
Best way to read aws credentials file
In my python code I need to extract AWS credentials
AWS_SECRET_ACCESS_KEY and AWS_ACCESS_KEY_ID
which are stored in the plain text file as described here:
https://docs.aws.amazon.com/sdkref/latest/...
- 235
0
votes
1
answer
589
views
Boto3 Error: botocore.exceptions.NoCredentialsError: Unable to locate credentials via shell script
When I run from CLI(terminal) it works, it finds the credentials via sso and passes on
EX:
2022-06-18 07:38:36,421 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-...
- 2,752
0
votes
1
answer
391
views
How to create spot instance with boto3 with required instance attributes instead of instance type
I am able to create spot instance with below code
import boto3
import datetime, random, string, json
client = boto3.client('ec2')
response = client.request_spot_instances(
DryRun=False,
...
- 4,968
0
votes
1
answer
163
views
How to get ip address of spot instance created with boto3
I am able to create spot instance with boto3 with the following code
import boto3
import datetime, random, string
client = boto3.client('ec2')
response = client.request_spot_instances(
DryRun=...
- 4,968
6
votes
1
answer
4k
views
How can I track the progress/status of an asynchronous AWS Lambda invocation?
I have an API which I use to trigger AWS Lambda jobs. Upon request, the API invokes an AWS Lambda job with InvocationType='Event'. Hereafter, I want to periodically poll if the AWS Lambda job has ...
- 486
0
votes
2
answers
1k
views
Problems whit import boto3 in a virtual env
I'm trying to import boto3 modeule in a virtual environemnt.
import os
import boto3
Although I'm having the following problem with import command:
AttributeError Traceback (...
- 47
2
votes
1
answer
2k
views
Using python Download the latest file from s3 bucket inside folder not from inside folder --folder
I want to download the latest file from s3-bucket inside folder only. Actually inside folder there are multiple folders along with files. But i need to download only file of latest date and upload it ...
- 177
0
votes
1
answer
722
views
Conditionally create a task definition in ECS
I have a boto3 script, which updates(create new revision) of ECS task definition and update the ECS service with the new task definition.
import boto3
import json
client = boto3.client("ecs"...
- 1,163
0
votes
1
answer
1k
views
Copy file from s3 subfolder in another subfolder in same bucket
I'd like to copy file from subfolder into another subfolder in same s3 bucket. I've read lots of questions in SO and I came finally with this code. It has an issue, when I run it it works, but it ...
- 1,069
0
votes
0
answers
86
views
Botocore firewall ports
I have a docker image running on EC2 instance that tries to issue an AWS command when I get this error
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "https://ec2.us-...
- 4,176
3
votes
3
answers
3k
views
How to upload the folder to digital ocean spaces?
From the link, https://www.digitalocean.com/community/questions/how-to-upload-an-object-to-digital-ocean-spaces-using-python-boto3-library. It only states to upload files to the spaces.
I want to ...
- 996
4
votes
1
answer
6k
views
find last modified date of a particular file in S3
According to this answer: https://stackoverflow.com/a/9688496/13067694
>>> import boto
>>> s3 = boto.connect_s3()
>>> bucket = s3.lookup('mybucket')
>>> for key in ...
- 3,390
3
votes
1
answer
2k
views
how to specify ContentType for S3 files?
I am trying to convert files to gzip files and then upload them to S3. When I check in S3, the files are there but they don't have a type specified. How can I specify the content type?
for i in ...
user13067694
1
vote
1
answer
99
views
boto3 print rules from bucket encryption
Im writing a python script to retrieve information of AWS and im trying to get only the SSEAlgorith but i get TypeError: list indices must be integers or slices, not str
Is there any way to do this? I ...
- 13
1
vote
0
answers
637
views
EMR Cluster: AutoScaling Policy For Instance Group Could Not Attach And Failed
I am trying to automate the EMR cluster creation through boto3. Unfortunately, I'm getting the following warning:
The Auto Scaling policy for instance group ig-MI0ANZ0C3WNN in Amazon EMR cluster j-...
- 2,757
0
votes
0
answers
103
views
Get Count of Spot Instances
we can use get_cost_and_usage to know the cost spent of spot instance for a particular period of time.
data = self.ce_client.get_cost_and_usage(
TimePeriod={'Start': start, ...
- 78
1
vote
1
answer
1k
views
Boto3 wait_until_exists for available image (object has not attribute)
I'm creating an AMI and I want to wait until the AMI is 'available'. I'm using the following code:
import json
import boto3
import os
import logging
ec2 = boto3.client('ec2')
images = ec2....
- 35
2
votes
1
answer
388
views
Unable to modify target capacity of spot fleet in AWS
I was using a python lambda function to terminate spot fleet instances using the below code,
import boto3
def lambda_handler(event, context):
client = boto3.client('ec2')
response =...
0
votes
1
answer
892
views
how to create an encrypted s3 bucket while creating new s3 bucket with boto3
how to create encrypted s3 bucket while creating new s3 bucket with boto3
I know how to encrypt an existing bucket. But, is there any way to encrypt while creating it via boto3?
- 1
1
vote
1
answer
2k
views
AWS MWAA Airflow Scheduler Celery Executor Fails Tasks With No Logs - Boto Parser Concurrency Error
Using AWS MWAA I'm trying to run tasks. Ocassionally, maybe on 5% of tasks, the tasks will fail with no logs and no evidence it ever made it to a worker. (EDIT: Noticed it happens when two tasks are ...
- 634
0
votes
1
answer
109
views
How to use boto3's export_image?
It seems like the EC2 client is missing the export_image function, even though its listed in the EC2 docs.
import boto3
session = boto3.Session(profile_name=my_aws_profile)
client = session.client('...
- 15.7k
0
votes
1
answer
383
views
Programatically generating web console URL to s3 object
Is there a way to generate urls to the s3 web console (https://s3.console.aws.amazon.com/s3/object/...) within python using botocore or boto3? I know boto can be used to generate presigned urls, but ...
- 31
0
votes
1
answer
2k
views
How to specify file name when executing query via Athena API client (Boto3)?
I have a query string and using the start_query_execution() method, I'm right now able to run my query via Athena and get the results in the form of a CSV file in my S3 bucket.
However, the file's ...
- 792
0
votes
0
answers
198
views
Migrating from Boto2 to Boto3 while using python 2.7
I am currently using boto2 to upload some files into an s3 bucket where the objects are sorted based on keys(which are generated) with the objects I am uploading. I am now required to migrate this ...
- 1
2
votes
4
answers
10k
views
Programmatically establish boto3 sessions with AWS SSO
I'm trying to establish a boto3 session with boto3.session.Session(profile_name='foo') but getting an UnauthorizedSSOTokenError error:
botocore.exceptions.UnauthorizedSSOTokenError: The SSO session ...
- 7,383
0
votes
1
answer
178
views
boto.exception.EC2ResponseError: EC2ResponseError: 403 Forbidden (rights issue or something else)
I am trying to fetch Elastic IPs for in my AWS system, i am not sure whether i am using incorrect code or its the permission issue. Fetching regions is working fine. However, when i try to fetch ip ...
- 975
2
votes
1
answer
2k
views
Python Boto3 MFA making connection with Access_Key_Id, Access_Key, Session_Token and MFA, without passing RoleArn
Can we make an AWS connection to list and fetch objects having temp session using python Boto3 with using only following? and without passing RoleArn?
_AWS_ACCESS_KEY_ID,
_AWS_SECRET_ACCESS_KEY,
...
- 975
0
votes
1
answer
3k
views
How to use Python boto3 to get count of files/object in s3 bucket older than 60 days?
I'm trying to get the count of all object which are older than 60 days? Is there any way to perform a query or any python boto3 method to get this required output?
0
votes
1
answer
442
views
How to move an object within the same S3 bucket using boto client?
I cant seem to find a function that can move a S3 object within the same bucket. I know there's other ways other than s3 client but I need to use S3 client in this case.
- 255
3
votes
2
answers
3k
views
Searching s3 for a bucket using boto3
I'm trying to create a python script that uploads a file to an s3 bucket. The catch is that I want this script to go to s3 and search through all the buckets and find a bucket that contains a certain ...
- 65
0
votes
0
answers
510
views
Is there any way to get the AWS QuickSight dataset refresh rate with boto3?
I have a use case where I need to get the refresh rate of Quicksight datasets programatically with Python for monitoring purposes.
Is there any way to achieve this with boto3?
I have looked up few ...
- 168
1
vote
1
answer
1k
views
How to Search through AWS Route53 Domain records through boto3 sdk?
I want a webpage listing all the Records in a hosted zone from AWS Route 53 and use all the operations like Search, Add and Edit on those records.
Till now, I am able to list all the records using ...
0
votes
1
answer
706
views
Using Python Variable in HTML / .format() for aws SES boto3
I am trying to add a variable inside html body but its not working.
import boto3
SENDER = "[email protected]"
RECIPIENT = "[email protected]"
AWS_REGION = "yyy"
SUBJECT = "...
- 133
0
votes
1
answer
847
views
Boto3 InvalidParameterException while executing the lambda function
I'm getting a Boto3 InvalidParameterException while running the lambda function.
I'm trying to find out a way to handle this exception.
I came across the below solution:
from boto.exception import ...
