All Questions
30 questions
1
vote
2
answers
967
views
get-log-events returns only a few results of many
I am logging to CloudWatch from a python lambda. In the AWS console I can see several hundred log entries, but if I try to use get-log-events from either a Python Boto call, or through the AWS CLI I ...
- 23
-3
votes
1
answer
525
views
What is efficient and fastest way to get number of records( no. of lines) in zip file using aws s3 command?
I want to get number of records in zip file which is present in s3 bucket. Could you please tell me what is the fastest way to get the result?
I am running below command but that is not working. ...
- 233
4
votes
3
answers
14k
views
Copy file from the s3 to the local?
I have a lot of files in my s3 bucket, so is there any aws cli command which I can use to find a most recent file with a prefix name in s3? and how can I copy that file from s3 to my local folder? Can ...
user13542716
5
votes
3
answers
7k
views
Faster way to Copy S3 files
I am trying to copy around 50 million files and 15TB in total size from one s3 bucket to another bucket.
There are AWS CLI option to copy fast. But in my case, I want to put a filter and date range. ...
- 1,548
2
votes
1
answer
626
views
List volumes missing specific tags
I am trying to list Volumes from the AWS CLI that is missing a specific tag key. While I can list volumes missing a specific key with below command.
aws ec2 describe-volumes --query 'Volumes[?!...
- 75
0
votes
0
answers
1k
views
How to iterate over s3 bucket to collect all keys from a given date of a subfolder-boto3
I am trying to iterate over the files in my s3 bucket and send all the files(or keys) that run time were in 2019, 2018, or 2017 to a csv file.
The structure of my s3 path goes like this:
file/...
- 344
0
votes
2
answers
380
views
Move a file in S3 to a new folder with the same name
I have an AWS S3 bucket with hundreds of files in it. I want to create a new folder for each file, and I want the folders to have the same name as the files.
Basically, from this:
bucket_name/
├── ...
- 161
0
votes
1
answer
54
views
Not able to install boto packages using pip 8.1.1
I have installed python 2.7 and pip 8.1.1 available in the same package. I am trying to install boto and aws cli using pip. I am getting the below error.
- 533
0
votes
1
answer
485
views
How to create an EC2 instance in aws through Ansible?
I'm trying to create an ec2 instance in aws using ansible.
i provided the access key id and the access key secret using aws configure beforehand
i took these keys from my account in rosttahub
this ...
-1
votes
1
answer
28
views
Few errors when attempting to write a script that uploads any data in EC2 Instance to S3 Bucket using BOTO
I have been trying to back up my EC2 Instance to an S3 Bucket but have contentiously come across few errors when I run the file. Most notable error being S3ResponceError: 403 Forbidden
FYI, I am ...
1
vote
1
answer
374
views
Why there is a difference in the number of launch configurations received from the python script and AWS CLI?
The python script that returns the list of launch configurations is as follows ( for the us-east-1 region):
autoscaling_connection = boto.ec2.autoscale.connect_to_region(region)
nlist = ...
- 7,441
6
votes
5
answers
33k
views
s3 - how to get fast line count of file? wc -l is too slow
Does anyone have a quick way of getting the line count of a file hosted in S3? Preferably using the CLI, s3api but I am open to python/boto as well.
Note: solution must run non-interactively, ie in an ...
- 320
0
votes
1
answer
573
views
Difference between python boto get_contents_to_filename and s3 cp?
I copy files from AWS S3 to my local file system. From an I/O perspective, what's the difference between using get_contents_to_filenamehttp://docs.pythonboto.org/en/latest/ref/s3.html from the boto ...
- 990
3
votes
2
answers
3k
views
How to deploy SAM template using boto3
I am trying to deploy
https://s3.amazonaws.com/serverless-chatbot-code/chatbot.yaml
using boto3 library function create_stack but getting an error
"CreateStack cannot be used with templates ...
- 2,534
-2
votes
1
answer
433
views
Boto to Boto3 function implementation
1) How can i implement this from boto into boto3 code:
conn = boto.connect_ec2() # boto way
sgs = conn.get_all_security_groups() # boto way
for sg in sgs:
if len(sg.instances()) == 0:
...
- 135
0
votes
1
answer
171
views
How to create ec2 instances from another instance? boto awscli
I'm looking to create an AWS system with one master EC2 instance which can create other instances.
For now, I managed to create python files with boto able to create ec2 instances.
The script works ...
- 172
2
votes
2
answers
1k
views
How Do I download latest files from S3 Bucket that contains A LOT of files
There are a lot of files in my S3 bucket, I am looking to download most 1000 recent ones (uploaded by date).
How do I go on about doing that with AWS cli or s3 boto
- 41
3
votes
3
answers
10k
views
HTTP Code 400 getting objectfrom AWS S3 bucket with Boto3 in python
I'm working on a script that is supposed to download a file from my s3 bucket and then make some changes on it.
The problem is that my script runs well when executed as root but does not when ...
- 455
4
votes
1
answer
2k
views
Cannot read a key from S3 with boto, but can with aws cli
Using this aws cli command (with access keys configured), I'm able to copy a key from S3 locally:
aws s3 cp s3://<bucketname>/test.txt test.txt
Using the following code in boto, I get ...
- 2,484
3
votes
1
answer
822
views
Automated way to release Unused Elastic IPS
Please Suggest any way or script to release unused ec2 elastic ips. I tried with boto and aws-cli but i'm unable to complete it. Any one could help . i thought of aws ec2 release-address --public-ip &...
- 365
2
votes
2
answers
209
views
AWS get security group ids from private IP
How to get the security group ids associated to the ec2-instance based on Private IP Address
So, want to automate process of modifying security group ingress.
eg: open port 22 on 10.0.0.10 from 10.0....
- 719
17
votes
7
answers
26k
views
How to check when is the last time S3 bucket has been updated?
Does S3 bucket has information in regard to when is the last time it has been updated? How can I find the last time any of the objects in the bucket were updated?
- 15.6k
1
vote
1
answer
83
views
AWS boto tags returning 0 instances when filtering by tag verified to exist
I'm using boto to return instances with a cluster_id tag which is a string uuid that uniquely identifies a cluster.
I'm trying to use boto to return the instances with that tag to ensure the cluster ...
- 1,181
5
votes
1
answer
8k
views
How to get the list of policies assigned to a user using boto3 for an aws profile?
I am still in the learning phase of boto3 and I can't seem to figure out the basics as to get the list of policies assigned to a user using boto3 for an aws profile?
For example:
>> import ...
- 139
1
vote
1
answer
1k
views
get aws cli 'multipart_chunksize' value using boto
I am developing an application in python and I need to calculate the number of S3 Puts, for this I require the total size of file to be divided with multipart_chunksize to find total number of chunks.
...
- 5,029
28
votes
10
answers
17k
views
PyCharm intellisense for boto3
having problems seeing full intellisense (code completion) options in PyCharm.
working with python 3.4 on Windows.
the suggests are partially working:
import boto3
s = boto3.Session() (boto3. will ...
- 513
1
vote
2
answers
561
views
Grant Amazon ML permission to read from Amazon S3 programmatically
I am developing an Amazon Machine Learning application and am reading my data from amazon s3. I ran into a read permission error and found a solution using amazon explorer. I was wondering if there ...
- 65
1
vote
1
answer
879
views
How can I download all files from Amazon S3 that are in a folder that was created last month?
How can I download a folder from Amazon S3 that was created last month until the present?
I have this code using boto:
for key in bucket.list():
if last_month < dateutil.parser(key....
- 77
56
votes
1
answer
38k
views
Difference between s3cmd, boto and AWS CLI
I am thinking about redeploying my static website to Amazon S3. I need to automate the deployment so I was looking for an API for such tasks. I'm a bit confused over the different options.
Question: ...
user1781186
0
votes
0
answers
387
views
How to use awscli to set a non-english ec2's name
I try to use awscli to set up a new name for my ec2 instances:
aws ec2 create-tags --resources i-891053a1 --tags "Key=Name,Value=我的主機"
'ascii' codec can't decode byte 0xe6 in position 0: ordinal not ...
- 7,957