All Questions
1,353 questions
-1
votes
1
answer
56
views
Compare Columns in DF & drop rows if Dates are matching between Columns
I have df as below. If Column Buy Exit Date is having Date (YYYY/MM/DD) matching with Column Buy Date having Date (YYYY/MM/DD) in next row, then drop the Buy Date row from its matching condition.
...
- 1,036
0
votes
1
answer
47
views
Convert matrix to dataframe keeping dates
I am running a function in the oce package which generates a matrix. I transpose the matrix and then convert the matrix to a dataframe, but my POSIXct formatted date becomes a list within the date ...
- 27
1
vote
2
answers
53
views
data frame breakdown by year, rounding up by duration of each period
I have a data frame of county executives and the first and last years they served.
I am running a panel study with county-year as the unit of analysis. The date range is 2000 to 2009.
I will like to ...
- 401
1
vote
2
answers
55
views
dataframe breakdown by year
I have a dataset on county executives and their year of inaguration. I need break down which year each executive was inaugurated.
The problem is that the notation under the "year" variable ...
- 401
1
vote
2
answers
65
views
getting the latest date for a a duplicated item in r [closed]
I have the following dataset (below).
I am trying to get the latest SEnd value for each individual tag (see Desired output) where I have the Tag, Owner and the latest SEnd date only. Essentially I am ...
- 105
1
vote
3
answers
64
views
Python Pandas: update last dates values of parameters with simple dynamics grouped by id
Here's a dataframe, with some parameters for each id by some regular quarterly dates. It's originally shuffled randomly, but, at first, let say, it is sorted by fab_date and id.
import pandas as pd
np....
- 175
0
votes
1
answer
37
views
Count unique individuals in each month [duplicate]
I have a dataframe that looks something like this:
data.frame(
stringsAsFactors = FALSE,
species = c("species1","species1",
"species1",...
- 13.8k
0
votes
2
answers
57
views
Return the name of a column which matches a date value
I have a data.frame with about 66k rows and 150 columns. For the purposes of this question, this is a sample. All columns are date objects.
structure(list(fixed_date = structure(c(19267, 19239, 19120, ...
0
votes
1
answer
34
views
thermal sum between two dates, from a daily data frame
I have a data frame with daily temperature data. I would like to calculate the sum in several columns, between two dates from a second data frame and create a output (results) in a new data frame.The ...
- 313
0
votes
2
answers
41
views
Average and sum between two dates, from a daily data frame
I have a data frame with daily temperature data. I would like to calculate the average between two dates from a second data frame and add the results in a new column.
The average temperature must be ...
- 313
-1
votes
1
answer
35
views
Select most recent date in a panel RStudio
I have a data frame which is a panel: I have different CLPs (it's like the ID of an event) repeated several times, one for each phase of the event: organization, set-up, start, end, clean-up.
Each ...
0
votes
1
answer
46
views
Filter Multiple Items on Different Conditions using pandas
I am attempting to filter a dataframe index based on two conditions, ID and DATE, where DATE will vary by ID.
An example of what would be filtering on, I have provided 'filt'
Is there a better way to ...
- 807
2
votes
2
answers
98
views
How to connect data points with line, where values are missing
I need to draw several biomarker changes by Date on one graph, but biomarker samples were measured in different dates and different times, so for example:
data = {
'PatientID': [244651, 244651, ...
- 25
0
votes
2
answers
61
views
How to handle duplicates using asfreq() function. Is there any other way to do this?
I have some hourly data on electricity generation from various sources in various countries. I download data from the ENTSO-E Transparency Platform website and found a problem with data inconsistency. ...
1
vote
2
answers
60
views
Create new date columns based on matching vector
I am trying to simulate visit schedules for participants in a study. I will start with a dataframe of participants and first visit dates, and my goal is to create columns with the dates of later ...
- 13
1
vote
3
answers
89
views
Counting number of days from event, till another event in R
data.frame(
scrape_date = as.Date(c("2023-01-01", "2023-01-02", "2023-01-03", "2023-01-04", "2023-01-05", "2023-01-06", "2023-01-07&...
- 11
0
votes
1
answer
29
views
How do I do right join with multiple conditions?
I have an example below where I need to perform a right join with multiple conditions:
1st dataframe:
df_1 <- data_frame(
dates = c(as.Date("2023-03-01"), as.Date("2023-03-05")...
- 199
0
votes
1
answer
103
views
How to reverse a mutate command in R? [closed]
I am doing the Bellabeat case study from the google analytics course. I am doing it in R and noticed that the date format was in character format rather than "Date" so I tried to use the ...
1
vote
0
answers
26
views
Count the number of business days between two columns in python (ideally using pandas) [duplicate]
I have a df that has two columns with dates in them. I want to find the number of business days between the two columns for each row.
For example, date and date2 are the inputs and bus_days would be ...
- 27
1
vote
1
answer
78
views
Handling missing values in lag column of dataframe
I need to find difference between sucessive datetimes ... and split the file when I find a time difference of more than 5 minutes. When I subtract periods I get answers in milliseconds and there is ...
- 135
2
votes
3
answers
96
views
Calculating the date difference by group and consecutive ids
I have a set of customers, their daily activity log, and if they had and issue raised on their account. A customer can have an issue raised on their account multiple times throughout their duration as ...
- 25
0
votes
2
answers
40
views
Combine two rows of date strings in a dataframe
I have a dataframe that looks similar to the one below:
Date A B C
01/01/2020 a1 b1 c1
03/17/2020 a2 b2 c2
03/30/2020 a3 b3 c3
06/24/2020 a4 b4 c4
06/25/2020 a5 b5 c5
11/01/...
- 83
0
votes
3
answers
64
views
Get months from dates
I have this tibble with a bunch of dates, below is a sample of the tibble.
library(tibble); library(lubridate)
c("25/10/1952", "27/05/1961", "07/11/1962", "20/12/...
- 13.8k
1
vote
1
answer
57
views
data.table::shift type="lag" allowing extending the data
I am using data.table::shift to lag my time series. What I would like now is to expand the dates to account for the shift beyond the time series maximum date.
For example, if I have a monthly time ...
- 3,508
-2
votes
2
answers
94
views
R - remove dates five days before and after a given list of dates [closed]
Let's suppose I have the following date data.frame df_date that contains a given number of dates in the format yyyy-mm-dd.
Based on dates of df_date I need to remove all the corresponding dates before ...
- 177
0
votes
0
answers
33
views
Changing x-axis intervals
I am very new to R-studio. I cannot support my question with code because I cannot figure out how to copy and paste code from R-studio Editor into a question, likewise results from Console.
I have ...
- 13
0
votes
2
answers
90
views
To find monthly average ndre and ndvi values
So, I have this data,
data = {
'farm_Id': ['farm_258', 'farm_344', 'farm_345', 'farm_346', 'farm_348'],
'2018-12-16_NDRE': [0.406, 0.380, 0.449, 0.432, 0.407],
'2018-12-16_NDVI': [0.581, 0....
- 5
1
vote
2
answers
128
views
dropping specific rows through datetime column by date
I am trying to drop specific rows of dates where there are only one class, so far there are no errors shown during the isolating of single class dates but the deletion part is where I'm having ...
- 57
4
votes
4
answers
85
views
Group and count date in R
I have a data set that describes the number of days (BedDays) each patient who stays in a hospital:
my_df<-data.frame(RecordID=c("1","2","3","4","5"...
- 75
0
votes
2
answers
104
views
Pandas dataframe column datetime format
Given the following dataframe, how do I change the datetime format from "2023-12-11 00:00:00" to "2023-12-11" for all columns?
When I print the dataframe, colum header are ...
- 3
0
votes
1
answer
43
views
Flag consecutive dates Python pandas
I have the dataframe
data = {
'MSISDN': [x12, x12, x12, x23, x23],
'SERVICE_FROM': ['2023-02-27 15:50:29', '2023-07-19 08:33:19', '2023-10-09 07:45:49', '2023-06-01 09:07:37', '2023-08-22 ...
- 373
1
vote
1
answer
131
views
How to convert string with abbreviated month name to Date using as.Date()?
I have a column that contains date and time separated by " " which I was able to split into 2 columns called "dates" and "time". Now I want to select the non empty rows ...
- 11
1
vote
1
answer
438
views
How to alter the date format of date column without changing the column data type using polars dataframe
How to alter the date format of date column without changing the column data type using polars dataframe.
I have dataframe as below
I wanted to convert the first column (i.e Date) to %d-%b-%y format ...
- 638
0
votes
2
answers
53
views
Data Frame- Remove all after year but keep information about year
My DF looks like below:
id date
1 21 July 2023 (abcd)
2 22 July 2023 00:00:01
3 23 July 2023 -abcda
I need to remove all after year (2023) but I want to keep it. So the result should be:
id date
1 ...
- 403
1
vote
2
answers
65
views
Get next date after an arbitrary date in a Python Series
I have a DataFrame in Python that has a "date" column (and several other columns).
What I want to do is to split off two DFs based on an arbitrary date in the "date" column. Let ...
- 149
0
votes
1
answer
48
views
Cross-referencing data frames to select rows within specific date ranges
I have two data frames:
1.df1 has information about exams done in a cohort, and has the following structure:
PersonID
ExamDate
ExamResult
001
02/03/2023
Positive
001
08/09/2019
Negative
002
01/30/...
- 1
1
vote
1
answer
81
views
Dataframe: Using "Name" Column as Segmentation, Get all dates between Startdate and Enddate columns While Excluding Weekend Dates
Let's say I have the starting input table below.
Name
_leavestartdate
_leaveenddate
Jerry Rice
2023-12-26
2023-12-26
Pablo Picasso
2023-08-15
2023-08-21
Ray Ramon
2023-03-16
2023-03-17
The Python ...
- 5,214
-2
votes
2
answers
36
views
Data Frame Export to Excel is not in the Desired Format
When I try to export the data frame into my output.xlsx file, the date format is in YYYY-MM-DD HH:MM:SS. My desired output is YYYY-MM-DD (or without time). Here is my code:
import pandas as pd
# ...
0
votes
1
answer
44
views
How to create new column in Data Frame to sort values for each id based on date column in Python Pandas? [duplicate]
I have Data Frame in Python Pandas like below:
Column my_date is in datetime format.
In my real DataFrame I have many more columns.
id
my_date
col1
111
2023-05-15
1
111
2023-05-14
11
111
2023-05-...
- 2,342
0
votes
1
answer
79
views
Get the peak value per cycle
I would like to get date time include ns in bellow strings.
0 2023-11-02 12:00:00.039\t-201
1 2023-11-02 12:00:00.996\t-201
2 2023-11-02 12:00:01.396\t-198
3 2023-11-02 12:00:01....
- 13
2
votes
3
answers
124
views
How to create new rows in Data Frame based on values in 2 columns managing also values in another columns in Python Pandas?
I have Data Frame in Python Pandas like below:
Input data: (columns survey and event are object date type)
data = [
(1, 2, 1, 0.33, '2023-10-10', '2023-09-25', 1, 20, 11),
(1, 2, 1, 0.33, '...
- 2,342
0
votes
2
answers
59
views
How to fill NaN values in 3 columns based on group of values in 2 other columns and stay untouchd values in rest cols in Data Frame in Python Pandas?
I have Data Frame in Python Pandas like below:
data = [
(1, None, None, None, '2023-01-10', None, None),
(1, None, None, None, '2023-01-10', 1, 0),
(1, 9, 0, 0.55, '2023-01-10', 15, None),
...
- 2,342
0
votes
3
answers
82
views
How to fill NaN values in 3 columns based on group of values in 2 other columns in Data Frame in Python Pandas?
I have Data Frame in Python Pandas like below:
data = [
(1, None, None, None, '2023-01-10'),
(1, None, None, None, '2023-01-10'),
(1, 9, 0, 0.55, '2023-01-10'),
(2, None, None, None, '...
- 2,342
2
votes
2
answers
137
views
How to create additional rows for each combination of values in 2 columns based on data in third column in Data Frame in Python Pandas?
I have Data Frame in Python Pandas like below:
data = [
(1, '2023-10-10', '2023-09-25', 1, 20, 11),
(1, '2023-10-10', '2023-10-04', 0, 10, 10),
(1, '2023-05-05', '2023-05-01', 0, 10, None),...
- 2,342
0
votes
3
answers
74
views
How to stay for each id in Data Frame only rows for one random month from column with dates in Python Pandas?
I have Data Frame in Python Pandas like below:
df = pd.DataFrame({'id': [1, 1, 1, 2, 2, 2],
'day': ['2023-03-01', '2023-03-10', '2023-04-05', '2023-03-15', '2023-04-20', '2023-04-25'...
- 2,342
2
votes
2
answers
60
views
How to expand Data Frame to artificially create dates up to the 31st day in each month no matter how many days that month has in Python Pandas?
I have Data Frame in Python Pandas like below:
data = {
'id': [1, 1, 2, 2, 3],
'nps': [8, 8, 7, 7, 5],
'target': [True, True, False, False, True],
'nps_score': [0.56, 0.56, 0.6785, 0....
- 2,342
0
votes
1
answer
44
views
dynamically add a sequential date column to a data frame with a starting with each date is repeated N times in consecutive rows
I've got the following data.frame:
import pandas as pd
import random
data = {
'Column1': [random.randint(1, 100) for _ in range(9)],
'Column2': [random.uniform(0, 1) for _ in range(9)],
'...
- 1,420
1
vote
1
answer
33
views
R Find the earliest date of a group and create a factor variable Y/N to indicate if it is the first date of the group
I have a dataframe that looks like that:
Issuer.Name Issue.Date
name1 01/12/2021
name2 05/04/2022
name2 21/10/2021
name3 08/09/2020
name4 30/08/2023
name4 12/05/2021
name4 ...
- 63
1
vote
1
answer
35
views
How to transform many variables (characters) to the dmy format using `dmy` function?
Let's say I have this df (but so much bigger)
library(lubridate)
df = data.frame(id=1:5,date1 = c("2023/01/02","2023/08/03","2023/09/09","2023/11/05","...
- 911
1
vote
2
answers
46
views
How to check if a series of dates are coherent in a dataframe?
Let's say I have this example df (but with so many dates) :
library(lubridate)
df = data.frame(id = 1:5,
date1 = dmy(c('01/02/2021','03/05/2022','01/03/2021','08/12/2021','01/02/2021')...
- 911