All Questions
2,350 questions
1
vote
1
answer
28
views
Assigning column from different data frame - role of index
import pandas as pd
df_1 = pd.DataFrame({'col1': ['a', 'a', 'a']})
df_2 = pd.DataFrame({'col1': ['b', 'b', 'b']})
df_2.index = [4,5,6]
df_1['col2'] = df_2.col1
I expect a simple copy in the above ...
- 7,712
2
votes
3
answers
90
views
how to extract a cell value from a dataframe?
I am trying to extract a cell value from a dataframe, then why I always get a series instead of a value.
For example:
df_test=pd.DataFrame({'Well':['test1','test2','test3'],'Region':['east','west','...
- 4,162
0
votes
0
answers
32
views
Algo trading Drawdown calculations SettingWithCopyWarning:
I am trying to calculate maximum drawdown using closing price of a stock.
STEP1: Imported the data and sliced that data in a new data frame df1 with date and closing price.
df = pd.read_csv('BHEL....
- 1
0
votes
0
answers
15
views
How to merge Headers of index and other headers to one row of headers and add numeric index as single index [duplicate]
I currently have this type of a df with 2 headers
client case client payment payment number
client
foo 1 200 1
boo 1 250 ...
- 21
0
votes
1
answer
49
views
Pandas: Nested use of .loc returns different 'type' for the same field
Why do I get 2 different data types while indexing using .loc, even though i retrieve a single value/cell.
Background: I'm looking up values of df_source in df_map. I do not get string datatype for ...
- 381
2
votes
3
answers
81
views
Find duplicate "group of rows" in pandas DataFrame
How can I find duplicates of a group of rows inside of a DataFrame? Or in other words, how can I find the indices of a specific duplicated DataFrame inside of a larger DataFrame?
The larger DataFrame:
...
- 345
2
votes
1
answer
71
views
Pandas .loc returns series or float inconsistently
I'm writing some code, and having trouble with some unexpected behaviour when using .loc in a pandas DataFrame, depending on the length of the dataframe itself. Some guidance on what is going on and ...
- 105
0
votes
1
answer
50
views
How can I multiply all company values by 'Total General' values while keeping 'Total General' unchanged in a DataFrame?
I have a Pivot table in Python with the following structure.
I want to create a new DataFrame where:
The values of all companies (Amazon, Chedraui, Jüsto, Rappi, Uber) are multiplied by the ...
2
votes
2
answers
121
views
How can Pandas .loc take three arguments?
I am looking at someones code and this is what they wrote
from financetoolkit import Toolkit
API_KEY = "FINANCIAL_MODELING_PREP_API_KEY"
companies = Toolkit(["AAPL", "MSFT&...
0
votes
1
answer
55
views
Multi-index lookup between 2 dataframes
I have 2 dataframes: one that acts as a lookup dataframe, and another that I insert values into where numerous rows match between them.
The lookup dataframe looks like this:
data1 = {
'store': ['1'...
- 411
0
votes
2
answers
54
views
complicated pandas operations involving 2 dataframes [duplicate]
I have a dataframe df_a that has a column called dr7objid and has 194478 rows. There's a 2nd dataframe df_indices with 245609 rows that also has an objid column as well as an asset_idcolumn. Both are ...
0
votes
1
answer
45
views
Create different columns using rows
I have this
input
ob1
88
ticket1
information ticket1
ob1
88
ticket2
information ticket2
and I need to get this
output
ob1
88
ticket1
information ticket1
ticket2
information ticket2
I'm using ...
2
votes
2
answers
93
views
Get a single row in a tuple-indexed DataFrame
I have a pandas DataFrame:
>>> f = pd.DataFrame.from_dict({"r0":{"c0":1,"c1":2},("r",1):{"c0":3,"c1":4}},orient="index")
...
- 59.9k
3
votes
1
answer
52
views
Python Pandas difference in boolean indexing between ~ != and ==
I am confused about different results of boolean indexing when using ~ after != versus when using just ==
I have a pandas df with 4 columns:
dic = {
"a": [1,1,1,0,0,1,1],
"b&...
- 55
0
votes
1
answer
39
views
Create a new column based on other columns for time series data in pandas
I have the following pandas dataframe with columns May, June, and July.
Month
June
July
Aug
June
a
d
g
July
b
e
h
Aug
c
f
i
I want to create a several new columns with a 1 month forecast, 2 month ...
- 1
1
vote
3
answers
87
views
How to create a dynamic index in a data frame based on a catalog?
I have two diferent data frames. The first one is a catalog of diferent requests type and how many request i had at the moment.
data = {'Req Type': ['A', 'B', 'C', 'D'], 'Req No': [20, 21, 19, 18]}
...
1
vote
2
answers
60
views
Why is no warning thrown for indexing a Series of values with a bool Series that's too long?
I have the following code:
import pandas as pd
series_source = pd.Series([1, 2, 3, 4], dtype=int)
normal_index = pd.Series([True, False, True, True], dtype=bool)
big_index = pd.Series([True, False, ...
- 4,978
0
votes
2
answers
61
views
Why does pandas .reindex() not keep data with equivalent sized index? [duplicate]
Lets say we have this dummy data :
dates=pd.date_range("2020-02-01","2021-02-01",freq="MS")
features=["foo","bar"]
cols=pd.MultiIndex.from_product([...
2
votes
1
answer
81
views
How to predict the resulting type after indexing a Pandas DataFrame
I have a Pandas DataFrame, as defined here:
df = pd.DataFrame({'Name': ['Alice', 'Bob', 'Aritra'],
'Age': [25, 30, 35],
'Location': ['Seattle', 'New York', 'Kona']...
- 4,978
3
votes
2
answers
76
views
unstack on single row index
data = {
'Store': ['Store_A', 'Store_A', 'Store_B', 'Store_B'],
'Product': ['Apples', 'Bananas', 'Apples', 'Bananas'],
'Sales': [200, 350, 150, 500]
}
df = pd.DataFrame(data)
df.set_index(...
- 180
2
votes
3
answers
166
views
How to use `pycountry.db.Country` objects as a `pd.DataFrame` index?
I am creating a dataset collecting data for a given set of countries. To avoid any ambiguity, I would like to use a pycountry.db.Country object to represent each country.
However, when setting the ...
- 1,457
1
vote
2
answers
48
views
Pandas reset_index() not working on grouped dataframe
I have a dataframe df as follows:
Year
Month
Rate
0
2022
1
0.1
1
2022
2
0.1
2
2022
3
0.1
3
2022
4
0.1
4
2022
5
0.1
5
2022
6
0.1
6
2022
7
0.1
7
2022
8
0.1
8
2022
9
0.1
9
2022
10
0.1
10
2022
...
- 45
3
votes
3
answers
72
views
Multiply two different Dataframes - Python
I have two tables:
Table 1: (Sales)
Code
2025
2026
2027
123
20000
21000
22000
456
10000
12000
14000
Table 2: (Inflation)
Code
2020
2021
2022
2023
2024
2025
2026
2027
2028
123
0.6
0.7
0.8
0.9
1
1....
2
votes
1
answer
57
views
Lookup by datetime in timestamp index does not work
Consider a date-indexed DataFrame:
d0 = datetime.date(2024,5,5)
d1 = datetime.date(2024,5,10)
df0 = pd.DataFrame({"a":[1,2],"b":[10,None],"c":list("xy")}, index=...
- 59.9k
0
votes
0
answers
23
views
python (Apply inflation table to sales table) [duplicate]
I have the below tables:
Sales Table:
enter image description here
Inflation Table:
enter image description here
I want python to update table one values by multiply each value in table one with ...
0
votes
1
answer
60
views
Name all rows with the Index name
I have a strange request, easy to understand but should be very complicated to realize in my opinion. I have a dataframe composed of several spreadsheet excel (using df concat).
I would like to use ...
- 15
2
votes
2
answers
58
views
How to correctly index a dataframe with a function?
Given a dataframe:
a b c
u 5 0 3
v 3 7 9
w 3 5 2
I'd like to select rows/columns in a dataframe using a function. This function gets the dataframe and returns a tuple of lists of labels,...
- 7,489
1
vote
1
answer
52
views
MultiIndex Pivot Table Pandas [closed]
I am trying to replicate a pivot table using pandas using a sample dataset below from a much larger table:
year type status paid balance count
2000 bank1 active 15 21 1
2001 ...
- 33
0
votes
0
answers
197
views
Creating Pandas Series from Dictionary
I'm going through the Python Data Science Handbook to learn about the Pandas library. The book says, and also shows an example, that when a Series is created from a dictionary, the index defaults to ...
- 17
0
votes
0
answers
32
views
While saving Datapane report using pandas library, I'm getting an attribute error saying that pandas has no attribute 'Int64Index [duplicate]
I'm having issues saving my datapane report because I keep getting this error message: "AttributeError: module 'pandas' has no attribute 'Int64Index'." What does this error mean, and how can ...
0
votes
3
answers
56
views
Index of duplicated values in pandas
I'm using pandas in python and using the duplicated(keep=False) I've made a check between the values of a same dataframe column. So I'm getting as result:
5063 True
5064 True
5065 True
...
- 285
1
vote
1
answer
80
views
how does Pandas actually perform indexing on custom based indices (integer and non-integer)
temp = pd.Series(np.random.randint(1, 10, 5), index=['John', 'Joe', 'Bob', 'Alice', 'Kris'])
temp[-1]
this outputs to the value associated with 'Kris'
however, when I do this:
temp2 = pd.Series(['...
- 13
1
vote
3
answers
50
views
Connecting a dataframe and another 1d dataframe isn't a problem, but when using the [...] operator it yields different results
I have an original dataframe og_df and a sublist dataframe, which is a part of the og_df. I want to create a new dataframe new_df which contains the elements of og_df and every n following elements in ...
- 165
3
votes
2
answers
64
views
filling an empty data frame or array with values from the column of another data frame if some conditions are met pandas
I need to creat an empty data frame that stores values from a column of another data frame base on some conditions being met in two columns of the same second data frame.
I have a data frame
...
- 33
0
votes
0
answers
41
views
How to apply python iloc on snakemake input file?
More in detail:
I would like to choose one specific column based on a header value from snakemake rule input file and then select this specific column's first value. I was considering something like ...
- 1
0
votes
1
answer
50
views
Can I add Multilevel Indexing for one-hot encoded features?
I am working on a dataset of mushroom features, almost all of which I encoded with pandas into binary but some are nominally encoded. I am wondering if I can take the original columns as a second ...
1
vote
1
answer
76
views
Concatenating DataFrames throws InvalidIndexError
I'm extracting information from URLs (RSS feeds) to create one big data frame with all the data I need for sentiment analysis. I did a function to get each URL in a dictionary, use a parser, and then ...
- 11
0
votes
1
answer
49
views
Make index and columns the same set (their union) in Pandas dataframe
In our problem, rows (index) and columns belong to the same category of objects. We want to enlarge a Pandas DataFrame, adding rows and columns filled with NaNs or predefined values, so that both the ...
- 49
0
votes
0
answers
31
views
Append Columns From Multiple CSVs into One File - Python [duplicate]
I am trying to use the Pandas library to append columns multiple CSV files of the same format into a single file, but cannot seem to get the syntax correct in my script. Here are the CSVs I am trying ...
- 349
1
vote
1
answer
48
views
Setting the index where the index value changes every 17 or so rows
I have pulled data for a bunch of Survivor seasons and have sorted the contestant tables into one data frame. I have added in a column to identify which season a contestant played in. I want to set ...
2
votes
2
answers
44
views
Renaming Panda Dataframe indexes
I am populating a dataframe with a sql statement which is giving me index values of a record count starting with 0. I am writing a comparison with a Snowflake table that I will import into another ...
-2
votes
1
answer
84
views
updating a row in python
I am trying to update a row in a table to the previous row with indexing. I am having trouble updating my real dataset and cannot reproduce the error on a toy example.
gd.loc[gd['Date'] == ts,teams] =...
- 177
0
votes
2
answers
320
views
How to insert a row into a pandas dataframe in a for loop then decrement the for loop?
So I'm trying to for loop through a table and find missing rows and add them in.
S-Value
S1
S2
S3
...
S106
S108
S109
So S107 is missing.
Currently I'm checking if count = S-value if they equal ...
- 13
-2
votes
1
answer
42
views
I need to drop rows that do not contain a certain value throughout the table
I have the following dataset:
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 513250 entries, 0 to 513249
Data columns (total 26 columns):
# Column Non-Null Count Dtype
--- ------ -----------...
1
vote
2
answers
72
views
Wrong datatime index in Pandas DataFrame with class
I have a Pandas DataFrame with and without class. In case of class I have wrong datatime index but if no class it is ok. Please help.
from datetime import datetime
import pandas as pd
class Records:
...
- 81
-3
votes
1
answer
64
views
How to make a counter based on other columns within the same row
I have a large dataframe that contains multiple kids' exact age and gender, along with some unrelated binary columns, as follows:
kid1_age kid2_age kid3_age kid1_gndr kid1_gndr ...
- 1
1
vote
1
answer
75
views
labeling segments of data based on multiple conditions
I have an indexed pandas dataframe with values like so:
data = {'ID':[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1],
'column1': [15, 16, 17, 14, 13, 5, 3, 2, 1.9,...
- 1,246
0
votes
1
answer
52
views
indexing periods in a dataset that spans an increasing and decreasing range
I have a dataframe like so:
data = {'column1': [1, 2, 3, 4, 5,4,3,2,1,1, 2, 3, 4, 5,4,3,2,1]}
df=pd.DataFrame(data)
column1
0 1
1 2
2 3
3 4
4 5
5 ...
- 1,246
0
votes
1
answer
214
views
transpose row into column for each date in time series using pandas dataframe
I have a series of swaption time series data for each expiry & tenor for the last year - I am looking to transpose it into a simple grid for each day (using date as index) using pandas dataframe (...
- 117
2
votes
1
answer
3k
views
reset_index acting in an unexpected way - unexpected keyword argument 'names'
I'm cleaning up a dataset for plotting. Pretty standard stuff, transposing, deleting unnecessary columns, etc.
Here is the code so far:
#File name
fname = r'E:\Grad School\Research\BOEM\Data\Grain ...
- 75