All Questions
Tagged with dictionary pandas
4,821 questions
1
vote
2
answers
42
views
Update Pandas DataFrame slice row-wise using dictionary
Question
I am trying to update the values in a pandas (version 2.2.3) dataframe by row (such that each row has the same values) using a dictionary with row indices as keys and row values as values.
In ...
- 161
-2
votes
1
answer
60
views
Arbitrary depth tree represented by a nested dictionary of lists to pandas DataFrame
I am trying to get a pandas dataframe from a nested dictionary that contains lists of children nodes. The depth and amount of nodes is completely uneven. Here I put an example of the dictionary that I ...
- 13
0
votes
1
answer
49
views
flatten dictionary with dataframe value to a dataframe
This encoding process will generate a mapping between each categorical value and its corresponding numeric value:
import category_encoders as ce
cols_a = ['group1', 'group2']
dfa = pd.DataFrame([['A1'...
- 1,136
0
votes
0
answers
92
views
Store iteration result in dict
I have this code and I face a problem:
# Prepare the DataFrame to store results
columns_accuracy = [
'method', 'accuracy_mohr_F1', 'accuracy_non_mohr_F1', 'accuracy_mohr_F1_total',
'...
- 121
1
vote
0
answers
36
views
Attempting to flatten a list of dictionaries to a Dataframe but can't get to the lowest level information with json-normalise [duplicate]
Below is the output of an API call - it is a dictionary which contains a list of dictionaries I'm interested in under 'response'
I filter fixture_ids for 'response' because the rest is not useful ...
- 13
2
votes
1
answer
66
views
Apply pandas dictionary with gt/lt conditions as keys
I have created the following pandas dataframe:
ds = {'col1':[1,2,2,3,4,5,5,6,7,8]}
df = pd.DataFrame(data=ds)
The dataframe looks like this:
print(df)
col1
0 1
1 2
2 2
3 3
4 ...
- 1,592
1
vote
2
answers
52
views
Assign multi-index variable values based on the number of elements in a dataframe that match a selection criteria
I have a large csv dataset the looks like the following:
id,x,y,z
34295,695.117,74.0177,70.6486
20915,800.784,98.5225,19.3014
30369,870.428,98.742,23.9953
48151,547.681,53.055,174.176
34026,1231.02,73....
- 6,107
1
vote
1
answer
45
views
How to merge dictionary of dataframes and make key values a column in merged dataframe
I have a dictionary of about 600 dataframes for connectome metrics for those 600 participants. I'm making a dummy dictionary though for conciseness. The dataframes all have 1 row. Ignore the fact that ...
- 851
2
votes
2
answers
55
views
transform an exploded data frame into a deeply nested dictionary, with headers
The function I am using to convert my data frame into a nested dictionary strips the column names from the hierarchy, making the dictionary diffictult to naviagte.
I have a large dataframe that looks ...
- 21
0
votes
4
answers
97
views
How to create a dictionary with distinct values from pandas dataframe?
I have a dataframe which looks like this:
col_a
col_b
a
1
a
2
b
1
b
2
b
3
b
4
c
1
c
2
c
3
I want to create a dictionary which looks like this (distinct/unique value for every key):
{a:1, b:...
- 53
1
vote
2
answers
66
views
Pandas Map Dictionary with Multiple Values to Dataframe
I have the following dataframe:
data = [['BALANCED', 'Corp EUR'], ['YIELD', 'Corp USD'], ['GROWTH', 'HG CHF']]
df = pd.DataFrame(data, columns=['STRATEGY', 'ASSET_CLASS']
df
STRATEGY ...
- 193
0
votes
2
answers
75
views
Converting JSON list with multiple nested dictionaries to csv or excel
I have a JSON that I download from a website that has multiple nested dictionaries inside the main list. This is a very simplified version of it.
[
{
"id": 1,
"...
- 9
1
vote
2
answers
72
views
Unpack values from column, which consist of list of dictionaries
I have dataframe, which looks like this:
import pandas as pd
df = pd.DataFrame({"item_id": [1, 2, 3],
"item_properties":[
...
- 11
2
votes
2
answers
67
views
find value in column, which contains list and take another value from next column and put it in first table to new column
I have two tables
df1 = pd.DataFrame([{'a': 1}, {'a': 2}, {'a': 8}])
df1['b'] = ""
df2 = pd.DataFrame([{'e': [1,2,3], 'f': 1},{'e': [4,5,6], 'f': 2},{'e': [7,8,9], 'f': 3}])
e f
I ...
- 23
1
vote
1
answer
38
views
Can't access local dataframe from dictionary comprehension expression [duplicate]
I am using dictionary comprehension to compare each dataframe from within a dictionary to its corresponding dataframe in locals(). For some reason, I am getting a key error when accessing the locals()...
- 1,115
0
votes
1
answer
92
views
Why does converting a nested python dictionary into a pandas dataframe result in "has no attribute 'items' error?
I have a nested dictionary stored in the variable nested_dict_variable.
The dictionary is retrieved by using SPSS valueLabels Property (Python)
type(nested_dict_variable) results in dict.
print(...
- 105
1
vote
1
answer
59
views
Creating a column based on column values and dictionaries using vectorization in pandas
I have three columns in a pandas dataframe A, B, and C. I also have 3 dictionaries, dict_A, dict_B, dict_C that have all values of A, B, and C as keys in each respective dataframe. I am trying to ...
- 69
0
votes
1
answer
39
views
Pandas Dataframe apply function multiple columns
product_code base_price promo_type q_bef q_aft
0 P11 190 50% OFF 34 52
1 P03 156 25% OFF 393 322
2 P07 300 ...
1
vote
1
answer
55
views
Fill Column Values using a dictionary and pattern matching
I am working on categorizing credit card transaction. Right now I am using a dictionary combined with np.select() as follows:
def cat_mapper(frame, targ_col, cat_col):
...
- 75
0
votes
0
answers
45
views
how to get list of values from nested panda dictionary and insert it in to a column [duplicate]
I have a nested dictionary which looks like following:
{'NSE_FO:NIFTY28DEC21000CE': {'ohlc': None,
'depth': {'buy': [{'quantity': 0, 'price': 0.0, 'orders': 0},
{'quantity': 1, 'price': 3.0, '...
- 111
0
votes
3
answers
75
views
CSV data format to nested dictionary
I have below csv format. I want it to convert some nested dict.
name,columns,tests
ABC_ESTIMATE_REFINED,cntquota,dbt_expectations.expect_column_to_exist
ABC_ESTIMATE_REFINED,cntquota,not_null
...
- 118
1
vote
1
answer
312
views
How to get the values of a dictionary type from a parquet file using pyarrow?
I have a parquet file which I am reading with pyarrow.
In [83]: pq.read_schema('dummy_file.parquet').field('dummy_column').type
Out[83]: DictionaryType(dictionary<values=string, indices=int32, ...
- 462
2
votes
4
answers
94
views
converting a dataframe to dictionary in python?
I have a dataframe like below:
id|title|url|timed
2|xyz|www.xyz.com|2024-02-01T00:00:00Z|
3|ghy|NaN|2024-03-05T00:00:00Z|
4|None|NaN|2024-04-05T00:00:00Z|
56|ghy|www.ghy.com|2024-05-05T00:00:00Z|
...
- 93
0
votes
1
answer
32
views
Convert a dataframe with lists to a dictionary [duplicate]
I have a dataframe that contains a identifier and list in each cell. The idea is to create a dictionary in which the keys are the identifiers and the values are a list with the elements that appears ...
0
votes
1
answer
85
views
Unordered txt file contents: How to design in proper dictionary
I have txt file and it's contents are unordered like below sample. I must select first row because it has train run exact time.
my txt file has couple of summary 1, 2 and so on. hence, keys are same ...
- 53
1
vote
1
answer
55
views
How to merge dictionaries contained in a Pandas dataframe as a groupby operation
Let us consider a pandas dataframe df containing dictionaries in one of its columns (column mydict):
mystring mydict
0 a {'key1': 'value1'}
1 a {'key2': 'value2'...
- 4,633
0
votes
1
answer
57
views
Iteratively source a value from a dictionary
I have a data set where I know a limited number of values and based on other values in a data set I can figure out what the subsequent values are. I could do this with a huge if function, but I know ...
- 597
2
votes
2
answers
54
views
Dyamically Update Column in Python DataFrame using substrings
Lets say I have a data frame lets call it A
Key_Word
Code
Market
A1
Theater
A2
And I have another Data Frame lets call it B
Sentence
Components_from_A
John went to the theater
Mary went to the ...
- 67
0
votes
1
answer
55
views
Is there a parameter problem, or not working with this method?
I wanted to predict the disease type using a function with a given string parameter. The parameter can contain more than one string. But it always gives an error message: KeyError: 'skin_rash'
This is ...
0
votes
1
answer
56
views
How do I create a multi-indexed DataFrame from a double-nested dictionary?
I am trying to create a pandas multi-indexed DataFrame from a double-nested dictionary but am quite stumped as to how to do this.
As some background I have scraped some data from a website and put it ...
4
votes
1
answer
32
views
Python: MultiIndex Dataframe to json-like list of dictionaries
I want to store this data frame
df = pd.DataFrame({
'id':[1,1,2,2],
'gender':["m","m","f","f"],
'val1':[1,2,5,6],
...
- 839
1
vote
2
answers
31
views
Creating a dataframe from a column
I have a dataframe df which has a column prediction in it. The column prediction has values which are list of dictionaries. A typical row of the column prediction looks like this :
[{'answer': 'My ...
- 195
2
votes
2
answers
102
views
Pandas Map Multiple Columns With A Filter
I have a dataframe like so (simplified for this example)
Site LocationName Resource#
01 Test Name 5
01 Testing 6
02 California ...
- 8,564
0
votes
0
answers
196
views
Creating Pandas Series from Dictionary
I'm going through the Python Data Science Handbook to learn about the Pandas library. The book says, and also shows an example, that when a Series is created from a dictionary, the index defaults to ...
- 17
1
vote
1
answer
62
views
converting a dictionary of nested lists into a row in a data frame
I have some data (shown below), which is a dictionary of nested lists of dictionaries.
I want to make the whole dictionary into one row. A very wide row.
At present I can get my desired result. It is ...
- 81
1
vote
2
answers
77
views
Use json_normalize with extraneous dictionary layer with single key?
I have automated identification system (AIS) data as nested dictionaries in a pandas dataframe. Here is an example:
dfAIS
Out[18]:
Message ... ...
- 1,115
0
votes
0
answers
28
views
How to create a dataframe of counter.keys and counter.values
I want to create a dataframe from 1 column that contains 1 of each variable in one column and the count in another.
This is what i have tried, i know zipping a and b is useless because its already a ...
- 29
1
vote
1
answer
60
views
How to parse nested dictionaries into Pandas DataFrame
I have a pretty crazy dictionary that I'm trying to parse out into a pandas dataframe. Here is a smaller version of what the dictionary looks like:
import datetime
from decimal import *
test_dict = [...
- 911
0
votes
1
answer
34
views
Row bind multiple data frames in dictionary using dict.keys() as column 1 of output pd.df
i found a way to concatenate my pandas data frames in a dictionary.
Que; how can i use the dictionary keys as first column in the final dataframe?
i tried but thar didn't work
pd.concat(test_list....
- 485
0
votes
2
answers
66
views
python dict is giving key error even if initialized with some values
data={}
vega={}
for coin in ['BTC']:
vega[coin] = {}
data[coin] = {}
data[coin]['columns']=['27-Apr-24', '28-Apr-24', '29-Apr-24', '03-May-24', '10-May-24', '17-May-24', '31-May-24']
...
- 326
0
votes
1
answer
91
views
How to Explode the Key value pair of dictionary into multiple rows of dataframe in Python
I have a column in dataframe that contains value in the form of dictionary. Now I want to explode that into multiple rows of the dataframe?
For example:
This is my input dataframe
I want something ...
- 71
1
vote
2
answers
42
views
Add column to existing dataframe based on condition to a few rows
table.loc[sub_table_condition, col_name] = value
i want to extend this condition where my value is a list of dict, it can have 1 dict, multiple dict or empty list.
i tried using
table.loc[...
- 79
0
votes
0
answers
18
views
How to match pandas column of lists to a dictionary [duplicate]
I have survey results that have output numbers rather than the countries they correspond to.
So I have a data frame like this:
company
geography
ABC
[1,3,5]
DEF
[8,9,12]
I have created a dict like ...
- 31
0
votes
2
answers
61
views
How can i insert x dictionary in a DataFrame?
I have to insert a series of dictionaries into a dataframe that I generate via a for loop, I would like to make the keys of the dictionaries become the labels of the columns of the dataframe and ...
1
vote
1
answer
58
views
Create Dict in Python from excel data
I want to create a dictionary from the values from excel cells. For example, the table is below:
Column 1
Column 2
Column C3
a
b
c
d
e
f
and I want to create a dictionary, like below, that ...
- 11
1
vote
3
answers
82
views
create columns dynamically depending on string structure in Python
To give background on my project, comparing two documents which has nested JSON structure using deepDiff in python. During comparison, if values of a field are changed, those are written to a ...
- 438
0
votes
2
answers
60
views
Is there a function in pandas which lets you create columns for dictionary key + value pairs efficiently?
I have a list of form [[[key1,value1],[key2,value2],...,500],...,n] - aka L2 orderbook data with 500 levels.
I want to translate this list into a dataframe, containing columns key_1,...,key_500 and ...
2
votes
2
answers
61
views
Python - Converting Date:Price list to new rows
I am trying to convert the following column into new rows:
Id
Prices
001
["March:59", "April:64", "May:62"]
002
["Jan:55", ETC]
to
id
date
price
001
March
...
- 53
0
votes
1
answer
51
views
Converting a List of Dictionaries to Pandas Dataframe
I have this code:
import urllib.request, json
url = "https://api.wto.org/timeseries/v1/indicator_categories?lang=1"
hdr ={
# Request headers
'Cache-Control': 'no-cache',
'Ocp-...
1
vote
1
answer
53
views
Concatenate the rows of each column being grouped by identify
Welcome Help.
I would like to merge multiple rows of each ID of a dataframe into a single cell, Below is input format:
I have the following list of columns and Dataframe in pandas respectively
dict1 =...
