All Questions
162 questions
1
vote
1
answer
43
views
How to create a DataFrame from GC-MS data with retention time, mass values, and intensities in Python with correct alignment?
I am processing gas chromatography - mass spectroscopy (GC-MS) data stored in a NetCDF file in Python, and I’m having trouble aligning the data correctly to create a structured DataFrame. My goal is ...
- 13
1
vote
1
answer
130
views
Python netcdf EOF analysis
I am performing a EOF analysis for JJAS NDVI.
Here is the NDVI data (https://drive.google.com/drive/folders/1-QBkzTlAJRq8etuD15x_7l08woFTR86Y?usp=sharing) and codes.
from netCDF4 import Dataset
...
- 47
1
vote
1
answer
44
views
Python NetCDF Prime Meridian Line blank [duplicate]
I am trying to draw climatology map of GPCC precipitation.
But I found out there is a blank (data cut) in Prime Meridian Line.
How can I fix the problem? I can use CDO, NCO, Python.
I also share the ...
- 47
0
votes
1
answer
55
views
Computing and Visualizing Linear Trends in Ocean Temperature Data
I have a dataset containing ocean temperature values for the time period 1950-2022, covering the entire globe. My goal is to calculate the linear trend for each surface grid box and visualize the ...
- 21
1
vote
2
answers
48
views
Assigning values to numpy array based on multiple conditions of multiple array
I have the ocean and atmospheric dataset in netcdf file. Ocean data will contain nan or any other value -999 over land area. For this eample, say it is nan. Sample data will look like this:-
import ...
- 3,868
-1
votes
1
answer
84
views
How do I extract all the values from one variable in a NetCDF file with Python?
I am trying to use the CHESS-SCAPE dataset (found here - https://data.ceda.ac.uk/badc/deposited2021/chess-scape/data/rcp60/01/monthly) to get the surface wind speeds across the UK throughout the ...
- 1
2
votes
1
answer
150
views
Python xarray, numpy, matplotlib netcdf masking ocean?
I am trying to draw linear regression map of NDVI.
But using countour map, it filled the ocean too.
I would like to remove the ocean without values.
using Nan or maskoceans
but maskoceans is not ...
- 47
0
votes
1
answer
98
views
Finding the xy values from lat lon from SSMI/S sea ice concentration data
I want to find the corresponding ice concentration values that are present at a series of lat and lon coordinates (such as lon=-74.7732772827148 and lat=-69.3250350952148). However, the dataset I am ...
- 13
1
vote
2
answers
193
views
Speeding up finding the longest continuous sequence/run of values
I have 30 years masked dataset containing values for heatwave observed days. I want to calculate the longest heat wave event occurrence in each grid cell. The below provided code works fine when I ...
1
vote
1
answer
67
views
Spatial Correlation calculation in python: Concatenation Error-Mismatched array sizes along dimension 0
I am attempting to plot the spatial correlation between two variables, SIC and spco2, which are stored in separate NetCDF files. However, the spco2 dataset has a different coordinate type than the ‘...
- 43
0
votes
1
answer
28
views
Merge dataset after being converted from numpy.array to data array with attributes
I am trying to save some variables as a netCDF file.
I am taking these steps to do it.
plev = xr.DataArray(plev,coords=[levels,lat,lon], dims=["lev","lat","lon"])
...
- 39
1
vote
1
answer
353
views
Export NumPy array as NetCDF4 file
I'm trying to export a simple 2D NumPy array as a .nc (NetCDF) file. I'm trying to follow this example without exit.
import numpy as np
import netCDF4 as nc
# Array example
x = np.random.random((16,...
- 317
0
votes
1
answer
463
views
Read and Interpolate data from NetCDF file throws shape mismatch error
I have this netcdf file with data having latitude, longitude and temperature. The temperature has the dimension 103x61 in terms of latitude and longitude. I want to resize this 103x61 grid to 70x70 by ...
- 15
0
votes
0
answers
56
views
Getting [Errno 22] Invalid argument in datetime.utcfromtimestamp() [duplicate]
I am using python 3.10.6. I have this function which takes a netcdf file and assigns dmin and dmax the minimum and maximum dates of the time index from that file's data. Whenever the dmin is below ...
- 15
0
votes
0
answers
333
views
MemoryError: Unable to allocate 33.1 GiB for an array with shape (192, 384, 60265) and data type float64
I have a large NetCDF data set (192 latitudes, 382 longitudes, and 60265 time steps, each with some float values) and would like to copy it to store in the copy the results of an complex calculation.
...
1
vote
0
answers
215
views
How to identify or determine Contiguous Grid Cells of a extreme events using daily NetCDF data
I've been working with daily precipitation netcdf data with dimensions time:1322, lat:68, lon:136 over my domain. For each time step (date), I want to identify/determine contiguous grid cells, i.e., ...
1
vote
0
answers
782
views
Flipping/Inverting a Raster using NumPy and Rasterio
I am currently working on a project creating raster files using numpy arrays containing data from netCDFs. The rasters were created smoothly but when I plotted them, I realized they were actually ...
- 33
0
votes
2
answers
112
views
Python : Replace a column in a dataframe by datetime values
I'm trying to replace a column an array of 4 columns by datetime values that I treated. The problem is that it's difficult to keep the same form between the different formats of dataframe, array,....
...
0
votes
1
answer
663
views
Converting Unix timestamp to datetime NETCDF4
In my NetCDF file I have a group 'vel' containing the variable 'time' that I am using as my X-Axis when plotting with matplotlib and numpy.
I would like the plot X axis as datetime (rather than Unix ...
- 31
0
votes
1
answer
836
views
How can I open an arbitrary netCDF file with xarray and get the *n*th time slice as a NumPy array?
When I open a netCDF file with xarray in Python, I open it as a Dataset object:
ds = xr.open_dataset(file_path)
How do I get the nth time slice of this dataset as a NumPy array?
I know that I can get ...
- 3,892
3
votes
1
answer
216
views
Extract a time and space variable from a moving ship from the ERA5 reanalysis
I want to extract the measured wind from a station inside a moving ship, which I have the latitude, longitude and time values and the wind value for each time step in space. I can extract a fixed ...
0
votes
1
answer
814
views
How to compute 2, 3 and 5-days accumulated rainfall from daily rainfall data in netcdf using Python
I'm trying to compute 2, 3 and 5-days accumulated rainfall from grided daily rainfall data (GPCC precisely) in order to look for the largest event in space. When I applied .rolling(time=2).sum() it ...
0
votes
1
answer
803
views
how to select range of time of a variable from netcdf using python
I have a daily netcdf data containing a point data of single lat and lon. The shape of vairable (var.shape) is ('time', 'z'), here z is depths in ocean. I want to select and plot a varaible within a ...
- 69
0
votes
1
answer
169
views
Display Noaa Elevation Data
I have found the example and would like to copy it:
import numpy as np
import matplotlib.pyplot as plt
from netCDF4 import Dataset
data = Dataset("Path/ETOPO1_Bed_g_gmt4.grd",'r')
print(...
- 283
2
votes
0
answers
1k
views
How to find nearest grid points when latitude and longitude are functions of dataset coordinates?
I have a netcdf file of gridded temperature data, and a csv of weather stations. I am trying to find the grid points that are closest to the weather stations. The problem I'm having is that the ...
- 221
0
votes
1
answer
126
views
How to sort 3d numpy masked array on time axis at each grid i.e at specific latitude and longitude for climate dataset
I have a NetCDF file of Climate dataset having 3D structure with a shape of 20 * 445 * 445 as (time, latitude, longitude)
I have read it as numpy.ma.core.MaskedArray using netCDF4 library
I want to ...
1
vote
1
answer
1k
views
Average all rows/columns in 3D numpy array at each timestep (band)
I have a 3D array of type numpy.ma.core.MaskedArray. The data are 10m wind direction values at every lat/lon point in a 4x4 grid. The dataset is hourly so I have 87672 individual matrices (ten years ...
- 13
2
votes
1
answer
1k
views
How to find daily percentiles of gridded data with xarray?
I have a netcdf file of 10 years of gridded daily temperature data for the United States. I created a baseline period of just the first 5 years of data. I now want to find the 90th percentile for each ...
- 221
0
votes
1
answer
1k
views
Trying to plot a CSV file that contains latitude and longitude values
I first downloaded NetCDF4 data from this website. Now, I wanted to make the file more cloud friendly so I converted it into CSV. After that, my rows and columns in the CSV looked like this.
Now, I ...
- 516
-1
votes
1
answer
843
views
Is there another alternative to DataArray.sel(time = slice(x, y)) in xarray?
For some reason, the DataArray.sel(time = slice(x, y)) is working for me without any problem for the months of January to June, where x and y are both equal to values ranging from 1 for January to 6 ...
- 35
2
votes
0
answers
533
views
Regridding and plotting netcdf file with irregular grid
I am trying to regrid my .nc data with irregular grid with the following code:
from mpl_toolkits.basemap import Basemap
from netCDF4 import Dataset as NetCDFFile
import matplotlib.pyplot as plt
import ...
- 43
0
votes
0
answers
42
views
Finding the average of data storage in a list of masked array
I need to take an average of a bunch of data in different netcdf files, that contain the following dimensions: TOTAL(TSTEP, LAY, ROW, COL)
TSTEP is the number of days in a year.
So basically i took ...
1
vote
1
answer
2k
views
Python : Correlation coefficient between two 2D arrays
Let's assume that we have two different 2D np.arrays
a = (np.random.randint(25,30,1512000)).reshape(1050,1440)
b = (np.random.randint(20,25,1512000)).reshape(1050,1440)
So, the dimension of a and b is ...
- 101
0
votes
0
answers
53
views
How to find nearest gridded latitude and longitude given a list of values? [duplicate]
I have a csv of weather stations and their corresponding latitudes and longitudes. I also have a gridded dataset of temperature trends. I want to find and create an array of the temperature trends of ...
- 221
0
votes
1
answer
149
views
Empty strings created by netCDF4.stringtochar()
I am using the netCDF4.stringtochar() function before saving an N-D array of strings before exporting to a netCDF file; however, the function adds empty strings between each character.
Is this a bug ...
- 3,200
1
vote
2
answers
2k
views
How to only select winter months of daily data in xarray?
I have gridded daily temperature data but am only interested in the winter months.
from netCDF4 import Dataset as netcdf_dataset
import numpy as np
import xarray as xr
#open NASA GISS gridded ...
- 221
0
votes
1
answer
3k
views
How to replace values in netcdf file with Nan?
I'm using a NASA GISS netcdf file with gridded monthly temperature values. According to the readme file "Missing data is flagged with a value of 9999.f" I am trying to plot the data but keep ...
- 221
0
votes
0
answers
85
views
How can I convert 3d variable in array with regridded another 1d variable(which is in same array) from netCDF4?
I'm trying to handle netCDF4 file and it has dimensions and variables like this.
<xarray.DataArray '/' ()>
array(<class 'netCDF4._netCDF4.Dataset'>
root group (NETCDF3_CLASSIC data model, ...
0
votes
0
answers
331
views
How to optimize my code for an xarray dataset?
i'm trying to calculate the trend of a large file using xarray module and pymannkendall.
These are the dimensions of my dataset:
dimensions:
time = UNLIMITED ; // (20 currently)
lon = 3502 ;
...
- 61
0
votes
0
answers
143
views
How can I index multiple dimensions at once in a Dask Array?
I am trying to run indexing and assignment on a 3D netCDF array loaded into dask to be used for multiprocessing purposes. At the moment Dask does not appear to support direct 3D indexing and ...
- 163
0
votes
1
answer
1k
views
replacing a netcdf variable and rewrite to a new netcdf file
could anyone please help me with the following issue.
I have a netcdf file with the following dimensions and variables:
dimensions(sizes): y(100), n2(2), x(100),
variables(dimensions): int64 ...
- 97
0
votes
1
answer
188
views
How to overlap (mach) and clip a radar image with a shapefile in python
I was wandering if someone could help me with the following issue. In advance, I really appreciate it.
I have a shapefile and a radar image. But they are in different coordinate systems. The radar ...
- 97
1
vote
1
answer
858
views
How to calculate a yearly sum xarray from a monthly means masked array?
I want to calculate the yearly sum of daily mean temperatures which are above 7 and below 33 degrees celsius. This is a customized calculation for biologically relevant degree days index. I want to do ...
- 39
1
vote
1
answer
146
views
Calculating total depth of a variable
I have calculated the Moist Brunt-Vaisala frequency.
Let's say that the variable is moistb and has a dimension of [height, lat, lon].
I would like to plot the horizontal distribution of the total ...
- 155
0
votes
1
answer
291
views
Can anyone explain why the maximum value of a concatenation between two arrays is so much higher than the max value in either single array?
I have the following two datasets - both from netCDF files:
ds1 = observed_1979_01
ds2 = observed_1979_02
I want to extract the variable labelled 'swvl1' from both datasets, and I do this by:
m = ds1....
- 91
1
vote
1
answer
912
views
Time Series Chart: Groupby seasons (or specfic months) for multiple years in xarray
Thank you for taking interest in my question.
I am hoping to do plot a temperature time series chart specifically between the months January to August from 1981-1999.
Below are my codes and attempts:
...
- 85
0
votes
0
answers
596
views
np.astype not changing array type
I have a many datasets I'm ingesting that include latitude and longitude values. One of these has their lat and lon fields set to float64 whilst everyone else is using float32. So, I'm initially ...
- 1,248
0
votes
0
answers
3k
views
ValueError: shape mismatch: value array of shape could not be broadcast to indexing result of shape
I am trying to run a Python script that will compute statistical math and am receiving an error somewhere in the calculations that is throwing the below error about mismatching shapes (of a 2D array?)....
- 1,131
0
votes
1
answer
124
views
Is there a function for finding the differences between the last time stamp of one netCDF file and the first time stamp of the next netCDF file?
I have a list of netCDF files. I've opened each of these netCDF files in xarray like this:
files = ['file_1.nc', 'file_2.nc', 'file_3.nc', 'file_4.nc']
for file in files:
xarray_object = xr....
