43 questions
1
vote
1
answer
75
views
In python xarray, how do I create and subset lazy variables without loading the whole dataarray?
I am trying to create a python function that opens a remote dataset (in a opendap server) using xarray and automatically creates new variables lazily. A use case would be to calculate magnitude and ...
- 436
0
votes
1
answer
75
views
How to get the right URL for thredds and load ncml data in R?
I'm trying to load some climate data from PAVICS with thredds in R.
library(ncdf4)
library(thredds)
url = "https://pavics.ouranos.ca/twitcher/ows/proxy/thredds/catalog/datasets/simulations/...
- 3,491
1
vote
1
answer
60
views
Writing an ncdf4 file to disk after accessing it through OPeNDAP using R?
I am accessing a ncdf file from a thredds server and subsetting using openDAP, however while I can open and access the file connection, I cannot seem to copy the entire *.nc file to my local disk. Is ...
- 13
0
votes
1
answer
71
views
Downloading from THREDDS Data Server fails when running in parallel, but works when running in series
I am attempting to download/subset a list of gridded parameters from the UCAR THREDDS servers.
Here is how I am setting things up:
import pandas as pd
import xarray as xr
import concurrent.futures
...
- 1,503
0
votes
0
answers
39
views
CFS thredds data don't work in xarray lib
I need to get u,v at 10 meters from CFS thredds from the historic forecast, but, when I open and try to get the variables, the error: FunctionalityNotEnabledError: Functionality not enabled pops.
My ...
0
votes
1
answer
327
views
Error when trying to remotely access MERRA-2
I'm trying to get PM2.5 data from the public NASA data base. Specifically, I'm following all the steps from this guide. I also created the three files just as explained on this requirements website.
...
1
vote
1
answer
181
views
Calculating the size (bytes) of subset large netCDF file from a THREDDS Data Server before reading the file using ncvar_get?
I'm using nc_open to get a DatasetNode from a THREDDS Data Server, and reading a subset of the data in ncvar_get by specifying start and count. Reproducible example below:
library(thredds)
library(...
- 182
0
votes
0
answers
275
views
Efficiently access small chunk of grid on OPeNDAP server with python xarray when grid is saved in a sparse format
I am trying to access gridrad data. This data is stored on an OPeNDAP server, with one URL/file for each timestamp (example). Normally xarray is great for accessing these OPeNDAP grids, because you ...
- 1,503
0
votes
1
answer
124
views
How do I determine which version of THREDDS Data Server I am using?
We are running THREDDS Data Server under Tomcat 7.0.76 and are trying to determine which version of TDS we are running. We're not seeing it in catalina.out or anywhere obvious in the config and were ...
- 117
0
votes
1
answer
36
views
ncss query - polygon instead of lat long box
I'm curious if there is a way to query the NDFD or other layers in the thredds server with a customized polygon instead of a rectangular lat long box? Thanks!
--Chris
- 13
1
vote
0
answers
145
views
Abort download.file in R if file becomes too large
In a shiny app I would like users to be able to download a remote (.nc) file with file size limit of say, 100MB.
For some files inspecting the header works well, as suggested here https://...
- 488
0
votes
1
answer
1k
views
How to download and subset netCDF files from NCEI THREDDS server
I am trying to download and subset the files located here: https://www.ncei.noaa.gov/thredds-ocean/catalog/ncei/archive/data/0129374/daily/catalog.html, but I'm not sure if I'm doing something wrong ...
- 16.9k
0
votes
0
answers
218
views
Running multiple queries on linked servers with sqlalchemy
There are ~150 linked servers and I have to run a query in each linked server. I have to run this as fast as I can.
The way I'm doing today:
def get():
p = Pool(20)
stores_sales = p.starmap(...
- 113
0
votes
1
answer
832
views
Reading in netCDF data from THREDDS fails
I am trying to load in netcdf from a THREDDS server but experience an unrecognizable error after a certain timestep.
def list_dates(start, end):
num_days = (end - start).days
return [start + ...
- 1,061
0
votes
1
answer
179
views
Running several scripts in parallel bash scripts on remote machines
in the following bash script , we want to run in parallel several scripts on remote machines
ssh $server_a /tmp/activate_nodes.bash &
ssh $server_b /tmp/activate_services.bash &
ssh $...
- 2,570
1
vote
1
answer
834
views
Pvlib-Python Datetime Error 'Is not convertible to datetime'
I am trying to access forecast data by using the PVLib-Python command model.get_data(longitude, latitude, start, end) or model.get_processed_data(latitude, longitude, start, end) (where model = GFS()) ...
1
vote
1
answer
268
views
Unable to start thredds under tomcat7
I've seen similar posts, but none that seem to be similar enough to help. I'm running Tomcat 7.0.68 and Java 1.8.0_231 on CentOS 6.10 and so far have been unable to install thredds successfully. I ...
- 117
0
votes
1
answer
301
views
How to format netCDF datasets to be compatible with Thredds OGC services
I have a netcdf dataset produced from the NASA LIS model that i want to be able to show through WMS using a Thredds server. The specific dataset and thredds server can be found at https://tethys.byu....
1
vote
2
answers
1k
views
How do I use pydap library to collect THREDDS data?
I have been trying to use the example get_nomads.py module from Will Holgren which he was nice enough to forward my way. In the code, there is a call to get the THREDDS data as follows:
from pydap....
- 1,832
0
votes
2
answers
388
views
NcML aggregation of remote THREDDS catalog
I want to aggregate all files within a specific directory of a remote THREDDS catalog. These are grib2 files for nam forecast. This is the main list of directories for each month. Here is my ncml file ...
- 207
-1
votes
1
answer
162
views
How to get Contours in ncWMS thredds vertical section for getVerticalSection request
How to get Contours in ncWMS thredds vertical section for getVerticalSection request. I need to obtain some pngs with contours on them for a project using thredds 5.0.0, I am unable to find find any ...
- 59
0
votes
1
answer
204
views
Filtering thredds datasetScan by both directory and file pattern?
In THREDDS server setup, I need to configure datasetScan filter element in catalog.xml so that it takes into account full file path, not just file name or directory name. Say, I have following folder ...
- 918
0
votes
1
answer
430
views
Thredds getCapabilities for ncWMS server
Thredds Data Server TDS provides getCapabilities for a netCDF file.
e.g. http://localhost:8080/thredds/wms/testAll/Filename.nc?service=WMS&version=1.3.0&request=GetCapabilities
Here, the ...
- 195
1
vote
1
answer
1k
views
THREDDS Server NetCDF file via WMS
I have setup TomCat and THREDDS server (loaded war file) and attempted to serve up some *.nc files using via WMS protocol.
I can request the file but all I seem to get back is a black image.
I had ...
- 260
0
votes
2
answers
2k
views
THREDDS Data Server: Deployment fails in Tomcat 8
I´m trying to deploy THREDDS Data Server (TDS) (http://www.unidata.ucar.edu/software/thredds/current/tds/tutorial/GettingStarted.html) as a WAR file in tomcat 8, which should be trivial. But when ...
- 646
0
votes
1
answer
161
views
Files used in THREDDS "best-time-series" aggregation
Is there a way to determine the files, and perhaps time-steps within the files, that are used to construct a "best-time-series" within THREDDS? We currently serve output from a numerical model via ...
- 83
1
vote
1
answer
766
views
Installation of Thredds Data Server
I am trying to deploy Thredds war file in the Apache Tomcat 7 server on Windows Server 2012 R2.
When I started threads data server it is showing error massage like:
Application at context path/...
- 103
1
vote
0
answers
445
views
Can't deploy war with TomEE 1.7.3 (tomcat 7.0.52 with drop-in TomEE war)
I am running tomcat 7.0.52 on a ubuntu 14.04 server with Java SE 1.8 (build 1.8.0_66-b17). Tomcat is running, manager works and I deployed two wars - a Vaadin application and the THREDDS war version 4....
- 31
0
votes
1
answer
652
views
Using Javascript to make parallel server requests THREDDS OPeNDAP
For the following THREDDS OPeNDAP server:
http://data.nodc.noaa.gov/thredds/catalog/ghrsst/L2P/MODIS_T/JPL/2015/294/catalog.html
I would like to note four Attributes of every file in there. The ...
0
votes
2
answers
452
views
THREDDS OPeNDAP speed Matlab
Using the following code in Matlab:
nc_file_list = {'http://data.nodc.noaa.gov/thredds/dodsC/ghrsst/L2P/MODIS_A/JPL/2015/287/20151014-MODIS_A-JPL-L2P-A2015287235500.L2_LAC_GHRSST_D-v01.nc.bz2'};
...
- 145
1
vote
0
answers
217
views
Harvesting from THREDDS using GeoNetwork
I have a THREDDS instance: https://wci.earth2observe.eu/thredds/catalog-earth2observe.html and I am looking for a way to get the data in an ISO-19115 standard format. I have tried many solutions and ...
- 31
0
votes
1
answer
193
views
Show timer in activity
I have created a timer in activity, some times it works perfectly but some time it run faster than the specified execution period.Here is my code
public void StartTimer()
{try
{
// Start ...
- 567
5
votes
2
answers
817
views
Easy, scriptable way to sub-sample unstructured THREDDS data?
I'm trying to get a subset of data from a triangular mesh model that is being served by THREDDS. I'd like to be able to specify a LAT/LON bounding box and just get the data from within that box. The ...
0
votes
1
answer
289
views
Thredds server WMS/Godiva working but WMS link just shows XML
I have the latest Thredds server working with Tomcat 7 and in the threddsConfig.XML file I have enabled WMS. This causes the Godiva link to show at the bottom in the Viewers category.
However, above, ...
- 19
0
votes
1
answer
670
views
Thredds threddsConfig.xml file issue with WMS
I have installed version 4.3 of Thredds (on tomcat 7) and it's working, except with WMS.
If I used an older threddsConfig.xml file (which is pretty much all defaults except for uncommenting the WMS, ...
- 19
3
votes
2
answers
391
views
regExp for matching directories
I have a somewhat complex directory structure for NetCDF files I want to create a THREDDS catalog for.
/data/buoy/A0121/realtime/A0121.met.realtime.nc
/A0121.waves.realtime....
- 3,794
1
vote
0
answers
92
views
Possible to add attributes via NcML to non-cached THREDDS datasets?
For several years on our THREDDS Data Server (TDS) we've been happily using the cache="false" functionality on datasetRoot to tell the TDS not to cache several netcdf files.
<service name="...
- 16.2k
3
votes
5
answers
1k
views
Using NCML to reduce dimensions in netCDF file
I'm trying to use NCML to 'convert' a CF-1.4 file to CF-1.6. Of particular interest is how to 1) remove dimensions and then 2) change the dimensions of variables. For example, below are the top ...
- 83
1
vote
1
answer
271
views
NcML aggregation of remote 3D and 4D variables?
Can NcML be used to aggregate 3D and 4D grids? I wasn't sure since they differ in the number of dimensions. e.g. sea surface height (ssh) and water temperature, where ssh has three dimensions [time, ...
- 123
1
vote
1
answer
344
views
Loading data on Unidata Integrated Data Viewer (IDV) from a THREDDS catalog?
I am trying to load some data to IDV from a THREDDS server via the catalog, but I get error messages such as
Server does not support Content-Length
I can add netcdf data from my local folders, but ...
- 724
1
vote
2
answers
234
views
Is there a hardwired limit to the NcML/THREDDS values element?
Is there a limit to the size or number of entries that can be placed within the values NcML element? I'm attempting to get a set of NetCDF files without lon/lat variables -
but with x/y variables - ...
- 13
1
vote
1
answer
4k
views
Best way for Excel users to access NetCDF files on a THREDDS Data Server?
I have seen the following example to access a netcdf file using excel.
I was wondering if it possible to access a certain part of the data based on specific indices (e.g. a time series of a ...
- 724
2
votes
2
answers
979
views
THREDDS server using Godiva viewer
Anyone knows how I can configure the godiva viewer in order to get a specific colour palette and specific range of values?
This is an example, where the colour palette is the default, but the range ...
- 1,961