All Questions
Tagged with amazon-redshift java
102 questions
0
votes
0
answers
28
views
Host name for LocalStack Redshift for Amazon Redshift JDBC driver
I have setup LocalStack in my windows machine and started with redshift and few other services.
Then I created a cluster and a DB.
>awslocal redshift describe-clusters --cluster-identifier my-...
- 1,396
1
vote
1
answer
59
views
Getting "Caused by: java.lang.ClassNotFoundException: java.sql.SQLException" while trying to connect to amazon redshift driver
Below is the POM dependency
<dependency>
<groupId>com.amazon.redshift</groupId>
<artifactId>redshift-jdbc42</artifactId>
<version&...
- 11
0
votes
0
answers
59
views
geeting error sql udf datashare not enabled while calling redshift procedure from java using Namedparameter
i am trying to call redshift stored procedure which contain multiple temporary table operation . but calling proc getting error "sql udf data share not enabled " redshift exception.
i am ...
- 1
0
votes
0
answers
222
views
Spring boot jpa + Redshift - pagination throws query syntax error
I am using creating a service in Spring boot Graphql, where I am using JPA to contact Redshift database. Regular queries run fine. However when I introduce pagination then I get query syntax error ...
- 8,340
1
vote
0
answers
241
views
Trouble establishing JDBC connection to Amazon Redshift workgroup from Lambda function in VPC
I am facing an issue while trying to establish a JDBC connection to an Amazon Redshift workgroup from a Lambda function that is also deployed within the same VPC. My goal is to execute insert queries ...
- 11
0
votes
0
answers
857
views
Protocol errors with Amazon Redshift database. "ERROR: Invalid protocol sequence 'P' while in PortalSuspended state."
I Am trying to run select * from table_name limit 500000, but it returns this error in java
Protocol errors with Amazon Redshift database. "ERROR: Invalid protocol sequence 'P' while in ...
- 1
0
votes
0
answers
520
views
How to use HikariCP dataSourceClassName with PostgreSQL
I am trying to write an application that uses HikariCP for its pooling.
(BTW, I am using Redshift and its JDBC driver which AFAI checked is almost a copy-paste of pgjdbc).
I can use jdbcUrl and get ...
- 4,928
1
vote
1
answer
1k
views
Redshift JDBC batch insert works slow for multiple rows
I have Java code for inserting into Redshift like this:
String query = "INSERT INTO table (id, name, value) VALUES (?, ?, ?)";
PreparedStatement ps = connection.prepareStatement(query); ...
- 837
1
vote
0
answers
714
views
Getting assertion error when loading data into Redshift concurrently
We've recently switched to Redshift from Postgres for one of our java base app. The application is doing ETL(transforming source data into gzipped CSV, uploading to S3, and then using COPY to load the ...
0
votes
0
answers
64
views
connecting Redshift DB to Java Application using temporary Credentail
I am trying to connect redshift DB to java application using temporary credetial
I am passing Secretkey ,sessionToken and accessID to jdbc url
its giving error like sessionToken is invalid
but when i ...
0
votes
1
answer
1k
views
Creating a Flink DataStream from database query results
In my problem I need to query a database and join the query results with a Kafka data stream in Flink. Currently this is done by storing the query results in a file and then use Flink's readFile ...
- 778
0
votes
0
answers
276
views
Exception while connecting to Redshift Database from android app using jdbc in Java (Android Studio)
I am trying to connect to my Redshift database from my android app using the JDBC driver(JDBC42 driver) provided by AWS. In the process of connecting, I am getting "java.lang....
- 1
0
votes
0
answers
357
views
Unable to execute a stored procedure. An error occurred while calling o91.prepareCall. [JDBC Driver]String index out of range: 33
I am using py4j to execute a stored procedure(Redshift). The JDBC connection is created in AWS glue. Below is the code:
import sys
from awsglue.transforms import *
from awsglue.utils import ...
0
votes
1
answer
494
views
Unable to call Redshift stored procedure using refcursor
I followed this document https://docs.aws.amazon.com/redshift/latest/dg/stored-procedure-result-set.html
But seems to have incomplete, there is no reference to the statement object which is used to ...
- 1,419
0
votes
1
answer
1k
views
Redshift : create table like fails when there is AUTO sort key in source table
I am trying to create a temp table in redshift using
CREATE TABLE tmp_users (LIKE users);
This was working for some time.. but recently I am experiencing failures like below in production.
org....
- 61
-1
votes
1
answer
4k
views
How to use AWS Redshift Data API to execute SQL using Java? [closed]
Can someone help me to get the exact code using Java & Redshift Data API to execute SQL?
We are going to create API in Spring Boot with Java where it will execute the SQL provided in request body. ...
- 59
-2
votes
2
answers
629
views
Insert data into RedShift by using java [closed]
I am a beginner here. I read data from kafka topic using kafka consumer. and that data is in console of the ide or on the local computer.
Now i have to push that data into redshift.
How to do that.
Is ...
- 11
0
votes
0
answers
260
views
JDBC Driver is found when doing UT, but not when the jar is compiled
I was trying to set up a JDBC connection to a redshift database in my java code and created a few UTs that would make calls to that database using the driver.
The problem lies (probably) in my Gradle ...
0
votes
1
answer
672
views
JMETER: JMeter 5.3 java.lang.OutOfMemoryError. During Jmeter execution
I have configured a Testplan using Jmeter shown below in the image and have been using the CLI to run my parallel load tests. MAC USER
I have configured a connection with my AWS RedShift database, ...
- 484
0
votes
1
answer
124
views
Not able to write csv data in aws redshift using spark(sqlContext)
spark version 2.4.1
artifact - spark-*_2.11
Dataset<Row> dataset = sqlContext.read().csv(url);
dataset
.write()
.format("com.databricks.spark.redshift")
.option("url", ...
- 15
2
votes
2
answers
3k
views
AWS Lambda query to Redshift once a day
I am fairly new to AWS ecosystem, especially with the data side.
I have a project that requires me to run a query against a table in Redshift every 24hrs automatically, and perhaps remove a few ...
- 2,647
0
votes
1
answer
2k
views
How to write mybatis mapper for this inserting in temporary table created in between?
I'm trying to write mapper using annotation mybatis for the following sql extract :
CREATE TEMP TABLE XYZ
.
/* Something */
.
CREATE TEMP TABLE ABC
as
Select ptype,pvalue,id,djob
from XYZ
where ...
- 93
1
vote
1
answer
1k
views
AWS Amazon Redshift driver bug with lists in clause IN
I'm using spring NamedParameterJdbcDaoSupport class with Redshift and I have a bug/crash when parameters are of type List/Set and have no element or have a single element (The weir case!).
_This is ...
- 21
0
votes
0
answers
242
views
Redshift dateformat error when uploading date data
I am trying to send json data to kinesis delivery stream and using lambda to process the data and send it to Redshift DB. I am getting this error error when sending the date data:
Invalid timestamp ...
- 13
0
votes
0
answers
235
views
Unable to insert data into Redshift database in Spring Boot application [duplicate]
I have a Spring Boot application where I need to insert an object to a redshift table. No problem with manual insertion of data using a query. But I'm facing a challenge while persisting data to ...
- 3,389
0
votes
1
answer
251
views
Gradle SLF4J Redshift JDBC conflict
I need to update the version of JDBC Redshift driver in gradle and it looks like 1.3X has the following problem:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:~...
- 1,570
0
votes
2
answers
1k
views
Pom.xml and Properties for org.springframework.boot 2.1.8.RELEASE Redshift Cluster V1.0.10393
The first error I'm getting
Invalid operation: SQL command
"drop sequence if exists hibernate_sequence"
not supported
Further down in the exceptions I see the following. I assume I have to ...
- 1,614
0
votes
1
answer
314
views
How do I write multiple values to a redshift table from a pcollection
So I have a template that write a single strings to a redshift table as a record.
public static void main(String[] args) throws Exception {
// Step 1: Create Options
Options options = ...
- 13
2
votes
1
answer
4k
views
Why Connection timed error in redshift connection?
I want to connect aws redshift with aws lambda.
But this gives me error that connection timed out
Class.forName("com.amazon.redshift.jdbc42.Driver");
System.out.println("\n\...
1
vote
2
answers
2k
views
Change the timestamp timezone in result of jdbc query connection
I use Redshift and jdbc.
The timezone of redshift server show timezone; is UTC.
My application(jvm) timezone is +2.
When I execute the query via jdbc select SYSDATE; I got the correct Timestamp with ...
- 632
0
votes
1
answer
2k
views
How to skip "information_schema.sequences" creation while using Hibernate in Java projects
I am using Redshift as a DB for my Java Project with Hibernate support. When the Hibernate starts, it tries to create a sequence, which I do not need.
public class HibernateUtil {
private static ...
- 151
0
votes
1
answer
1k
views
Check redshift cluster status programmatically
Is there any way to ping redshift cluster and get its status. in JAVA, if it is "available" or not. I have to ping a cluster during downtime and check if it is available or not.
- 9
2
votes
1
answer
1k
views
How to connect via jdbc to Amazon RDS with sslmode= require?
I know that Amazon have sslmode=require for ODBC driver. but
I didn't find such mode for jdbc driver. How can I specify sslmode= require for JDBC.
The same question for default and prefer mode.
- 632
1
vote
2
answers
2k
views
"Driver does not support this optional feature" while trying to run quartz jobs with Amazon Redshift
I am trying to run quartz scheduler with Amazon Redshift as the data source. I am getting Driver does not support this optional feature error while storing quartz job.
My quartz datasource ...
- 689
2
votes
1
answer
4k
views
Redshift for local testing
For local query testing can we have an embedded redshift instance possibly in-memory one? I am using java and I don't want to use docker instance running locally.
Even if it is mocked one, I would be ...
- 325
0
votes
2
answers
3k
views
Redshift - select query table name is in double quotes does not work
I am writing unit tests where I am using in-memory H2 DB to test the logic. The actual code runs on redshift since redshift uses psql JDBC driver I chose H2 as a workaround.
But my redshift table ...
- 325
1
vote
1
answer
10k
views
"java.sql.SQLException: [Amazon](500310) Invalid operation: syntax error at end of input" for redshift query using JDBC
I have a redshift query which works fine when I run through SQL Workbench but when I run it through JDBC connection it throw error.
Here is my query.
SELECT
ST.TRANSACTION_ID AS TRANSACTION_ID
...
0
votes
1
answer
2k
views
How to insert entity using Hibernate in Amazon Redshift
I'm trying to make a insert into a table using Hibernate.
The problem is that Redshift doesn't accept sequences, but instead accept the creation of a IDENTITY for a row.
I have the following scenario:...
7
votes
2
answers
5k
views
java.sql.SQLNonTransientException: [Amazon][JDBC](10900) Not all parameters have been populated
I am building a Spring Boot application that is multithreaded utilizing the Spring @EnableAsync and @Async annotations. When I run the application with a single thread (CorePoolSize 1, MaxPoolSize 1) ...
- 525
1
vote
1
answer
1k
views
Redshift connection occasionally drops
I am connecting to redshift cluster using redshift jdbc driver but the connection drops after sometime.I have tcpKeepAlive set to 1 minute but that didn't help either.
Here is the exception i get:
...
- 265
2
votes
1
answer
3k
views
jdbc ResultSet.get<DataType> returns null
So I have the following issue.
I have a Java application that connects to a Redshift-DB via jdbc. Executing a simple SELECT-query yields a ResultSet that is not null (I checked).
Now, when I try to ...
- 253
1
vote
1
answer
2k
views
How to connect with Amazon Redshift from Scala
Is there any scala driver available to connect with redshift or query with redshift?
Or have any idea that how to fetch the data from redshift using:
client = new AmazonRedshiftClient(credentials);
...
- 116
3
votes
1
answer
2k
views
Redshift jdbc connection configuration with Spring boot 2 not working
Here are my dependencies in build.gradle.
compile group: 'org.springframework.boot', name: 'spring-boot-starter-web', version: '2.0.2.RELEASE'
compile group: 'org.springframework.boot', name: 'spring-...
- 127
0
votes
1
answer
85
views
Generate CSV for table with a lot of data
I have a table in postgres database(AWS redshift actually), the data from this table needs to be exported to a CSV after some operations.
As an example,
consider a table Test, with columns A, B, C, D....
- 1
2
votes
0
answers
293
views
Using liquibase in java gradle project
I added liquibase plugin and consumed liquibase-core,redshift-ext dependencies. I created liquibase.properties.
url="jdbc url" //redshift
username=xxxxx
password=xxxxx
driver=com.amazon.redshift....
- 127
0
votes
3
answers
6k
views
Java connect to Amazon Redshift
I am having trying to connect to Amazon Redshift Database with my Java code. I found a code snippet on AWS website that should work. However I am running into problems with implementing the JDBC ...
- 823
1
vote
2
answers
2k
views
Troubleshooting COPY errors on AWS Redshift
Update: If figured this out but am still interested in an explanation. The problem was that I was running the code below while also connected to my Redshift cluster from SqlWorkbenchJ (both running on ...
- 535
0
votes
4
answers
6k
views
Redshift server closes connection after 10 minutes
I have a statement that takes about 20 minutes to run, which is of the form:
create table new_table diststyle key distkey(column1) sortkey(column2)
as (select ....);
When I run it using an SQL IDE ...
- 131
0
votes
1
answer
869
views
Server closes connection when trying to connect Java application to Redshift database
I'm trying to connect a Java application to a Redshift database. When I run DriverManager.getConnection(), it sits for a long time (minutes) and finally throws an exception:
Exception in thread "...
- 805
3
votes
2
answers
15k
views
Include redshift jdbc driver: Unable to instantiate driver class "com.amazon.redshift.jdbc.Driver": java.lang.NoClassDefFoundError
We use Wildfly 10 with a postgres datasource (postgres driver is deliverd in /modules folder) and in addition, a Redshift DB should be attached.
So I followed Configure a JDBC Connection with Apache ...
- 1,545