All Questions
Tagged with spring-integration-sftp spring-batch
26 questions
0
votes
0
answers
26
views
spring integration SFTP SSH2_DISCONNECT_PROTOCOL_ERROR issue in spring batch
My spring batch application is trying to download file from mainframe via SFTP (JDK 17)
once the download is over, process continues with read,process and write . This happens twice as there are two ...
- 21
0
votes
1
answer
105
views
Merge two huge files in SFTP server Spring Boot
I have 2 huge files >5gb in two different locations in an SFTP server . One way is to download the files and merge them and send it to SFTP again. But as we are using openshift the pods will have ...
0
votes
1
answer
864
views
Spring Integration - SFTP Polling - Inbound Adapter to fetch file name without copying to local directory
Context
We are using a spring integration flow to poll for new files in the input directory. The file name is passed to an existing spring batch job that fetches the file and handles the business ...
0
votes
1
answer
522
views
Spring Integration - Concurrent access to SFTP outbound gateway GET w/ STREAM and accessing the response from Queue Channel
Context
Per the spring docs https://docs.spring.io/spring-integration/docs/current/reference/html/sftp.html#using-the-get-command, the GET command on the SFTP outbound gateway with STREAM option would ...
0
votes
1
answer
956
views
Read multiple folders from sftp - Spring Integration & Batch
we have spring integration with spring batch application where we process a file from a sftp server using a poller. have few questions on spring integration
@Bean
public MessageSource<File> ...
0
votes
1
answer
436
views
Spring Batch Integration idle behavior during polling remote SFTP
I'm using Spring Batch Integration for polling and process a SFTP server, we receive thousands of XMLs files (between 50MB ~ 200MB each file).
Actually I'm running 6 instances of my app to process ...
0
votes
0
answers
238
views
Trigger Spring integration from a Spring batch job
I'm looking for a way to trigger a spring integration that does file download from a spring batch job. Currently I've seen examples of triggering batch job after the integration has received the files....
- 986
0
votes
1
answer
256
views
Using Spring Integration SftpInboundChannel In Conjunction With Spring Batch
Problem description
Dear Stack-Overflow-Community,
I recently came across sftpinbound channel from Spring-Integration-Framework.
Is is possible to start the inbound channel by batch job or not?
My ...
1
vote
1
answer
2k
views
How to upload multiple files using sftp in spring batch
I have 8 files that I want to upload to an FTP server using sftp in the spring batch. I am not able to configure the Tasklet for that can anyone tell me how to do that. Also, the file name should ...
- 17
1
vote
1
answer
954
views
Spring SFTP : Unable to rename .writing file
I am using Spring SFTP integration to transfer the file and many time I got this error. It seems two threads are trying to transfer same file and conflict with each other
2020-08-03 08:31:55,766 INF [...
- 473
0
votes
1
answer
2k
views
spring batch ftp integration timeout error - use spring-boot/spring-batch for ETL
I am new to spring, and looking for a spring batch solution for simple ETL process.
I followed the following link: https://examples.javacodegeeks.com/enterprise-java/spring/batch/spring-batch-etl-job-...
- 593
0
votes
1
answer
211
views
How to write integration tests for spring-batch-integration?
I'm using spring-integration bundled with spring-batch and got stuck trying to write integration tests to test the whole flow, not just single config.
I've created Embedded Sftp Server for this tests ...
1
vote
1
answer
2k
views
Spring Integration is setting SFTP Remote Directory as null
I have created a tasklet to download a file at SFTP server using Spring-Integration-Batch. While running the batch, it seems remote directory is not setting correctly in ...
- 11
0
votes
0
answers
597
views
SFTP @Poller not triggering polling nothing happens
I am trying to set the spring boot application which will pool the csv . i do not see any activity happning in the spring boot application nor on filezilla SFTP server but if I change the same code to ...
- 15
1
vote
1
answer
1k
views
Spring Batch Java configuration - Writing to remote sftp xml file without local file
I have a requirement to write xml file to a sftp server in a Spring Batch application. Currently below code writes xml file to local file system using StaxEventItemWriter. I need to write directly to ...
- 11
2
votes
1
answer
1k
views
how can i synchronize a directory with subdirectories over ftp using spring integration with spring batch?
I've been trying to use SftpInboundFileSynchronizer with a remote directory that contains a subdir, say /myfiles/mysubdir/lefile.txt, I have set a filter to grab the files inside the dirs:
mysync....
- 35
0
votes
0
answers
1k
views
How to launch spring batch job from integration flow?
I have a problem with Spring Integration and Spring Batch. I want to pass csv file from sftp to a batch job then convert information into POJO and pass to output. How can I do this? I have next ...
- 1
0
votes
0
answers
66
views
How can I manipulate files from Spring app
Hi :) My plan is to create and edit some files on ssh server from Spring web app.
Both are on the same server. I read some topics & articles but I'm not sure which one is ok and if it's possible.
...
- 13
0
votes
1
answer
976
views
File upload as parallel process in spring batch with integration
I am trying to upload multiple files to SFTP server using Spring batch with integration. Multiple files are uploaded parallelly using Future with threadPoolExecutorService. But I want to execute the ...
- 59
1
vote
1
answer
3k
views
Spring batch step won't stop by itself after finishing
I am trying to upload multiple files in SFTP server using Spring batch and spring integration. For that I am using ThreadPoolTaskExecutor for parallel processing.
Execute the file upload in each ...
- 185
0
votes
1
answer
530
views
How to use the channel in Spring Integration but Using just java config
I have been reading the Spring Documentation and forums on Internet. I did something very similar to this: sftp with java config in spring
The point is I would like to use the channel to send a ...
- 491
6
votes
0
answers
997
views
Spring-integration-ftp polls file even when my application is stopped
As a follow up to the question -
Same file gets picked up again and again in spring-ftp but with different names
I have the following configuration in my application.xml
<?xml version="1.0" ...
- 1,084
1
vote
2
answers
841
views
Spring batch job start processing file not fully uploaded to the SFTP server
I have a spring-batch job scanning the SFTP server at a given interval. When it finds a new file, it starts the processing.
It works fine for most cases, but there is one case when it doesn't work:
...
- 531
0
votes
1
answer
114
views
Start and stop file inbound or message inbound adaptor in spring batch
I have below requirements,
Receiving file from diff systems and convert into JMS text message and send to IBM MQ
Receiving message from IBM MQ and store into database.
Display status of the messages ...
- 25
1
vote
1
answer
2k
views
Download multiple files using SourcePollingChannelAdapter- start/stop issue?
My requirement is to donwload multiple files from a remote directory in a file server in my spring-batch application. One of the example here suggests the usage of SourcePollingChannelAdapter. In ...
- 303
1
vote
1
answer
5k
views
Spring Batch issue : "Problems occurred while synchronizing remote to local directory"
We PUT file on some SFTP server and then run spring batch.when spring batch run below steps were executed.
pick file from FTP location
process the file.
delete file.
we are facing below exception ...
- 3,005