I have a Java application that writes content to a file in a AWS S3 bucket.
The writer is created with this code
SequenceWriter getBufferedWriter(final ObjectWriter newWriter)
throws IOException {
var key = "myFile.csv";
manager = new StreamTransferManager(bucket, key, client.getClient()).numStreams(1)
.numUploadThreads(1)
.queueCapacity(1)
.partSize(PART_SIZE_MB);
outputStream = manager.getMultiPartOutputStreams()
.get(0);
return newWriter.writeValues(outputStream);
}
Then I write values with
writer.writeValue(myData);
The application works fine, and when it is finished, the data is in the S3 file. However, I'd like to have the content written (and flushed) while the application is running, so if for any reason the application crashes, I still get partial content in the file.
I'd actually like to "flush" it programmatically, so when a certain event occurs in my application, I force the flush
I've tried using writer.flush()
but it didn't achieve what I wanted.
How can I force the content to be written to S3?
partSize
.